This document outlines our end-to-end process for building features, from requirements to deployment.

Development Phases

Every feature follows these seven phases:

  1. Requirements (30 min - 2 hours)
  2. Database Design
  3. API Design
  4. Data Flow Diagrams (when needed)
  5. Implementation
  6. Testing
  7. Documentation

Phase 1: Requirements (30 min - 2 hours)

Understand Business Context

Create a one-pager that answers:

What are we building?

  • Feature name and brief description
  • User story: “As a [user], I want to [action] so that [benefit]”

Why are we building it?

  • Business objective
  • User pain point we’re solving
  • Expected impact

Success metrics

  • How will we measure success?
  • What’s the acceptance criteria?

Example One-Pager:

  # Feature: Order Status Webhooks

## What
Implement webhooks to notify merchants when order statuses change.

## Why
Merchants need real-time notifications to integrate with their fulfilment systems.
Currently, they poll our API every 5 minutes, causing unnecessary load.

## User Story
As a merchant, I want to receive webhook notifications when order statuses change,
so that I can update my fulfilment system in real-time without polling.

## Success Metrics
- 80% of merchants enable webhooks within 1 month
- API polling requests decrease by 60%
- Webhook delivery success rate > 95%

## Acceptance Criteria
- [ ] Merchants can register webhook URLs
- [ ] Webhooks fire on order status changes (pending → processing → completed)
- [ ] Webhooks include order ID, old status, new status, timestamp
- [ ] Webhook signature verification for security
- [ ] Retry mechanism (3 attempts with exponential backoff)
- [ ] Webhook logs viewable in merchant dashboard
  

Break Down Features and Prioritise

List all sub-features and prioritise:

  1. Must Have - Core functionality (MVP)
  2. Should Have - Important but not critical
  3. Nice to Have - Can be added later

Example:

  ## Feature Breakdown

### Must Have (MVP)
1. Webhook registration endpoint
2. Webhook firing on status change
3. Signature verification
4. Basic retry logic (3 attempts)

### Should Have
1. Webhook logs in dashboard
2. Test webhook endpoint
3. Webhook disable/enable toggle

### Nice to Have
1. Custom retry configuration
2. Webhook filtering (only specific statuses)
3. Multiple webhook URLs per merchant
  

Phase 2: Database Design

MANDATORY: Use dbdiagram.io

Before writing any code:

  1. Go to dbdiagram.io
  2. Design your schema
  3. Include all relationships, indexes, and constraints
  4. Share with team for review
  5. Iterate based on feedback

Example: Webhook Feature Schema

  Table webhook_subscriptions {
  id integer [primary key, increment]
  user_id integer [not null, ref: > users.id]
  url varchar(500) [not null]
  is_active boolean [default: true]
  secret varchar(255) [not null]
  events varchar(100)[] [note: 'Array of event types to subscribe to']
  created_at timestamp [default: `now()`]
  updated_at timestamp

  indexes {
    user_id
    is_active
    (user_id, is_active) [name: 'idx_user_active']
  }
}

Table webhook_deliveries {
  id integer [primary key, increment]
  subscription_id integer [not null, ref: > webhook_subscriptions.id]
  event_type varchar(100) [not null]
  payload jsonb [not null]
  status varchar(50) [not null, default: 'pending']
  attempts integer [default: 0]
  last_attempt_at timestamp
  response_code integer
  response_body text
  created_at timestamp [default: `now()`]

  indexes {
    subscription_id
    status
    event_type
    created_at
    (subscription_id, status) [name: 'idx_subscription_status']
  }
}

Table orders {
  id integer [primary key]
  user_id integer [ref: > users.id]
  status varchar(50) [not null]
  // ... other fields
}
  

Review Checklist

Before moving to implementation:

  • All foreign keys have indexes
  • Frequently queried fields are indexed
  • Composite indexes for common query patterns
  • Appropriate data types chosen
  • Constraints defined (NOT NULL, UNIQUE, etc.)
  • Timestamps included (created_at, updated_at)
  • Soft delete pattern used where appropriate
  • Database design reviewed by another developer

Phase 3: API Design

Write Endpoints Before Coding

Document all endpoints in docs/api/endpoints.md BEFORE implementation.

Template:

  ### Register Webhook Subscription

**Endpoint**: `POST /api/v1/webhooks/subscriptions`

**Authentication**: Required (JWT)

**Description**: Register a webhook subscription for order status changes.

**Request Body**:
```json
{
  "url": "https://merchant.example.com/webhooks/orders",
  "events": ["order.created", "order.status_changed"],
  "secret": "optional-custom-secret"  // Optional, auto-generated if not provided
}
  

Response (201 Created):

  {
  "id": 123,
  "url": "https://merchant.example.com/webhooks/orders",
  "events": ["order.created", "order.status_changed"],
  "secret": "whsec_a1b2c3d4...",
  "is_active": true,
  "created_at": "2025-01-15T10:30:00Z"
}
  

Errors:

  • 400: Invalid URL format
  • 422: Validation error (missing required fields)
  • 429: Rate limit exceeded

Example cURL:

  curl -X POST https://api.yourapp.com/api/v1/webhooks/subscriptions \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://merchant.example.com/webhooks", "events": ["order.status_changed"]}'
  
  
### Estimate Response Times

For each endpoint, estimate:
- Expected response time (target: <500ms)
- If >500ms, plan async processing

**Example**:

```markdown
| Endpoint | Expected Time | Approach |
|----------|---------------|----------|
| POST /webhooks/subscriptions | <100ms | Synchronous (just DB insert) |
| POST /webhooks/test | 2-5 seconds | Async (queue background job) |
| GET /webhooks/deliveries | <200ms | Synchronous (paginated query) |
  

Phase 4: Data Flow Diagrams (When Needed)

Create data flow diagrams for:

  • External API integrations
  • Webhook processing
  • Complex multi-step workflows
  • Payment processing

Example: Webhook Delivery Flow

sequenceDiagram
    participant O as Order Service
    participant W as Webhook Service
    participant Q as Redis Queue
    participant WK as Worker
    participant M as Merchant Server

    O->>O: Update order status
    O->>W: Trigger webhook event
    W->>W: Find active subscriptions
    W->>Q: Queue webhook delivery jobs
    W->>O: Continue (don't block)

    Q->>WK: Dequeue delivery job
    WK->>WK: Generate signature
    WK->>M: POST webhook payload
    M->>WK: 200 OK
    WK->>W: Mark delivery successful

    Note over WK,M: If delivery fails
    WK->>WK: Retry with exponential backoff
    WK->>M: POST webhook payload (retry)

Phase 5: Implementation

Coding Order

ALWAYS code in this order:

  1. Models - Database tables (SQLAlchemy)
  2. Schemas - Request/response validation (Pydantic)
  3. Services - Business logic
  4. Endpoints - API routes (controllers)
  5. Tests - Unit and integration tests

This order ensures:

  • Dependencies are built before dependents
  • Business logic is separated from HTTP layer
  • Each layer can be tested independently

Example Implementation Sequence

Step 1: Models

  # app/models/webhook_subscription.py
from datetime import datetime
from sqlalchemy import Column, Integer, String, Boolean, DateTime, ForeignKey, ARRAY
from sqlalchemy.orm import relationship

from app.db.base_class import Base


class WebhookSubscription(Base):
    """Webhook subscription for receiving event notifications."""

    __tablename__ = "webhook_subscriptions"

    id = Column(Integer, primary_key=True, index=True)
    user_id = Column(Integer, ForeignKey("users.id"), nullable=False, index=True)
    url = Column(String(500), nullable=False)
    is_active = Column(Boolean, default=True, index=True)
    secret = Column(String(255), nullable=False)
    events = Column(ARRAY(String), nullable=False)
    created_at = Column(DateTime, default=datetime.utcnow, index=True)
    updated_at = Column(DateTime, onupdate=datetime.utcnow)

    # Relationships
    user = relationship("User", back_populates="webhook_subscriptions")
    deliveries = relationship("WebhookDelivery", back_populates="subscription")
  

Step 2: Schemas

  # app/schemas/webhook.py
from typing import List, Optional
from datetime import datetime
from pydantic import BaseModel, HttpUrl, Field


class WebhookSubscriptionCreate(BaseModel):
    """Schema for creating webhook subscription."""

    url: HttpUrl = Field(..., description="Webhook URL to receive notifications")
    events: List[str] = Field(..., description="Event types to subscribe to")
    secret: Optional[str] = Field(None, description="Custom webhook secret (auto-generated if not provided)")


class WebhookSubscriptionResponse(BaseModel):
    """Schema for webhook subscription response."""

    id: int
    url: str
    events: List[str]
    secret: str
    is_active: bool
    created_at: datetime

    class Config:
        from_attributes = True
  

Step 3: Services

  # app/services/webhook_service.py
import secrets
from typing import List
from sqlalchemy.orm import Session

from app.models.webhook_subscription import WebhookSubscription
from app.schemas.webhook import WebhookSubscriptionCreate


class WebhookService:
    """Service for managing webhook subscriptions."""

    @staticmethod
    def create_subscription(
        db: Session,
        user_id: int,
        subscription_in: WebhookSubscriptionCreate
    ) -> WebhookSubscription:
        """
        Create a webhook subscription.

        Args:
            db: Database session
            user_id: ID of user creating subscription
            subscription_in: Subscription data

        Returns:
            WebhookSubscription: Created subscription
        """
        # Generate secret if not provided
        secret = subscription_in.secret or f"whsec_{secrets.token_urlsafe(32)}"

        subscription = WebhookSubscription(
            user_id=user_id,
            url=str(subscription_in.url),
            events=subscription_in.events,
            secret=secret,
            is_active=True,
        )

        db.add(subscription)
        db.commit()
        db.refresh(subscription)

        return subscription
  

Step 4: Endpoints

  # app/api/v1/endpoints/webhooks.py
from fastapi import APIRouter, Depends, status
from sqlalchemy.orm import Session

from app.api import deps
from app.schemas.webhook import WebhookSubscriptionCreate, WebhookSubscriptionResponse
from app.services.webhook_service import WebhookService

router = APIRouter()


@router.post("/subscriptions", response_model=WebhookSubscriptionResponse, status_code=status.HTTP_201_CREATED)
def create_webhook_subscription(
    subscription_in: WebhookSubscriptionCreate,
    db: Session = Depends(deps.get_db),
    current_user: User = Depends(deps.get_current_user),
) -> WebhookSubscriptionResponse:
    """
    Create a webhook subscription.

    Args:
        subscription_in: Subscription creation data
        db: Database session
        current_user: Authenticated user

    Returns:
        WebhookSubscriptionResponse: Created subscription
    """
    subscription = WebhookService.create_subscription(
        db,
        current_user.id,
        subscription_in
    )
    return subscription
  

Step 5: Tests

  # tests/test_api/test_webhooks.py
def test_create_webhook_subscription(client, auth_headers):
    """Test creating webhook subscription."""
    response = client.post(
        "/api/v1/webhooks/subscriptions",
        json={
            "url": "https://example.com/webhook",
            "events": ["order.status_changed"]
        },
        headers=auth_headers
    )

    assert response.status_code == 201
    data = response.json()
    assert data["url"] == "https://example.com/webhook"
    assert "whsec_" in data["secret"]
    assert data["is_active"] is True
  

Write Docstrings for All Functions

Every function MUST have a Google-style docstring.

See ../02-standards/code-standards.md for examples.

Commit Frequently with Clear Messages

Commit every logical change:

  • After completing a model
  • After completing a service method
  • After completing an endpoint
  • After tests pass

See git-workflow.md for commit message format.

Example commits during implementation:

  feat(webhooks): add webhook subscription model
feat(webhooks): add webhook subscription schemas
feat(webhooks): add webhook service create method
feat(webhooks): add create subscription endpoint
test(webhooks): add subscription creation tests
  

Phase 6: Testing

Write Tests As You Code

Don’t leave testing until the end.

For each service method or endpoint:

  1. Write the code
  2. Write the test
  3. Run the test
  4. Move to next method

Create Postman Collection

For manual testing and documentation:

  1. Create collection: “Webhooks API”
  2. Add all endpoints
  3. Add environment variables ({{base_url}}, {{token}})
  4. Write test scripts for each request
  5. Share collection with team

See ../06-tooling/api-testing.md for detailed guide.

Manual Testing Checklist

Before considering feature complete:

  • Happy path works end-to-end
  • Error cases handled correctly
  • Edge cases tested (empty data, invalid data)
  • Authentication works
  • Authorisation works (users can’t access others’ data)
  • Response times are acceptable (<500ms)
  • Database queries are efficient (check with EXPLAIN)
  • Background jobs process correctly
  • Webhooks deliver successfully
  • Error messages are helpful

Phase 7: Documentation

Update API Documentation

MANDATORY: Update docs/api/endpoints.md with:

  • All new endpoints
  • Request/response examples
  • Error responses
  • cURL examples

Update README (if needed)

Update if you:

  • Added new environment variables
  • Changed setup process
  • Added new dependencies
  • Modified deployment process

Ensure All Docstrings Are Complete

Review all code and verify:

  • Every function has a docstring
  • Every class has a docstring
  • Complex logic has inline comments explaining WHY

Complete Example: Building Order Status Webhooks

Week 1, Day 1: Requirements & Design (2-4 hours)

  1. ✅ Write one-pager
  2. ✅ Design database schema on dbdiagram.io
  3. ✅ Document API endpoints in docs/api/endpoints.md
  4. ✅ Create data flow diagram
  5. ✅ Get team review and approval

Week 1, Day 2-3: Implementation (8-12 hours)

  1. ✅ Create migration for webhook tables
  2. ✅ Implement models
  3. ✅ Implement schemas
  4. ✅ Implement services (create, list, delete subscriptions)
  5. ✅ Implement endpoints
  6. ✅ Implement webhook delivery worker
  7. ✅ Write unit tests
  8. ✅ Write integration tests
  9. ✅ Commit frequently with clear messages

Week 1, Day 4: Testing (2-4 hours)

  1. ✅ Create Postman collection
  2. ✅ Manual testing all scenarios
  3. ✅ Load testing webhook delivery
  4. ✅ Fix bugs found during testing

Week 1, Day 5: Documentation & PR (2-3 hours)

  1. ✅ Update API documentation
  2. ✅ Ensure all docstrings complete
  3. ✅ Create pull request
  4. ✅ Address code review feedback
  5. ✅ Merge and deploy

Total time: ~1 week for a medium-sized feature


Tips for Success

Start Small

Build the simplest version first:

  • Basic CRUD before advanced features
  • Synchronous before async
  • Manual before automated

Get Feedback Early

Don’t wait until it’s “perfect”:

  • Share database design after 30 minutes
  • Show working prototype after 2 hours
  • Demo to team after 1 day

Focus on One Thing

Resist scope creep:

  • Build what’s in the spec
  • Note “nice to haves” for future
  • Don’t gold-plate

Test Continuously

Catch bugs early:

  • Run tests after every change
  • Manual test frequently
  • Don’t assume it works

Next Steps