Feature Development Workflow
This document outlines our end-to-end process for building features, from requirements to deployment.
Development Phases
Every feature follows these seven phases:
- Requirements (30 min - 2 hours)
- Database Design
- API Design
- Data Flow Diagrams (when needed)
- Implementation
- Testing
- Documentation
Phase 1: Requirements (30 min - 2 hours)
Understand Business Context
Create a one-pager that answers:
What are we building?
- Feature name and brief description
- User story: “As a [user], I want to [action] so that [benefit]”
Why are we building it?
- Business objective
- User pain point we’re solving
- Expected impact
Success metrics
- How will we measure success?
- What’s the acceptance criteria?
Example One-Pager:
# Feature: Order Status Webhooks
## What
Implement webhooks to notify merchants when order statuses change.
## Why
Merchants need real-time notifications to integrate with their fulfilment systems.
Currently, they poll our API every 5 minutes, causing unnecessary load.
## User Story
As a merchant, I want to receive webhook notifications when order statuses change,
so that I can update my fulfilment system in real-time without polling.
## Success Metrics
- 80% of merchants enable webhooks within 1 month
- API polling requests decrease by 60%
- Webhook delivery success rate > 95%
## Acceptance Criteria
- [ ] Merchants can register webhook URLs
- [ ] Webhooks fire on order status changes (pending → processing → completed)
- [ ] Webhooks include order ID, old status, new status, timestamp
- [ ] Webhook signature verification for security
- [ ] Retry mechanism (3 attempts with exponential backoff)
- [ ] Webhook logs viewable in merchant dashboard
Break Down Features and Prioritise
List all sub-features and prioritise:
- Must Have - Core functionality (MVP)
- Should Have - Important but not critical
- Nice to Have - Can be added later
Example:
## Feature Breakdown
### Must Have (MVP)
1. Webhook registration endpoint
2. Webhook firing on status change
3. Signature verification
4. Basic retry logic (3 attempts)
### Should Have
1. Webhook logs in dashboard
2. Test webhook endpoint
3. Webhook disable/enable toggle
### Nice to Have
1. Custom retry configuration
2. Webhook filtering (only specific statuses)
3. Multiple webhook URLs per merchant
Phase 2: Database Design
MANDATORY: Use dbdiagram.io
Before writing any code:
- Go to dbdiagram.io
- Design your schema
- Include all relationships, indexes, and constraints
- Share with team for review
- Iterate based on feedback
Example: Webhook Feature Schema
Table webhook_subscriptions {
id integer [primary key, increment]
user_id integer [not null, ref: > users.id]
url varchar(500) [not null]
is_active boolean [default: true]
secret varchar(255) [not null]
events varchar(100)[] [note: 'Array of event types to subscribe to']
created_at timestamp [default: `now()`]
updated_at timestamp
indexes {
user_id
is_active
(user_id, is_active) [name: 'idx_user_active']
}
}
Table webhook_deliveries {
id integer [primary key, increment]
subscription_id integer [not null, ref: > webhook_subscriptions.id]
event_type varchar(100) [not null]
payload jsonb [not null]
status varchar(50) [not null, default: 'pending']
attempts integer [default: 0]
last_attempt_at timestamp
response_code integer
response_body text
created_at timestamp [default: `now()`]
indexes {
subscription_id
status
event_type
created_at
(subscription_id, status) [name: 'idx_subscription_status']
}
}
Table orders {
id integer [primary key]
user_id integer [ref: > users.id]
status varchar(50) [not null]
// ... other fields
}
Review Checklist
Before moving to implementation:
- All foreign keys have indexes
- Frequently queried fields are indexed
- Composite indexes for common query patterns
- Appropriate data types chosen
- Constraints defined (NOT NULL, UNIQUE, etc.)
- Timestamps included (created_at, updated_at)
- Soft delete pattern used where appropriate
- Database design reviewed by another developer
Phase 3: API Design
Write Endpoints Before Coding
Document all endpoints in docs/api/endpoints.md BEFORE implementation.
Template:
### Register Webhook Subscription
**Endpoint**: `POST /api/v1/webhooks/subscriptions`
**Authentication**: Required (JWT)
**Description**: Register a webhook subscription for order status changes.
**Request Body**:
```json
{
"url": "https://merchant.example.com/webhooks/orders",
"events": ["order.created", "order.status_changed"],
"secret": "optional-custom-secret" // Optional, auto-generated if not provided
}
Response (201 Created):
{
"id": 123,
"url": "https://merchant.example.com/webhooks/orders",
"events": ["order.created", "order.status_changed"],
"secret": "whsec_a1b2c3d4...",
"is_active": true,
"created_at": "2025-01-15T10:30:00Z"
}
Errors:
- 400: Invalid URL format
- 422: Validation error (missing required fields)
- 429: Rate limit exceeded
Example cURL:
curl -X POST https://api.yourapp.com/api/v1/webhooks/subscriptions \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{"url": "https://merchant.example.com/webhooks", "events": ["order.status_changed"]}'
### Estimate Response Times
For each endpoint, estimate:
- Expected response time (target: <500ms)
- If >500ms, plan async processing
**Example**:
```markdown
| Endpoint | Expected Time | Approach |
|----------|---------------|----------|
| POST /webhooks/subscriptions | <100ms | Synchronous (just DB insert) |
| POST /webhooks/test | 2-5 seconds | Async (queue background job) |
| GET /webhooks/deliveries | <200ms | Synchronous (paginated query) |
Phase 4: Data Flow Diagrams (When Needed)
Create data flow diagrams for:
- External API integrations
- Webhook processing
- Complex multi-step workflows
- Payment processing
Example: Webhook Delivery Flow
sequenceDiagram
participant O as Order Service
participant W as Webhook Service
participant Q as Redis Queue
participant WK as Worker
participant M as Merchant Server
O->>O: Update order status
O->>W: Trigger webhook event
W->>W: Find active subscriptions
W->>Q: Queue webhook delivery jobs
W->>O: Continue (don't block)
Q->>WK: Dequeue delivery job
WK->>WK: Generate signature
WK->>M: POST webhook payload
M->>WK: 200 OK
WK->>W: Mark delivery successful
Note over WK,M: If delivery fails
WK->>WK: Retry with exponential backoff
WK->>M: POST webhook payload (retry)
Phase 5: Implementation
Coding Order
ALWAYS code in this order:
- Models - Database tables (SQLAlchemy)
- Schemas - Request/response validation (Pydantic)
- Services - Business logic
- Endpoints - API routes (controllers)
- Tests - Unit and integration tests
This order ensures:
- Dependencies are built before dependents
- Business logic is separated from HTTP layer
- Each layer can be tested independently
Example Implementation Sequence
Step 1: Models
# app/models/webhook_subscription.py
from datetime import datetime
from sqlalchemy import Column, Integer, String, Boolean, DateTime, ForeignKey, ARRAY
from sqlalchemy.orm import relationship
from app.db.base_class import Base
class WebhookSubscription(Base):
"""Webhook subscription for receiving event notifications."""
__tablename__ = "webhook_subscriptions"
id = Column(Integer, primary_key=True, index=True)
user_id = Column(Integer, ForeignKey("users.id"), nullable=False, index=True)
url = Column(String(500), nullable=False)
is_active = Column(Boolean, default=True, index=True)
secret = Column(String(255), nullable=False)
events = Column(ARRAY(String), nullable=False)
created_at = Column(DateTime, default=datetime.utcnow, index=True)
updated_at = Column(DateTime, onupdate=datetime.utcnow)
# Relationships
user = relationship("User", back_populates="webhook_subscriptions")
deliveries = relationship("WebhookDelivery", back_populates="subscription")
Step 2: Schemas
# app/schemas/webhook.py
from typing import List, Optional
from datetime import datetime
from pydantic import BaseModel, HttpUrl, Field
class WebhookSubscriptionCreate(BaseModel):
"""Schema for creating webhook subscription."""
url: HttpUrl = Field(..., description="Webhook URL to receive notifications")
events: List[str] = Field(..., description="Event types to subscribe to")
secret: Optional[str] = Field(None, description="Custom webhook secret (auto-generated if not provided)")
class WebhookSubscriptionResponse(BaseModel):
"""Schema for webhook subscription response."""
id: int
url: str
events: List[str]
secret: str
is_active: bool
created_at: datetime
class Config:
from_attributes = True
Step 3: Services
# app/services/webhook_service.py
import secrets
from typing import List
from sqlalchemy.orm import Session
from app.models.webhook_subscription import WebhookSubscription
from app.schemas.webhook import WebhookSubscriptionCreate
class WebhookService:
"""Service for managing webhook subscriptions."""
@staticmethod
def create_subscription(
db: Session,
user_id: int,
subscription_in: WebhookSubscriptionCreate
) -> WebhookSubscription:
"""
Create a webhook subscription.
Args:
db: Database session
user_id: ID of user creating subscription
subscription_in: Subscription data
Returns:
WebhookSubscription: Created subscription
"""
# Generate secret if not provided
secret = subscription_in.secret or f"whsec_{secrets.token_urlsafe(32)}"
subscription = WebhookSubscription(
user_id=user_id,
url=str(subscription_in.url),
events=subscription_in.events,
secret=secret,
is_active=True,
)
db.add(subscription)
db.commit()
db.refresh(subscription)
return subscription
Step 4: Endpoints
# app/api/v1/endpoints/webhooks.py
from fastapi import APIRouter, Depends, status
from sqlalchemy.orm import Session
from app.api import deps
from app.schemas.webhook import WebhookSubscriptionCreate, WebhookSubscriptionResponse
from app.services.webhook_service import WebhookService
router = APIRouter()
@router.post("/subscriptions", response_model=WebhookSubscriptionResponse, status_code=status.HTTP_201_CREATED)
def create_webhook_subscription(
subscription_in: WebhookSubscriptionCreate,
db: Session = Depends(deps.get_db),
current_user: User = Depends(deps.get_current_user),
) -> WebhookSubscriptionResponse:
"""
Create a webhook subscription.
Args:
subscription_in: Subscription creation data
db: Database session
current_user: Authenticated user
Returns:
WebhookSubscriptionResponse: Created subscription
"""
subscription = WebhookService.create_subscription(
db,
current_user.id,
subscription_in
)
return subscription
Step 5: Tests
# tests/test_api/test_webhooks.py
def test_create_webhook_subscription(client, auth_headers):
"""Test creating webhook subscription."""
response = client.post(
"/api/v1/webhooks/subscriptions",
json={
"url": "https://example.com/webhook",
"events": ["order.status_changed"]
},
headers=auth_headers
)
assert response.status_code == 201
data = response.json()
assert data["url"] == "https://example.com/webhook"
assert "whsec_" in data["secret"]
assert data["is_active"] is True
Write Docstrings for All Functions
Every function MUST have a Google-style docstring.
See ../02-standards/code-standards.md for examples.
Commit Frequently with Clear Messages
Commit every logical change:
- After completing a model
- After completing a service method
- After completing an endpoint
- After tests pass
See git-workflow.md for commit message format.
Example commits during implementation:
feat(webhooks): add webhook subscription model
feat(webhooks): add webhook subscription schemas
feat(webhooks): add webhook service create method
feat(webhooks): add create subscription endpoint
test(webhooks): add subscription creation tests
Phase 6: Testing
Write Tests As You Code
Don’t leave testing until the end.
For each service method or endpoint:
- Write the code
- Write the test
- Run the test
- Move to next method
Create Postman Collection
For manual testing and documentation:
- Create collection: “Webhooks API”
- Add all endpoints
- Add environment variables ({{base_url}}, {{token}})
- Write test scripts for each request
- Share collection with team
See ../06-tooling/api-testing.md for detailed guide.
Manual Testing Checklist
Before considering feature complete:
- Happy path works end-to-end
- Error cases handled correctly
- Edge cases tested (empty data, invalid data)
- Authentication works
- Authorisation works (users can’t access others’ data)
- Response times are acceptable (<500ms)
- Database queries are efficient (check with EXPLAIN)
- Background jobs process correctly
- Webhooks deliver successfully
- Error messages are helpful
Phase 7: Documentation
Update API Documentation
MANDATORY: Update docs/api/endpoints.md with:
- All new endpoints
- Request/response examples
- Error responses
- cURL examples
Update README (if needed)
Update if you:
- Added new environment variables
- Changed setup process
- Added new dependencies
- Modified deployment process
Ensure All Docstrings Are Complete
Review all code and verify:
- Every function has a docstring
- Every class has a docstring
- Complex logic has inline comments explaining WHY
Complete Example: Building Order Status Webhooks
Week 1, Day 1: Requirements & Design (2-4 hours)
- ✅ Write one-pager
- ✅ Design database schema on dbdiagram.io
- ✅ Document API endpoints in
docs/api/endpoints.md - ✅ Create data flow diagram
- ✅ Get team review and approval
Week 1, Day 2-3: Implementation (8-12 hours)
- ✅ Create migration for webhook tables
- ✅ Implement models
- ✅ Implement schemas
- ✅ Implement services (create, list, delete subscriptions)
- ✅ Implement endpoints
- ✅ Implement webhook delivery worker
- ✅ Write unit tests
- ✅ Write integration tests
- ✅ Commit frequently with clear messages
Week 1, Day 4: Testing (2-4 hours)
- ✅ Create Postman collection
- ✅ Manual testing all scenarios
- ✅ Load testing webhook delivery
- ✅ Fix bugs found during testing
Week 1, Day 5: Documentation & PR (2-3 hours)
- ✅ Update API documentation
- ✅ Ensure all docstrings complete
- ✅ Create pull request
- ✅ Address code review feedback
- ✅ Merge and deploy
Total time: ~1 week for a medium-sized feature
Tips for Success
Start Small
Build the simplest version first:
- Basic CRUD before advanced features
- Synchronous before async
- Manual before automated
Get Feedback Early
Don’t wait until it’s “perfect”:
- Share database design after 30 minutes
- Show working prototype after 2 hours
- Demo to team after 1 day
Focus on One Thing
Resist scope creep:
- Build what’s in the spec
- Note “nice to haves” for future
- Don’t gold-plate
Test Continuously
Catch bugs early:
- Run tests after every change
- Manual test frequently
- Don’t assume it works
Next Steps
- Review
pr-process.mdfor submitting your work - Check
git-workflow.mdfor commit standards - See
deployment.mdfor releasing your feature