SwadhyAI Notes

All System Design notes

System Design Patterns

CQRS Pattern

A creator posts once; millions of followers need a feed that loads in milliseconds. The write must land safely once. Then the read path often serves a precomputed list, not a join of every table on every scroll.

(Command Query Responsibility Segregation) is the pattern that separates those jobs: a updates the source of truth; a reads a shaped for the screen. When reads dwarf writes, that split is how timelines stay fast.

Timeline journey

1

Validate rules and commit the write model once.

2

Update feed caches, search indexes, and analytics views.

3

Serve denormalized lists in milliseconds, not multi-table joins.

4

Use optimistic UI or read-your-writes for the author.

What actually happens

The timeline story above is the default tension whenever write shape and read shape diverge. A post or order must commit correctly once, while feeds and history must load in milliseconds for millions of readers. One normalized schema for everything looks tidy until a celebrity timeline fans out into multi-join work under read amplification while writers wait on the same hot tables.

splits the path. The stays normalized so orders, posts, and inventory rules stay correct. After a successful , events or a change stream feed one or more : order history, product search, analytics, each a tuned for that .

That is why a POST can return 201 while an immediate GET still shows old data: between write and read paths. Products hide that lag with , , polling, or push. Indexes tuned for writes hurt screen queries and vice versa; dual-writing the DB and cache without one atomic boundary creates gaps; pretending the read model is always fresh after a 201 lies to the author.

can store every change in an , but simple CQRS often writes a relational row and projects from a durable log. No full event store is required.

How to choose CQRS

Once you see that write truth and read speed pull in different directions, three questions decide whether the split is worth the ops cost.

1. Do read and write patterns diverge? 100:1 feeds, search, and dashboards usually yes. Simple CRUD with similar queries usually no.

2. Can the product tolerate projection lag? If authors must see their write instantly, plan or . Do not pretend the is always fresh.

3. Can you rebuild? Every needs replay, lag alerts, and a rebuild runbook, or the dual model becomes a permanent mystery.

Simple rule of thumb

Use when screens need denormalized speed and the write path needs different invariants. Skip it for blogs and todo lists with no scaling pressure. In simple terms: write for truth, read for screens, and design the lag into the UX.

When it is essential

  • 100:1+ read/write ratio (social feeds, e-commerce catalogs) so reads must scale alone.
  • Different query shapes: full-text search, analytics, and timelines from one write stream.
  • Multiple read models (cache, search, analytics) from the same durable change log.
  • Event-driven architecture already in place, so projections are a natural next consumer.

When it is overkill

  • Simple CRUD apps (blog, todo list) where one schema already answers every screen.
  • Read/write ratio near 1:1, so separate models buy little throughput.
  • Same query patterns for reads and writes, so denormalized views stay unused.
  • Small team with a limited complexity budget and no clear scaling pain.

From problem to mechanism

Here is the celebrity-timeline failure slowed down so you can see where the naive shared schema breaks and what the split changes.

Problem. A post must commit correctly once, while feeds and history must load in milliseconds for millions of readers. Write shape and read shape diverge.

Naive system. One normalized schema for everything: every feed scroll joins posts, follows, likes, and media. A celebrity timeline query fans out into multi-join work under read amplification while writers wait on the same hot tables.

Break it. Indexes tuned for writes hurt screen queries and vice versa. Dual-writing the DB and cache without one atomic boundary creates gaps. Pretending the is always fresh after a 201 lies to the author.

Mechanism. keeps a as the source of truth for business rules, and builds screen-shaped for . A successful must not depend on a lagging projection for correctness. The user-visible consistency contract is explicit: the author gets within a budget; followers may see lag of T milliseconds to seconds. Projections are rebuildable; they are not a second write authority.

Real systems already use this split. A social timeline stores the post once, then fan-out writes the post ID into each follower's timeline cache so a GET is a precomputed list (celebrities with 100M+ followers often mix fan-out on read for them with fan-out on write for everyone else). A LinkedIn-style feed keeps activity storage for durability and a relevance-scored precomputed feed per member so writes optimize for durability while reads optimize for latency across hundreds of millions of members.

Read models, lag UX, and numbers to know

The mechanism above names ; this is what they look like in practice, how the UI hides , and the lag budgets you should be able to compute in an interview.

Read model projections

A projection is a stored answer optimized for a particular query. An order screen might combine order status, item names, and a payment summary in one row even though the write model stores them separately. Events or captured changes update that row. The read model duplicates facts for speed, so document which write-side authority owns each fact and how the projection handles deletion, duplicate updates, and out-of-order versions.

Commands must validate business invariants against authoritative state. A projection that says “one seat available” may be milliseconds behind; it is useful for display but cannot authorize the sale. The seat service’s transaction or conditional update decides whether the reservation succeeds, and the resulting event eventually updates the display model.

The stays normalized. Each projection below is built from the same events: add a new screen by adding a consumer, without a write-side schema migration.

Write model (normalized)

plaintext
orders:     {id, user_id, status, total, created_at}
order_items: {order_id, product_id, qty, price}
products:    {id, name, category, price}
users:       {id, name, email}

→ 4 JOINs needed for "order history with product details"

Projection 1: Order history view

javascript
order_history: {
  order_id, user_name, user_email,
  items: [{product_name, qty, price}],
  total, status, created_at
}
→ Single read, zero JOINs, sub-millisecond response

Projection 2: Product analytics view

javascript
product_stats: {
  product_id, product_name,
  total_orders, total_revenue,
  avg_rating, review_count,
  last_30_days_sales
}
→ Pre-computed, updated every event

Projection 3: Search view

javascript
product_search: {
  product_id, name, description, category,
  price, rating, in_stock, tags
}
→ Full-text search, faceted filtering

Handling lag in the UI

After the projections exist, the gap users notice is still the same: the succeeded, but the has not caught up yet.

The gap users notice

plaintext
POST /orders → 201 Created (write model updated)
GET /orders  → still shows old data (read model not updated yet)
Lag: typically 10ms-2s depending on event pipeline

1. Return write result in the command response

javascript
POST /orders → { order_id: 456, status: "created", items: [...] }
→ Client uses this response directly for UI update
→ No need to query read model immediately

2. Optimistic UI

Optimistic UI displays the intended change before the command has finished. For a like button, show the tentative state and remember the prior value; if the server rejects it, reconcile the view and explain the failure. This improves perceived responsiveness but does not establish a database consistency guarantee. Avoid presenting a speculative payment or reservation as final success when its authoritative command has not completed.

3. Read-your-writes consistency

plaintext
Response header: X-Write-Timestamp: 1709300000
Subsequent read: if read_model.timestamp < X-Write-Timestamp
  → Read from write DB directly (bypass read model)
  → Or wait briefly for read model to catch up

This is : the author sees their own update even while followers still lag.

4. Polling with version

plaintext
Client: GET /orders?after_version=123
Server: if read_model.version >= 123 → return data
        else → return 304 (not ready) or stale data
Client polls every 500ms until version matches

5. WebSocket push

When a projection applies the command’s event, it can notify a connected client that a particular version is visible. Include the entity version or command identifier so an older update cannot overwrite a newer local view. End-to-end delay includes the transaction, outbox relay, queue, projection work, and delivery, so an under-100ms result is a workload target rather than an automatic property of WebSockets.

Numbers to know

Lag budget (worked)

plaintext
25_000 writes/sec × 4 read models
  → 100_000 projection updates/sec required

Consumers apply 80_000/sec
  → backlog grows 20_000/sec
  → 60s of writes → 1.2M delayed updates

Alert on age of oldest unprojected event,
not only queue depth.

Replay math

Rebuilding 100M events at 50K events/sec takes about 33 minutes before index writes and validation. Rebuilding 1B events takes about 5.6 hours, so keep or per-tenant checkpoints. At timeline scale, 500M users × 800 posts is 400B timeline entries: full fan-out rebuilds take hours even with massive parallelism, so hot-fix by rebuilding only affected users' timelines.

StepMechanicSafety check
Schema versionTreat new event fields as additive; consumers tolerate old and new versions.Contract tests replay old events into new projection code.
Shadow projectionBuild a new read model beside the existing one.Dual-read sampled users and compare totals, counts, and missing rows.
CutoverSwitch reads once lag is zero and diff rate is inside tolerance.Keep old projection hot until rollback window closes.

Page on projection lag SLO breach, consumer error rate, and shadow-diff rate. Keep a rebuild playbook before you need it (see Event store and rebuild below).

Event store, rebuilds, and CQRS without event sourcing

The numbers above only stay honest if you can rebuild. is one way to keep that history; most teams start with a simpler change log and only add a full when audit and replay are the product.

Event store implementation

Event store schema

javascript
events: {
  event_id:      UUID,
  aggregate_type: "Order",
  aggregate_id:   "order_123",
  event_type:     "OrderCreated",
  event_data:     { user_id: 456, items: [...], total: 100.00 },
  metadata:       { user_ip: "...", correlation_id: "..." },
  version:        1,  // per-aggregate version for optimistic locking
  created_at:     "2024-03-01T12:00:00Z"
}

Example event stream

plaintext
v1: OrderCreated    { items: [{...}], total: 100 }
v2: PaymentReceived { payment_id: "pay_789", amount: 100 }
v3: ItemAdded       { product_id: 42, qty: 2 }
v4: OrderShipped    { tracking: "1Z999..." }

Rebuilding state & optimistic concurrency

javascript
events = SELECT * FROM events WHERE aggregate_id = 'order_123' ORDER BY version
state = events.reduce((state, event) => apply(state, event), initialState)

-- Optimistic concurrency:
INSERT INTO events (aggregate_id, version, ...) VALUES ('order_123', 5, ...)
→ If version 5 already exists → UNIQUE violation → retry with latest state

Tools

  • EventStoreDB: purpose-built store with subscriptions and built-in projections.
  • Durable log: append-only stream that doubles as a natural event bus.
  • Relational database: works with a careful schema plus version indexes.
  • Managed key-value store: partition key equals aggregate_id, sort key equals version.

Rebuilding read models

Build a replacement projection under a new version while the existing version continues serving users. Replay a consistent snapshot plus changes from a known position, or the retained event history when available, then catch up to the live stream. Compare counts and representative query results before moving reads to the replacement. A snapshot without a matching change-log position can leave a gap or replay changes twice.

Replay consumers should update projection state without resending emails, charging cards, or issuing other historical side effects. Persist progress with projection writes and make handlers idempotent so restarting the rebuild repeats safe work. Keep the old projection long enough to roll back a bad transformation.

Rebuild when projection logic is wrong, a new is needed, or the read schema changes. Use so replay does not start at event zero (see Numbers to know above for wall-clock cost).

When to rebuild

  • Bug in projection logic: fix the consumer, then replay history into a clean store.
  • New read model needed: add a consumer and backfill from the existing event stream.
  • Schema change in the read model: shadow-build the new shape, then cut over safely.

Rebuild process

plaintext
1. Create new read model table/index (v2)
2. Start consumer from beginning of event stream
3. Process all historical events → build v2
4. Once caught up: switch traffic from v1 → v2
5. Delete v1

Snapshot strategy

javascript
Every 1000 events per aggregate, save snapshot:
  { aggregate_id: "order_123", version: 1000, state: {...} }
Rebuild: load snapshot + replay events after version 1000
Reduces replay from millions of events to < 1000

Simple CQRS (most common)

does not require . Many production systems write a relational row, publish to a durable log, and let consumers update cache and search.

plaintext
Write: relational database (normalized)
Read:  in-memory store (denormalized cache) + Elasticsearch (search)

Write path:
  API → Validate → Write to relational database → Publish event to durable log

Read path:
  durable log consumer → Update in-memory store cache
  durable log consumer → Update Elasticsearch index
  API → Read from in-memory store or Elasticsearch

No event store needed! Events propagate changes to read models.

Failure timeline, and what still breaks after the fix

Putting the pieces together, here is how a successful POST still looks broken to the author, followed by the ways an improved design can still fail, and what you pay for the split.

Timeline: POST 201, GET still stale

t=0      Client POST /posts → write model commits; API returns 201 + post body
t=50ms   Projection consumer lags (GC / backlog); feed cache not updated
t=80ms   Client GET /feed (no read-your-writes) → missing own post → “it vanished”
t=200ms  If dual-write to cache failed after DB commit → permanent gap until repair
t=2s     Consumer catches up; feed shows post; author already refreshed angrily
t=30m    Bug in projection logic → wrong counts; need shadow rebuild + cutover
Outcome  Write truth held; user-visible contract was unspecified → product incident

Counterexample to "CQRS means reads are always fresh": separating models buys speed only if UX owns lag, or you route the writer to a fresher path with or .

Still fails when…

  • Consumers fall behind under burst writes; authors see ghosts without read-your-writes.
  • A buggy projection silently serves wrong counts until shadow diffs catch it.
  • Command validation moves into the read model and recreates racey business rules.
  • Dual-writing DB and cache without outbox/CDC recreates permanent gaps after commit.
  • Skipping snapshots makes billion-event rebuilds operationally impossible under pressure.
  • Treating event sourcing as mandatory CQRS bloats systems that only need a change log.

Trade-offs

  • Fast denormalized reads versus duplicated data and projection logic to maintain.
  • Independent read scaling versus more operational moving parts and rebuild runbooks.
  • Event sourcing pairs well for audit, but is not required for most CQRS feeds.

Nearest alternatives

Single normalized model plus caching; read replicas with lag SLOs; a search index as a side projection only; full event sourcing when audit and replay are the product requirement, not when you only need faster feeds.

The split introduces duplicated storage, consumer lag, replay operations, and schema compatibility between writers and readers. It is worthwhile when the query shape, scale, or ownership meaningfully differs from the command path. For a small CRUD application, a single model with appropriate indexes may be easier to operate and easier for users to understand. CQRS can be adopted for one expensive screen without making every table event-sourced.

Choosing and explaining CQRS

A complete explanation connects the mechanism to a requirement. Describe what improves when you introduce it, which new failure cases become possible, and how you would know that the trade-off is worthwhile.

Explain the core idea

CQRS separates command models from query models so each can scale and evolve independently.

Writes protect invariants; read models are denormalized projections optimized for screens and APIs.

The read side is usually eventually consistent, so the UX must handle projection lag.

When it belongs

Read volume or query shape differs sharply from write volume.

Feeds, dashboards, search, or reporting need denormalized views.

A single normalized model creates slow joins or awkward indexes.

How it fails

Projection workers fall behind.

Read model rebuilds produce inconsistent user-visible state.

Command validation accidentally moves to the read model.

Trade-offs to weigh

Fast reads versus duplicated data and projection logic.

Independent scaling versus more operational moving parts.

Event sourcing pairs well but is not required.

Common misunderstandings

Using CQRS for simple CRUD with no scaling pressure.

Forgetting replay and backfill strategies for read models.

Hiding eventual consistency from the product experience.

Try it: CQRS write log versus read model lag

Append commands to an event log, project them into a read-optimized balance, and issue reads before projection finishes to see stale query results.

Command log (write side)

  1. No commands yet.

Read model (query side)

Projected version: 0 / 0

Balance: $100

Log length
0
Projection lag (events)
0
Query balance
$100

Append credit commands to the write side. The read model catches up asynchronously; reads may lag behind the log.

Going deeper

Production deep dive: independent models, explicit recovery

CQRS separates the model that protects write invariants from models optimized for queries. It does not require event sourcing: the write side can use ordinary tables and publish changes through an outbox.

What must stay true

A command is acknowledged only after the authoritative write model has enforced its invariants; projections may lag but must converge deterministically from the ordered change stream.

How the pieces fit

Change stream

12k events/s

Normal projection input.

Backfill

48k events/s

Shadow projector rate under a separate quota.

500M replay

≈ 3.9 h

500M ÷ (48k − 12k) net catch-up, before validation.

Command invariants

Commands carry actor, idempotency key, aggregate ID, expected version, and intent. The write model checks authorization, balance/stock constraints, and optimistic version in one transaction. A stale command is rejected; a read-model result is never trusted to authorize the write.

Projection idempotency and versioning

Store last_event_version per aggregate or processed event IDs in the same transaction as the projection update. Projection code has its own version; a semantic change builds v2 beside v1 rather than mutating an unrepeatable live model.

Replay, backfill, and dual-run

Snapshot source position, build a shadow v2 from history, tail new changes, compare counts and sampled query results, then dual-read or shadow-read before switching the alias. Quota backfill IO so it cannot delay live projection.

CQRS versus event sourcing

CQRS is the separation of command and query models. Event sourcing stores domain events as the write source of truth. They combine naturally, but CQRS can use current-state tables; event sourcing can project one read model without exposing a distinct command architecture.

Concrete rollback plan

Keep v1 consuming while v2 warms. Switch reads through a reversible alias or feature flag. If validation fails, route reads back to v1, stop v2 at a known offset, fix it, discard/rebuild v2, and never roll the authoritative write schema back incompatibly.

Worked example: Projection cutover timeline

  1. 1T0: record source offset O, start v2 backfill from snapshot, and keep v1 serving.
  2. 2T+4 h: v2 reaches O, then tails live events until lag < 5 seconds. Dual-run query comparisons for one hour.
  3. 3T+5 h: flip read alias to v2 for 5%, then 50%, then 100%; retain v1 and its cursor for the rollback window.

The cutover is a routing change, not a destructive migration. Rollback takes seconds because v1 never stopped receiving events.

CQRS failure matrix

ScenarioConsequenceHow to contain it
Duplicate change eventCounters increment twice.Atomic last-version check or event-ID inbox with update.
Projection skips version 91Read model silently diverges.Reject gap, pause that key, fetch/replay missing event, alert.
Backfill saturates primaryAuthoritative writes slow down.Snapshot/export source, quotas, replicas, and abort threshold.

What to watch in production

  • Monitor projection lag in seconds, version gaps, duplicate drops, rebuild ETA, source IO, and v1/v2 query divergence.
  • Expose “updated as of” when staleness affects users and route consistency-critical reads to the write model.
  • Practice reversible alias cutover and retain old projections until rollback confidence expires.

A scenario to reason through

A customer changes an email, then immediately reads the old value from the projection. Should the write be repeated?

One solid approach

No. Repeating the command may create a second event. Return the committed version/session token, wait until projection offset reaches it, or route that one read to the write store.

Going deeper

CQRS pattern

The mechanism and its boundary

CQRS separates the write model from the read model when the same schema cannot serve both correctness and query speed. The write side owns invariants; the read side is optimized, denormalized, and rebuildable.

Write and read models

1

Command

2

Write model

3

Event stream

4

Read projection

Work through the sizing assumptions

If writes are 5,000/sec and each write fans out to 4 read projections, projection consumers must process 20,000 events/sec plus replay headroom.
A 10-minute replay of 100M events requires about 166,000 events/sec of rebuild throughput.

When it belongs

Read queries need denormalized views that would slow writes. The product can tolerate read-model lag. Auditability or event replay is valuable.

Assumptions that matter

The event stream is durable and replayable. Read models can be rebuilt from source-of-truth writes. Users understand or are shielded from projection delay.

Easy mistakes

A small application with straightforward reads and writes can gain more operational complexity than query benefit from separate models. Invisible projection lag makes a successful user action appear lost, so expose pending state or provide a read path that can confirm the accepted version. The authoritative write model must enforce the business invariant; validation on a stale read projection cannot reliably prevent conflicting writes.

How it fails

Projection consumer falls behind and users see stale state. Events are not versioned and old projections cannot replay. Dual writes create gaps between command state and read state.

Trade-offs to name

A read projection can answer a query without expensive joins, but maintaining it adds consumers, duplicate data, lag monitoring, and a recovery process. Read and write capacity can scale separately, while asynchronous propagation means a successful command may not appear in the next read unless the application provides a session-aware path. A replayable source makes projections rebuildable, but old events must remain interpretable as schemas and business rules evolve, so versioning and rebuild validation are continuing responsibilities.

Wrap-up

You have seen the failure, the mechanism, and the operational signals. Use these exercises to check the ideas stuck, then follow the references for the primary sources.

Exercises

  1. Lag growth. Writes are 40K/sec into 3 projections; consumers do 90K updates/sec total. Does lag grow? If a 2-minute burst doubles write rate, how large is the backlog when the burst ends?
  2. Author freshness. Author must see their post within 200ms; followers may lag 2s. Design read-your-writes vs optimistic UI and name one failure if you skip both.
  3. Shadow rebuild. Projection bug undercounted likes for a week. Outline shadow rebuild, cutover, and the metric that proves it is safe to delete v1.

Topic-specific references

  • Martin Fowler: CQRS (Command Query Responsibility Segregation).
  • Greg Young: CQRS Documents / talks (write vs read models and eventual consistency).
  • Microsoft Azure Architecture Center: CQRS pattern.

Key takeaway

CQRS separates write truth from read speed. Projections build denormalized views; UX must own lag. Event sourcing is optional. Rebuild with replay and snapshots. Use it when read/write patterns diverge, like timelines serving billions of reads, not for simple CRUD.

Practice and further reading

Use these exercises to connect the lesson to a design decision. State your assumptions, trace what happens to one request, and explain the rule that must remain true even during a failure. That rule is the design's correctness invariant.

Focus for this lesson: Separate command invariants from query projections only when independent models justify replay, versioning, backfill, dual-run, and rollback cost.

Explain the decision

Does CQRS require event sourcing?

Read the reasoning

No. Commands and reads can use separate models over conventional state; event sourcing is a distinct persistence choice that often complements CQRS.

Apply it to a production scenario

A new projection produces wrong totals after deployment. How do you recover?

Read one approach

Stop promotion, keep the old projection serving, fix versioned logic, rebuild from a checkpoint/event source, compare, then switch atomically.

Work through the numbers

Rebuilding 2 billion events at 80,000/s takes how long ideally?

Read the worked calculation

25,000 seconds, about 6.9 hours, before throttling, state writes, retries, and live catch-up.

Reason through a failure

What must projection processing guarantee?

Read the recovery reasoning

Event ID/version checkpoints and idempotent updates prevent replay from double-counting, and serving never mixes incompatible projection generations.