System Design Case Studies

Case Study Playbook

Imagine an interviewer says: "Design Instagram stories," or "Design Uber matching," and the clock starts. The strong answer is not a memorized box diagram. It is a calm conversation that turns a vague product into requirements, rough , APIs, data, bottlenecks, and trade-offs.

This playbook is the repeatable recipe. Use it for every case study in this section so you spend interview time on the hard part, not on remembering which step comes next.

What happens first?

Clarify the product

What users can do, and what you are not building.

Name the promise that must never break.

→

Rough numbers that force a design move.

Deep dive the hard part

Fan-out, locking, ordering, or overload: pick one.

Try it: walk the case-study playbook on one problem

Adjust DAU and actions, compute QPS, then commit to cache, shard, or queue when peak traffic exceeds a single node's comfort zone. The path advances as you complete each interview stage.

DAU (millions): 10Actions/user/day: 20Peak multiplier: 5×

Clarify scope
Capacity math
API & data
Hot path
Failure modes

Highlighted hops show where the last action moved traffic. A stopped hop means backpressure, rejection, or a consistency guard blocked further progress on that path.

Average QPS: 2,315
Peak QPS: 11,575
Peak multiplier: 5×
Mitigations chosen: None

Capacity pressure

math

Hot read paths above tens of thousands of QPS rarely survive without edge or regional cache. Write-heavy paths need partition keys that match access.

Consistency choice

trade-off

Cache and async fan-out both introduce staleness windows. State the invariant (ordering, no duplicate charge) before picking AP vs stronger guarantees.

Interview path

closure

Close with observability: QPS, cache hit ratio, queue age, and one explicit degraded mode.

Set traffic assumptions, run capacity math, then pick one mitigation per bottleneck the numbers expose.

One interview question becomes a structured conversation

The interviewer gives you a product name. You answer with a story: who the user is, what action they take, and what must stay true. That decides whether you need , strong consistency, or whether a slightly stale cache is acceptable.

Next you estimate. , actions per user, and a peak multiplier become average and peak . Storage, bandwidth, and math turn into decisions:, a , a queue with , or a CDN for egress.

Only then do you draw boxes. A good diagram separates the read path, write path, async path, and failure path. Then zooms into the one place where correctness or scale is hardest.

How to choose what to deep dive

1. Where does work amplify? Feeds, notifications, and group chat create . Spend time on write amplification and hybrid push/pull.

2. Where is correctness fragile? Payments, seats, and inventory need , state machines, and reconciliation, not just more replicas.

3. Where does the system melt under load? Hot keys, queue lag, and region outages need , admission control, and graceful degradation.

Simple rule of thumb

Start simple: one service, one database, clear APIs. Add cache, queues, shards, and multi-region only when a requirement or a number forces it. Do not invent complexity to sound advanced.

In simple terms: state the , show the math, then name the component that protects that promise.

Case-study upgrade

Capacity Math Worksheet

Use this worksheet before drawing the final architecture. The aim is simple: every big number should force one design decision.

Assumptions and formulas

Write daily active users, actions per user per day, payload size, retention, peak multiplier, and hot object risk.
Average QPS (Queries Per Second) = users x actions per user per day / 86,400. Peak QPS = average QPS x 3 to 10.
Storage = rows per day x bytes per row x retention x replication x index overhead.

Concrete math

Read-heavy example: 100M users x 20 reads/day = 2B reads/day = 23K average QPS; 5x peak is 115K QPS.
Write-heavy example: 50M devices x 2 messages/minute = 1.67M writes/sec; this needs partitioned ingestion and backpressure.
Correctness-heavy example: 10K checkout attempts/sec x 10-minute holds = 6M live holds; stale holds must expire automatically.

Design decisions forced by numbers

100K+ read QPS means cache, read replicas, and clear cache invalidation.
10K+ writes/sec means shard writes and use a queue or partitioned log for async consumers.
One hot object with 100K QPS needs replication, local cache, and request coalescing.

Scaling to millions

At 1M users: keep the design simple, usually one region with cache and a durable database.
At 10M users: add sharding, async workers, cache metrics, and replayable events.
At 100M+ users: add regional traffic routing, overload policy, hot-key handling, and degraded modes.

Readable flow

Clarify scope

Estimate numbers

Choose APIs

Model data

Draw paths

Deep dive hard part

Interview follow-ups to expect

Which invariant must never break?
Which number first forces sharding, caching, or a queue?
Which path is synchronous, asynchronous, cacheable, or eventually consistent?

Operational readiness checklist

After the math, explain how this capacity plan is measured, protected, launched, and evolved.

Observability

Attach each capacity estimate to a dashboard metric: p99 latency, queue age, cache hit rate, shard skew, error rate, and freshness lag.
Alert on rate-of-change and saturation, not only hard failures; backlogs and hot keys usually warn before an outage.

Security and abuse

Budget for auth checks, rate limits, quota isolation, audit logging, privacy filters, and abuse-review side paths.
Separate trusted internal events from user-controlled input so retries and backfills cannot bypass policy checks.

Degraded modes

Define the fallback when a dependency is slow: stale cache, partial results, read-only mode, delayed async work, or admission control.
Protect the invariant first; reduce freshness, personalization, or analytics quality before risking duplicate or corrupt state.

Testing and rollout

Test average, peak, and hot-object traffic; include retry storms, cold caches, lagging workers, and regional dependency loss.
Use canaries, shadow reads, dual-write validation, feature flags, and explicit rollback criteria for risky path changes.

Rollout, backfill, and runbooks

Make migrations and backfills idempotent, resumable, rate-limited, and visible through progress and error metrics.
Write the runbook before launch: owner, dashboards, paging thresholds, mitigation switches, replay steps, and repair commands.

10x evolution

For capacity math worksheet, state which number forces the next move: cache memory, shard count, write amplification, queue lag, egress, or failover reserve.
Add the next layer only when the metric forces it: sharding, regional reads, async replay, hot-key isolation, or stricter backpressure.

Production-ready answer moves

Invariant-first design

State the promise before naming components: no double booking, no lost acknowledged message, no double charge, no private content leak.
Use the invariant to decide consistency, idempotency, locking, reconciliation, and what can safely be stale.

Numbers to decisions

Turn every estimate into one move: cache for read QPS, partition for write QPS, queue for worker concurrency, CDN for egress, or admission control for spikes.
Say the threshold that changes the design, such as hot key at 100K QPS, fan-out above 1M inserts/sec, or cache footprint above one cluster.

Trace four paths

Draw read path, write path, async path, and failure path separately so synchronous work, retries, and degraded behavior are visible.
Mark which path is authoritative, which is cacheable, which is replayable, and which can return partial or stale results.

Operational checklist

Cover observability, security, degraded modes, load and failure testing, canary rollout, backfill/replay safety, and runbooks.
End with the 10x evolution: which metric breaks first and what component you would change next.

How the answer flows

Use the same flow for URL shortener, feeds, maps, payments, caches, and key-value stores. The details change, but the conversation shape stays stable.

Clarify scope and users

Write functional requirements

Write non-functional targets

Estimate QPS, storage, bandwidth

Define APIs

Model data and indexes

Draw read, write, async paths

Pick hard part

Scale

Sharding, caching, queues

Correctness

Transactions, idempotency, consistency

Reliability

Retries, failover, backpressure

State tradeoffs and failure modes

Interview stepsHard-part choice

Beginner goal

Start by defining the user journey and its success condition. For a shortener, creation succeeds when the mapping is durable, while a redirect succeeds when it returns the intended destination within its latency budget. Derive a small API surface and data model from that scope. Then draw the simplest components that can meet it, identifying one bottleneck to investigate first.

Intermediate goal

Use each estimate to test a concrete assumption. If one million requests/s hit a cache with a 99% hit rate, the database still receives 10,000 misses/s; at 95%, that grows to 50,000/s. Compare those rates with measured database capacity before calling the cache sufficient. Recalculate under a cold cache, a hot key, or one lost zone because the average healthy path hides the difficult case.

Advanced goal

Walk a failure through the same path as a normal request. Name what has committed, which response may have been lost, who retries, and how repeated work is recognized. Describe the resulting user state, such as pending payment or a stale feed, and the recovery mechanism. This makes idempotency, queues, and reconciliation consequences of the product contract instead of unexplained component names.

1. The answer loop

1. Clarify the product

Beginner: State what users can do and what you are not building.

Advanced: Turn vague product words into explicit APIs, invariants, and exclusions.

2. Split functional and non-functional requirements

Beginner: List features separately from scale, latency, consistency, and availability.

Advanced: Rank requirements because trade-offs depend on what must never break.

3. Estimate scale

Beginner: Use rough DAU, QPS, storage, and bandwidth numbers.

Advanced: Separate average, peak, hot-key, fan-out, retention, and regional traffic.

4. Design APIs

Beginner: Show the main reads and writes.

Advanced: Add idempotency keys, pagination cursors, auth boundaries, and retry behavior.

5. Model data

Beginner: Identify primary entities and relationships.

Advanced: Choose partition keys, indexes, TTLs, denormalization, and consistency rules.

6. Draw high-level architecture

Beginner: Client -> gateway -> services -> data stores.

Advanced: Separate read path, write path, async path, caches, queues, and regional failover.

7. Deep dive the hard part

Beginner: Pick the feature that makes this system unique.

Advanced: Discuss race conditions, overload, ordering, retries, backpressure, and recovery.

8. Close with trade-offs

Beginner: Explain why your design is good enough.

Advanced: Name what you sacrificed and how you would evolve the design at 10x scale.

2. Spoken answer skeleton

plaintext

1. Clarify scope
   "I will focus on core write/read flows first. I will exclude admin tools unless time permits."

2. State requirements
   Functional: create/read/update core object, search/list, notifications.
   Non-functional: p99 latency, availability, consistency, durability, cost.

3. Estimate
   DAU -> QPS -> peak QPS -> storage/day -> retention -> bandwidth.

4. Propose APIs and data model
   Include idempotency, pagination cursor, auth, and primary indexes.

5. Draw high-level design
   Client -> API gateway -> services -> cache/database/queue/object storage/search.

6. Deep dive one hard part
   Examples: feed fan-out, seat locking, payment idempotency, matching engine, CRDT sync.

7. Failure handling and trade-offs
   Retries, dedup, backpressure, failover, monitoring, what gets degraded first.

3. Pattern map

When you see this	Reach for these patterns
Read-heavy content	CDN, cache-aside, precomputed views, pagination, search index
Write-heavy ingestion	Queues, partitioned logs, batching, backpressure, async consumers
Real-time collaboration	WebSockets, session routing, operation log, CRDT/OT, presence cache
Money or inventory	Idempotency, transactions, state machine, audit log, reconciliation
Geo systems	Geohash/H3, nearby search, location TTL, regional partitioning
Analytics/reporting	Event stream, deduplication, late data windows, OLAP store, rollups
Hot celebrities/events	Hybrid fan-out, virtual waiting room, token admission, sharded counters
Global availability	Multi-region reads, async replication, conflict policy, graceful degradation

4. Calculation playbook

Interview calculations are useful when they force a design decision. Convert every number into a bottleneck: backend QPS, cache size, shard count, queue lag, bandwidth, or storage growth.

Traffic

DAU x actions/user/day / 86,400 = average QPS. Multiply by 3x to 10x for peak.

Storage

Rows/day x bytes/row x retention x replication factor. Include indexes, metadata, and tombstones.

Bandwidth

Requests/sec x payload size. Separate ingress, egress, internal replication, and async events.

Cache

Hot set size x bytes/item x overhead x replicas. Convert hit rate into backend fallback QPS.

Fan-out

Writes/sec x recipients/write. Decide where the write amplification is paid.

Queues

Events/sec x processing time = worker concurrency. Queue age shows whether capacity is keeping up.

5. Scaling to millions of users

Traffic assumptions

State active users, actions per user, read/write ratio, peak multiplier, and regional split.
Call out hot users, hot objects, and scheduled spikes because averages hide the real bottleneck.

Bottlenecks

Name the first component to saturate: database writes, cache hot keys, fan-out workers, object-store egress, ranking CPU, or provider limits.
Attach one metric to each bottleneck so the design is measurable.

Partitions and shards

Choose partition keys from access patterns: reader_id for timelines, conversation_id for chat, short_code for redirects, object_id for media.
Explain what happens during resharding and how much data moves.

Caches and edge delivery

Separate object caches, metadata caches, local hot-key caches, and negative caches.
Mention TTL jitter, request coalescing, stale-while-revalidate, and explicit invalidation for deletes or privacy changes.

Async pipelines

Move analytics, notifications, media processing, indexing, ranking features, and cleanup off the hot path.
Define retry, dedup, dead-letter, replay, and lag behavior before picking infrastructure.

Failure and degradation

Decide which path must remain available and which feature can become stale, partial, or temporarily disabled.
Tie recovery to idempotency keys, checkpoints, reconciliation jobs, and operator alerts.

6. Questions to ask yourself

Requirements

Beginner question: What does the user do?

Advanced question: What is the correctness invariant?

Example: Seats cannot be sold twice. Payments cannot be charged twice.

Scale

Beginner question: How many users and requests?

Advanced question: Where are the spikes and hot partitions?

Example: 1M people join a ticket sale in 60 seconds.

API

Beginner question: What endpoints are needed?

Advanced question: How do retries, pagination, and auth work?

Example: POST /payments requires Idempotency-Key.

Data

Beginner question: What tables or collections exist?

Advanced question: What is the partition key and index strategy?

Example: timeline:{user_id} as cache sorted set, score = rank or timestamp.

Architecture

Beginner question: What services are involved?

Advanced question: Which path is synchronous, which path is async?

Example: Checkout is synchronous; notification and analytics are async.

Reliability

Beginner question: What if one component fails?

Advanced question: How do replay, dedup, timeout, and failover work?

Example: Replay an event log from last committed offset into an idempotent consumer.

7. From simple to advanced

Single service + SQL

Cache hot reads

Queue async work

Shard by access pattern

Multi-region + degradation

Do not start at the final architecture. Show the simple version first, then add each component because a requirement forced it: lower latency, higher throughput, isolation, durability, or regional resilience. For example, begin a file service with one metadata database and object storage. Add resumable upload sessions when a large upload can lose its connection; add a change log when devices need incremental sync; shard metadata when one owner or namespace exceeds capacity. At each step, state the new failure mode introduced and how the design contains it.

8. Common traps and fixes

No scale assumptions

Add rough DAU, QPS, storage, peak multiplier, and retention.

One database for everything

Separate OLTP, cache, search, object storage, stream, and OLAP when the access patterns differ.

Ignoring retries

Add idempotency keys, request hashes, dedup stores, and safe state transitions.

No hard-part deep dive

Pick the unique bottleneck: fan-out, matching, locking, ordering, ranking, or ingestion.

Only happy path

Discuss overload, partial failure, duplicate messages, cache stampede, and region outage.

Overengineering early

Start with a simple design, then evolve when a requirement forces complexity.

9. Practice order

Beginner

URL shortener
Text snippet sharing
Typeahead
File storage

Intermediate

News feed
Chat
Notification system
Food delivery marketplace

Advanced

Collaborative document editing
Metrics and alerting platform
Ad attribution pipeline
Trading exchange

10. Capacity worksheet template

Fill this in out loud before naming components. Every blank should become a design decision or an explicit assumption.

plaintext

DAU / MAU: __________
Actions per user per day: __________
Average QPS = DAU x actions/day / 86,400 = __________
Peak multiplier (3x-10x): __________  -> Peak QPS = __________
Read:write ratio: __________
Payload size (avg / max): __________ bytes
Retention window: __________ days
Rows per day = __________
Bytes per row (lean / conservative): __________ / __________
Storage/day = rows/day x bytes/row = __________
Storage with indexes (1.5x-2.5x): __________
Storage with replication factor R: __________
Cache hot set size: __________ keys
Cache bytes/item: __________  -> Cache footprint = __________
Cache hit rate target: __________%  -> Backend fallback QPS = peak x (1 - hit rate)
Async events/sec: __________
Processing sec/event: __________  -> Worker concurrency = __________
Bandwidth egress = QPS x payload = __________
Queue lag target: __________ sec

11. Numbers to decisions

When you calculate	Design move	Example threshold
100K+ read QPS after cache miss math	Add distributed cache, read replicas, CDN for static payloads	Backend fallback still above 5K QPS at 95% hit rate
10K+ sustained writes/sec	Partition by access key, append-only log, or write-sharded store	Single relational primary saturates around 1K-5K simple writes/sec
1 TB+ hot cache footprint	Shard cache cluster, pick eviction policy, replicate hot keys	One 64 GB node holds roughly 50 GB usable data
Fan-out > 1M recipients/write	Hybrid fan-out: async workers plus merge-on-read for celebrities	500 followers x 10K posts/sec = 5M timeline inserts/sec
Queue age > processing budget	Scale consumers, shed load, or move work off sync path	If 100 ms/event and lag is 30 sec, you need 300x more workers
Hot key > 50K QPS on one shard	Local cache, key replication, request coalescing, or key splitting	One viral short code can exceed entire cluster average
Storage/day > object-store tier comfort	Tiered retention, rollups, compression, lifecycle rules	Raw analytics at 100 bytes/event and 40K/sec is ~346 GB/day

12. Worked examples by workload type

Read-heavy: URL shortener

Assumptions: 100M links/month, 100:1 read:write, lean row 264 bytes, 99% cache hit at peak.

Average redirect QPS: 3.33M/day writes = 38.6 write QPS; reads ≈ 3,860 QPS average.

Peak redirect QPS: 10× average ≈ 39K QPS.

Backend mapping QPS: 39,000 peak redirects/s × 1% misses gives roughly 390 database reads/s with the assumed 99% cache hit rate. At 90% hits it becomes 3,900 reads/s, and a cold cache exposes all 39,000. These are scenarios to test against database capacity; the number alone does not prove that a cache is mandatory. Miss coalescing and gradual warm-up protect the transition.

Decision: Keep redirect sync; analytics async; replicate hot codes; never block redirect on rollups.

Write-heavy: metrics platform

Assumptions: 2M samples/sec, 16 bytes/sample payload, 30-day hot retention, cardinality quotas enforced at ingest.

Ingress bandwidth: 2M × 16 B ≈ 32 MB/sec before labels and protocol overhead.

In-flight work: At two million samples/s and 200 ms residence time, Little’s Law gives about 400,000 samples in flight. If a flush batch contains 1,000 samples, that is roughly 400 concurrent batches under the same assumptions, not 400,000 batch slots. Include batch-formation time, replicas, and retry headroom before converting that estimate into worker concurrency.

Decision: Push ingest to partitioned log, reject label explosions early, compact blocks asynchronously, query fan-out with caches.

Correctness-heavy: ticket booking

Invariant: a seat is available, held, sold, or blocked, never sold twice.

Peak crowd: 1M users waiting; inventory service budget 10K hold attempts/sec, not 1M/sec.

Hold cardinality: If 10,000 successful new holds/s each remain active for the full 600 seconds, the upper illustrative steady-state count is six million holds. Attempt rate is not successful-hold rate, and actual live holds are also bounded by available seats and early purchases or cancellations. For a 50,000-seat event, inventory contention and admission are the immediate constraints, not a six-million-row hold table.

Decision: Waiting room admission, signed tokens, atomic hold on authoritative inventory, idempotent payment confirm.

13. Interview scoring rubric and scored sample

Scope and requirements (15%)

States invariants, exclusions, and functional vs non-functional targets.

Capacity math (20%)

Shows formulas; ties at least two numbers to components.

API and data model (15%)

Idempotency, pagination, partition keys, and indexes match access patterns.

Architecture paths (20%)

Separates read, write, async, and failure paths clearly.

Hard-part deep dive (20%)

Explores races, overload, retries, or ordering on the unique bottleneck.

Trade-offs and ops (10%)

Names degraded modes, metrics, and 10× evolution without overengineering v1.

Fully scored sample: ticket booking (staff-level target ≈ 92/100)

Rubric area	Score	Why
Scope	14/15	Defines assigned vs GA seats, 10-min hold, bot defense, payment timeout; excludes resale marketplace.
Capacity	18/20	Derives 200K queue-status QPS from 1M users / 5 sec polling; caps admission at 10K holds/sec; computes 6M live holds.
API + data	14/15	Queue token, reserveSeat with version, confirmBooking with idempotency_key; seat row is authoritative state machine.
Architecture	19/20	Browse path cached; hold path hits inventory; payment saga with compensation; expiry worker releases seats.
Deep dive	18/20	Atomic hold, stale seat-map vs authoritative hold, payment-after-expiry reconciliation, hot-event shard isolation.
Ops	9/10	Metrics on double-sold invariant, degraded browse-with-stale-map, 10× regional waiting rooms.

Total: 92/100: strong pass. Missing points: explicit DR/fencing story for inventory primary failover.

14. Time-boxed interview worksheets

Mid-level (45 min)

0-5 min: clarify users, core flows, one invariant
5-12 min: DAU → QPS → peak → one storage estimate
12-20 min: 4-6 APIs + primary entities
20-32 min: diagram client → gateway → services → stores
32-40 min: deep dive the obvious bottleneck
40-45 min: one failure mode + trade-off

Senior (60 min)

0-5 min: requirements + explicit non-goals
5-15 min: full capacity worksheet with peak and hot-key callout
15-25 min: APIs, idempotency, partition keys
25-38 min: read/write/async/failure paths on board
38-52 min: hard-part deep dive with numbers
52-60 min: degraded modes, metrics, 10× evolution

Staff (75 min)

0-8 min: product framing, invariants, multi-region scope
8-20 min: average vs peak vs hot partition math
20-30 min: data model + consistency boundaries
30-48 min: end-to-end paths and state machines
48-62 min: correctness under retries, overload, partial outage
62-75 min: operability: SLOs, rollout, reconciliation, org trade-offs

15. Follow-up question banks by level

Mid-level follow-ups

What happens if this database is down for five minutes?
How would you paginate this read API?
Where would you add a cache first, and what TTL?
What is the single hottest key or partition?
What metric tells you the design is unhealthy?

Senior follow-ups

Walk me through a retry storm after a 30-second outage.
How do idempotency keys prevent duplicate side effects here?
What can be eventually consistent vs must be authoritative?
How does the design change at 10× traffic with one new constraint?
What degraded mode do users see when ranking or search lags?

Staff follow-ups

How would you roll this out without risking money or inventory correctness?
What reconciliation job proves the invariant held overnight?
Where would multi-region placement help vs hurt correctness?
What organizational boundary would you draw between teams and why?
Which requirement would you push back on first, and what cheaper alternative exists?

Key takeaway

A strong case-study answer is a conversation, not a dump of components. Clarify the product, name the invariant, estimate until a number forces a move, draw read/write/async/failure paths, then deep dive the one hard part. Close with trade-offs and how the design evolves at 10×.

Primary sources

Google SRE: Engaging with Production: how production constraints shape design talk.
AWS Architecture Center: grounding case studies in real patterns.
RFC 2119: Requirement levels: MUST/SHOULD language for invariants.

Practice and further reading

Use these exercises to connect the lesson to a design decision. State your assumptions, trace what happens to one request, and explain the rule that must remain true even during a failure. That rule is the design's correctness invariant.

Focus for this lesson: Apply a timed case-study loop that turns ambiguous requirements into estimates, contracts, state ownership, bottlenecks, failure proofs, and a scored recap.

Explain the decision

What should every case-study answer make explicit before choosing storage?

Read the reasoning

Core entities, access patterns, scale, transaction/invariant boundaries, latency and durability goals, retention, and failure assumptions.

Apply it to a production scenario

At minute 25 the interviewer asks about regional failure. How do you pivot?

Read one approach

Freeze the core design, identify regional state and dependencies, then walk traffic evacuation, write fencing, RPO/RTO, reconciliation, and failback.

Work through the numbers

A 45-minute interview leaves 15 minutes for deep dive and 5 for recap. How much remains for discovery, estimates, and core design?

Read the worked calculation

25 minutes; a practical split is 5 discovery, 5 estimates/contracts, and 15 core architecture.

Reason through a failure

What must the final recap contain?

Read the recovery reasoning

The primary invariant, largest bottleneck, chosen trade-off, degraded mode, recovery path, and the measurement that proves success.

Sources and further study

Continue with Realistic System Design Case Studies.