System Design Case Study
Scalable web applications
A successful product grows from one app server and a database into traffic that would melt that pair by lunch. Scaling a web app is less about one clever algorithm and more about layered defenses: , a , caches, , and queues.
Keep the request path thin: authenticate, authorize, read through , write to a single owner store, and push slow work to .
Request journey
Static + cacheable GETs.
Healthy stateless instances.
Replicas for reads; primary writes.
Queue what can wait.
What actually happens
Browsers hit a for assets and sometimes public API responses. Authenticated traffic reaches a that spreads load across identical app instances. Those instances stay so deploys and autoscaling do not require sticky sessions.
Hot reads use ; the database scales reads with and writes with vertical upgrades first, then when forced. Expensive or fan-out work lands on queues as .
Abuse and cascading failure are contained with , timeouts, and . comes after single-region excellence—not before.
How to choose the design moves
1. Is it read-heavy or write-heavy? Read-heavy systems lean on CDN + cache + replicas. Write-heavy systems need careful primary capacity, batching, and eventual shard plans.
2. What must be consistent? Profile display can lag a second; payment confirmation cannot. Route "read your write" to primary or version tokens instead of hoping replicas caught up.
3. Sync or async? If the user is not blocked on the result, enqueue it. If they are blocked, keep the critical section small and observable.
Simple rule of thumb
Scale the easy layers first (CDN, app replicas, cache). Shard and multi-region only when numbers force them— each adds failure modes you must staff.
In simple terms: make every server disposable, make every hot read cached, and make every slow task a job.
Case-study upgrade
Capacity Math Worksheet
Use this worksheet before drawing the final architecture. The aim is simple: every big number should force one design decision.
Assumptions and formulas
- Write daily active users, actions per user per day, payload size, retention, peak multiplier, and hot object risk.
- Average QPS (Queries Per Second) = users x actions per user per day / 86,400. Peak QPS = average QPS x 3 to 10.
- Storage = rows per day x bytes per row x retention x replication x index overhead.
Concrete math
- Read-heavy example: 100M users x 20 reads/day = 2B reads/day = 23K average QPS; 5x peak is 115K QPS.
- Write-heavy example: 50M devices x 2 messages/minute = 1.67M writes/sec; this needs partitioned ingestion and backpressure.
- Correctness-heavy example: 10K checkout attempts/sec x 10-minute holds = 6M live holds; stale holds must expire automatically.
Design decisions forced by numbers
- 100K+ read QPS means cache, read replicas, and clear cache invalidation.
- 10K+ writes/sec means shard writes and use a queue or partitioned log for async consumers.
- One hot object with 100K QPS needs replication, local cache, and request coalescing.
Scaling to millions
- At 1M users: keep the design simple, usually one region with cache and a durable database.
- At 10M users: add sharding, async workers, cache metrics, and replayable events.
- At 100M+ users: add regional traffic routing, overload policy, hot-key handling, and degraded modes.
Readable flow
Interview follow-ups to expect
- Which invariant must never break?
- Which number first forces sharding, caching, or a queue?
- Which path is synchronous, asynchronous, cacheable, or eventually consistent?
Operational readiness checklist
After the math, explain how this capacity plan is measured, protected, launched, and evolved.
Observability
- Attach each capacity estimate to a dashboard metric: p99 latency, queue age, cache hit rate, shard skew, error rate, and freshness lag.
- Alert on rate-of-change and saturation, not only hard failures; backlogs and hot keys usually warn before an outage.
Security and abuse
- Budget for auth checks, rate limits, quota isolation, audit logging, privacy filters, and abuse-review side paths.
- Separate trusted internal events from user-controlled input so retries and backfills cannot bypass policy checks.
Degraded modes
- Define the fallback when a dependency is slow: stale cache, partial results, read-only mode, delayed async work, or admission control.
- Protect the invariant first; reduce freshness, personalization, or analytics quality before risking duplicate or corrupt state.
Testing and rollout
- Test average, peak, and hot-object traffic; include retry storms, cold caches, lagging workers, and regional dependency loss.
- Use canaries, shadow reads, dual-write validation, feature flags, and explicit rollback criteria for risky path changes.
Rollout, backfill, and runbooks
- Make migrations and backfills idempotent, resumable, rate-limited, and visible through progress and error metrics.
- Write the runbook before launch: owner, dashboards, paging thresholds, mitigation switches, replay steps, and repair commands.
10x evolution
- For capacity math worksheet, state which number forces the next move: cache memory, shard count, write amplification, queue lag, egress, or failover reserve.
- Add the next layer only when the metric forces it: sharding, regional reads, async replay, hot-key isolation, or stricter backpressure.
Scope you should nail down
Product surface
- Web/mobile clients with auth (session or token)
- CRUD APIs, feeds/lists, search, file/media uploads as needed
- Background email, indexing, notifications
- Admin tooling and observability (not optional at scale)
Scale targets
- ~50M MAU, ~10M DAU (adjust in interview)
- ~100K peak QPS across APIs; p99 <200ms for cached reads
- Write path p99 <500ms excluding async side effects
- 99.9%–99.99% availability depending on product criticality
Capacity picture
Reads: 10M DAU × 200 reads/day ≈ 23K avg QPS; 5× peak ≈ 115K—cache and CDN are mandatory.
App tier: at 1K QPS/instance → ~100–150 instances at peak with headroom; autoscale on CPU + latency, not only CPU.
DB: 2K writes/s may fit one well-tuned primary; plan replicas early and sharding criteria before you panic at 10K writes/s.
Queue: if each write fans out to 3 jobs, workers must absorb ~6K jobs/s average and bursts without blocking HTTP.
Read path
Read the diagram definition
sequenceDiagram
autonumber
participant U as User
participant CDN as CDN / edge
participant LB as Load balancer
participant App as Stateless app
participant Cache as Cache
participant RR as DB read replica
participant P as DB primary
U->>CDN: GET /app.js (static)
CDN-->>U: 200 cache hit
U->>CDN: GET /api/profile
alt edge cacheable + hit
CDN-->>U: cached JSON
else miss / private
CDN->>LB: forward
LB->>App: route healthy instance
App->>Cache: GET profile:{id}
alt cache hit
Cache-->>App: profile
else cache miss
App->>RR: SELECT profile
RR-->>App: row
App->>Cache: SET profile:{id} TTL
end
App-->>U: 200 profile
Note over P: writes go to primary only
endSeparate public cacheable GETs from private authenticated reads. Never put personalization behind a shared CDN key without Vary/auth scoping.
Write path with async side effects
Read the diagram definition
sequenceDiagram
autonumber
participant U as User
participant App as App tier
participant P as DB primary
participant Cache as Cache
participant Q as Job queue
participant W as Workers
participant S as Search / email / etc.
U->>App: POST /posts (auth)
App->>App: validate + rate limit
App->>P: INSERT post (sync)
P-->>App: ok
App->>Cache: DEL / SET feed keys
App->>Q: enqueue index + notify
App-->>U: 201 Created
Q->>W: deliver job
W->>S: update secondary systems
alt worker failure
W->>Q: retry with backoff
Note over Q: DLQ after N attempts
endThe 201 means the durable write succeeded. Search lag or email delay should not fail the user's create—unless the product definition says otherwise.
APIs worth sketching
POST /auth/login → session/token
GET /users/{id} → profile (cacheable privately)
PATCH /users/{id} → profile (invalidate cache)
POST /resources → create (idempotency-key header)
GET /feeds/home → paginated feed
POST /uploads/session → presigned URL
GET /healthz | /readyz → LB probesCore data model
User
Session / Token
Resource
Cache keys
Job
Shard map
High-level architecture
App tier, caching, and data
Horizontal pods behind the LB. Store sessions externally. Use connection pools carefully so 200 app boxes do not open 200 × 100 DB connections.
Cache entities and computed feed pages. Invalidate on write. Protect hot keys with singleflight. Separate cache clusters for sessions vs application data if eviction policies differ.
&
Add replicas before sharding. When sharding, pick a key that matches request locality (usually user_id) and ban casual cross-shard joins in the app.
Idempotent workers, visible timeouts, DLQs, and poison-message handling. Backpressure: if the queue depth explodes, shed non-critical job types first.
Cache stampede sketch
value = cache.get(key)
if value: return value
if tryLock(key): // or singleflight
value = db.load(key)
cache.set(key, value, ttl=jitter(60s))
unlock(key)
return value
else:
sleep(brief); return cache.get(key) or degradeEdge, limits, and resilience
A and WAF absorb bots and static load. API gateways apply per user and route. Outbound calls to payments, email, and search use timeouts, retries with jitter, and so one dependency cannot exhaust the app tier.
Feature flags and dark launches reduce release risk. Prefer rolling deploys with readiness probes over big-bang restarts that empty the LB pool.
Multi-region (when forced)
Start with multi-AZ in one region. For , active-passive failover is the default interview answer unless the product needs active-active reads/writes. Pin writes to a home region; replicate read-only data asynchronously; accept conflict rules explicitly if dual writes exist.
// Practical progression
1. Single primary + replicas (multi-AZ)
2. CDN + cache + queued async (still one region)
3. Active-passive DR region with proven RPO/RTO
4. Active-active reads; careful home-region writes
5. Only then: multi-master with conflict policyFailure modes
Cache stampedes
Popular key expires; thundering herd hits the DB. Use locks/singleflight and jittered TTLs.
Replica lag surprises
User writes then immediately reads a stale replica. Sticky primary reads or version tokens fix the product bug.
Connection pool exhaustion
Autoscaled apps open too many DB connections. Centralize pools or use a proxy; alert on waiting clients.
Queue poison messages
Bad payloads retry forever. DLQ + replay tooling; idempotent handlers so replays are safe.
Cascading timeouts
Downstream slow → threads pile up → total outage. Timeouts + bulkheads + breakers contain the blast radius.
Trade-offs
Consistency vs scale
Replicas and caches multiply read capacity and introduce lag. Name which screens tolerate it.
Monolith vs services
A modular monolith scales far with a stateless tier. Split services when teams and failure domains demand it—not because diagrams look cooler.
Sync vs async
Async improves latency and isolation but adds eventual consistency and operational surface (DLQs, replays).
Single region vs multi-region
Global UX and DR conflict with write simplicity. Earn multi-region with measured RPO/RTO needs.
Production readiness
SLOs and metrics
- Gateway and endpoint latency/error burn rates
- Cache hit ratio, stampede locks, replica lag
- Queue depth, age, DLQ rate; DB connections in use
- Autoscaling events correlated with p99, not only CPU
Degraded modes
- Cache down → serve DB with aggressive shedding of non-critical reads
- Replica lag high → route sensitive reads to primary
- Worker outage → accept writes; show "processing" for async features
Security
- TLS everywhere; short-lived tokens; session revocation
- WAF + rate limits; upload scanning; least-privilege IAM to storage
- Secrets outside images; audit admin actions
Choosing and explaining scalable web applications
A complete explanation connects the mechanism to a requirement. Describe what improves when you introduce it, which new failure cases become possible, and how you would know that the trade-off is worthwhile.
Explain the core idea
Layer CDN, stateless apps, cache-aside, primary/replica storage, and async jobs.
Tie each layer to capacity math: QPS, hit rate, write rate, queue depth.
Call out consistency edges: read-your-writes, cache invalidation, and replica lag.
When it belongs
The prompt is a generic large web/mobile backend without a specialized product twist.
You need a backbone before specializing into feed, chat, or marketplace deep dives.
Interviewers ask how you grow from one box to millions of users.
How it fails
Sticky sessions preventing scale-out.
Cache miss storms after deploys or TTL alignment.
Unbounded DB connections from autoscale events.
Trade-offs to weigh
Replica lag versus read capacity.
Async side effects versus immediate consistency of secondary systems.
Sharding complexity versus vertical scaling runway.
Common misunderstandings
Jumping to multi-region Kafka meshes before CDN and caching.
Treating microservices as a scalability prerequisite.
Ignoring idempotency on creates and job handlers.
Takeaway
Scalable web apps are layered: edge for bytes, disposable compute for logic, caches and replicas for reads, a careful primary for writes, and queues for everything that can wait.
Review the design
Edge: CDN/WAF for static and cacheable GETs; protect origin.
Compute: stateless app tier behind LB with real readiness probes.
Reads: cache-aside + read replicas; manage stampede and lag.
Writes: primary ownership, idempotency keys, targeted cache invalidation.
Async: queues, retries, DLQs for email/index/notify.
Resilience: rate limits, timeouts, circuit breakers; multi-region last.
Compare a starting answer with a more complete design
A starting answer
Put a load balancer in front of multiple app servers, add a cache and database replicas, and process slow tasks in a background queue.
Developing the reasoning
Discuss cache-aside stampedes, read-your-writes against replica lag, connection pooling under autoscaling, idempotent jobs/DLQs, and a staged path to sharding and multi-region.
Deeper walkthrough
Scalable web app deeper walkthrough
A general-purpose large web application must serve many concurrent users with low latency while keeping writes correct and side effects reliable.
Requirement refinements
Clarify read/write mix, consistency needs, auth model, media uploads, and background workloads.
State invariants: stateless compute; durable write before 2xx on creates; async jobs idempotent.
API design details
Idempotency keys on creates; separate health vs readiness; paginate list endpoints.
Presign large uploads so app servers are not byte proxies.
Data model and storage
Primary entities keyed for future sharding; cache keys versioned; job records for at-least-once work.
Avoid cross-entity transactions that will not survive a shard boundary.
Scaling strategy
Scale CDN and app replicas first; add cache and DB replicas; shard when write QPS or storage forces it.
Autoscale on latency and saturation signals; cap DB connections per instance.
Reliability, recovery, and operations
Timeouts and breakers on dependencies; DLQs for poison jobs; degrade reads when cache or replicas fail.
Monitor p99, error budget, cache hit rate, replica lag, queue age, and pool wait time.
Primary sources
- AWS Well-Architected: reliability and performance efficiency pillars.
- Google SRE Book: SLOs, error budgets, and load balancing themes.
- Azure Architecture Center patterns: cache-aside, circuit breaker, queue-based load leveling.