SwadhyAI Notes

All System Design notes

System Design Fundamentals

Building Blocks

Request Uber fires. The map updates, a fare estimate appears, and a driver match starts. That one tap walks through the same kinds of boxes you will draw in almost every system design: discovery, edge, load balancing, APIs, data, and async work.

Building blocks are the standard pieces in system diagrams. Learn the job of each component first, then learn when traffic, failure, or consistency actually forces you to add it.

One ride request

1

Find where the app lives.

2

Serve nearby static bytes.

3

Pick a healthy app tier.

4

Cache + DB +

Truth now; slow work later.

What actually happens on the request path

Imagine opening a saved note. DNS resolves the API hostname before the connection is established; it is not a server that every application request passes through. Once connected, the browser can reuse that connection for several requests. A CDN may answer a cacheable request itself, while an authenticated note request usually continues to the application. These are alternative paths through the diagram, so a cache hit can remove entire downstream steps.

When saving the note, the application checks permission and validates the update before committing it to its authoritative database. Only after the required durable commit should it report that the note is saved. Updating a search index and sending a sharing notification can happen afterward. If the response is lost, a retry must recover the result of the same operation instead of creating another note.

The client asks for an address, may hit a for static assets, then reaches a and often an that handles auth and routing. Stateless app servers run product logic and keep durable state out of process memory.

Hot reads check a before the primary database. Slow work (receipts, notifications, map tile prep) lands on a so the user still gets a fast acknowledgment. When data no longer fits one owner, appears; when retries are inevitable, keeps side effects safe.

The component explanations below connect each job to its mechanism and its failure behavior, including cached DNS answers and stale application data.

Try it: compose the request path

Building blocks are jobs, not decorations. Turn each block on or off and watch which hops stay on the synchronous path. Reads care about cache hits; writes care about durable commit and optional async follow-up.

CDN

Static assets may skip the origin.

Load balancer

Spreads read traffic across app servers.

Cache

Hot reads can stop before the database.

Queue + workers

Optional for read-heavy paths.

Active hops (left to right)
  1. Client
  2. DNS
  3. Load balancer
  4. App
  5. Cache hit
Teaching latency (sync path)
~115 ms
Hops on critical path
5
Async branch
None

Toggle building blocks and switch between read and write. Active hops light up; latency is a teaching estimate, not a benchmark.

Read vs write paths differ. A CDN can remove origin load for static bytes but not for authenticated note bodies. A queue shrinks user-facing latency on writes only when the product accepts later completion.

Once you can name those jobs, the next skill is choosing which boxes earn a place on the whiteboard.

How to choose which blocks to add

Start from one user action, then ask what the request needs. Add a block only when a clear load shape or failure risk asks for it. Name the component and its trade-off, not just the technology brand.

1. Ask what the request needs. Read-heavy, write-heavy, latency-sensitive, or correctness-sensitive?

2. Add blocks only when needed. A fixes repeated reads; a fixes slow work; fixes data that no longer fits one owner.

3. Name the failure. If you cannot say what breaks first when a tier is overloaded, you do not understand the block yet.

Request path at a glance

EdgeSync pathDataAsync
  1. 1

    User action

    Click, tap, refresh

  2. 2

    Edge & routing

    DNS · CDN · LB

  3. 3

    Compute & APIs

    Gateway · services

  4. 4

    Data & cache

    Hot reads · truth (bidirectional)

  5. 5

    Queue for slow work

    Buffer · retry

  6. 6

    Workers

    Background finish

Choice language is useless without sizing. The metrics below turn "do I need a cache?" into load math you can defend.

Capacity scaffold

Numbers to know

A better interview move than naming brands is asking what load hits the source of truth, and what would make that safe. Use these labels on your diagram; later sections cross-ref here instead of restating the math.

QPS (queries per second)

Metric

Start with

How many user actions per day, and how spiky is peak traffic?

Use it for

Sizes API servers, database writes, queue partitions, and rate limits.

Storage

Metric

Start with

How many records or blobs are created per day, and how long are they retained?

Use it for

Decides single database vs partitioning, object storage, archival tiers, and backups.

Bandwidth

Metric

Start with

How large is each response or media object, and how often is it downloaded?

Use it for

Justifies CDN placement, compression, pagination, and regional edge copies.

p95 latency (95th percentile)

Metric

Start with

What response time must 95% of users experience under load?

Use it for

Creates a budget across edge, API, cache, database, and network hops.

Fanout (one write, many readers)

Metric

Start with

Does one write create work for many followers, subscribers, or devices?

Use it for

Reveals queue pressure, hot keys, celebrity users, and push vs pull.

Cache hit ratio

Metric

Start with

What percentage of reads can be served without hitting the source of truth?

Use it for

Turns vague cache talk into database load reduction math.

Worked example: 20M feed reads/day is ~230 average QPS. With a 5x peak, design for ~1.2K read QPS. If the feed cache hits 90%, the database sees ~120 QPS; if it hits 50%, the database sees ~600 QPS. That one assumption can decide whether a single primary plus replicas is fine or whether you need deeper partitioning.

With metrics in hand, map each trigger to a likely block, a proving metric, and the downside you accept.

Decision matrix

When should you add each block?

Do not add a component because it sounds advanced. Add it when a clear metric or failure risk asks for it (capacity figures live in Numbers to know).

TriggerLikely blockMetric to prove itDownside
Repeated reads are hitting the source of truthCache or read replicaCache hit ratio, database read QPSStale reads and invalidation work
Slow work blocks the user requestQueue plus workersQueue depth, oldest message age, retry countDelayed completion and duplicate handling
One database is too large or too busyShard or partitionWrite QPS, disk size, hot partition rateHarder queries and resharding
Users are far from serversContent delivery network (CDN)Download latency, bandwidth from originCache purge and freshness complexity
One server cannot handle trafficLoad balancer plus more app serversCPU, requests per second, error rateMore deploy and health-check paths
A failure can take down all usersReplication and failoverRecovery time, replication lag, error budget burnHigher cost and consistency choices

Typical Request Path

After the decision matrix, here is the same story as a flow: edge, app, data, then async.

  1. 1

    User

    Browser starts the request

  2. 2

    DNS

    Name to IP so clients connect

  3. 3

    CDN

    Nearby copies of static bytes

  4. 4

    LB

    Spread load across healthy servers

  5. 5

    Gateway

    Auth and limits at the front door

  6. 6

    Services

    Product logic on the request path

  7. 7

    Cache / DB

    Fast reads plus the source of truth

  8. 8

    Queue

    Buffer so spikes do not pile on

  9. 9

    Workers

    Finish slow jobs off the request

Sync path: stays within the user latency budget.

Data path: cache speeds reads; DB stores truth.

Async path: queue plus workers handle slow work off the request thread.

The linear path above rearranges when the main pressure is reads, writes, or background work.

Read, write, and async paths look different

A read path is usually optimized to retrieve an existing answer cheaply. A write path must decide whether a state change is allowed and where it becomes durable. An asynchronous path accepts an obligation now and finishes it later. Keep these promises distinct: “uploaded,” “accepted for processing,” and “processed successfully” represent different states and should not share a misleading success message.

For an illustrative workload of 2,000 reads per second with a 90% cache hit rate, only 200 reads per second reach the database, assuming each miss needs one query. If that cache disappears, database demand becomes 2,000 reads per second immediately. The cache reduces normal load, but safe operation also needs a bound on database concurrency, a fallback policy, or enough reserve capacity to handle misses.

The same blocks rearrange based on the main pressure. Compare the paths before choosing caches, queues, or shards.

Read-heavy path

Cache hit?

yes

no

Write-heavy path

Async-heavy path

With the paths clear, use the component explanations below as a catalog: what each component does, how it works, and which trade-off it introduces.

Traffic path & edge

Getting bytes to your code safely and evenly.

DNS

Discovery

Turns friendly names into IPs so the client knows where to open a connection.

Clients almost never hard-code server IPs. They ask a resolver for a hostname, get back one or more IPs (sometimes weighted for failover or geography), then open TCP/TLS to that address. Corporate and ISP resolvers cache answers for the record TTL, so DNS is both flexible and slightly sticky during moves.

Why it matters: During an address change, cached answers and existing connections can keep users on the old destination. Keep that destination available through the planned transition window.

CDN

Edge speed

Keeps copies of static assets near users: images, JS bundles, video chunks.

Origin servers sit in one or a few regions; a CDN caches responses at PoPs worldwide so repeated downloads do not cross the planet every time. You control cacheability with HTTP headers (Cache-Control, etc.). Dynamic API responses are often marked uncacheable and still hit your origin.

Why it matters: A cacheable response can bypass the origin entirely, reducing both transfer distance and origin load. Uncacheable database-backed requests still need an efficient application path.

Load balancer

Spread traffic

Splits requests across healthy servers; health checks kick out bad instances.

Incoming connections terminate at the LB (or pass through in some modes); it picks a backend using round-robin, least connections, or similar. Unhealthy nodes stop receiving traffic after failed checks. At scale you often have many LBs and DNS or anycast in front of them.

Why it matters: Connection-level balancing selects a destination for the whole connection; HTTP-aware balancing can route individual requests using paths or headers. The difference matters for long-lived multiplexed traffic.

API gateway / reverse proxy

Front door

Routing, TLS, auth hooks, rate limits, one choke point before many services.

From the client view there is one HTTPS endpoint; behind it the gateway routes to internal services by path or header. It is a natural place for authentication, coarse rate limits, request logging, and sometimes protocol translation. Keeping it thin avoids it becoming a second monolith.

Why it matters: Central routing simplifies policy, but expensive business transformations can turn the gateway into a shared bottleneck. Keep ownership and failure boundaries clear.

Application servers (stateless)

Business logic

Horizontally scaled processes; durable user state lives in database or cache, not RAM on one server.

Any instance should be able to handle any request (for that tier). Sticky sessions can simplify a stateful workload, but they couple a user to one instance. For ordinary request handling, signed tokens or shared session storage let another instance serve a retry. Choose session-store durability according to what losing a session means. That lets you add or remove servers during deploys or spikes without losing correctness.

Why it matters: Removing an application instance should not remove the only copy of a session or uploaded file. Shared state lets another healthy instance handle a retry.

Data, cache & storage

Where truth lives, and where you cheat latency.

Cache

Hot reads

In-memory speed layer; deliberately trades perfect freshness for latency.

The database is correct but comparatively slow; cache holds derived or repeated reads in memory. Common pattern is cache-aside: read cache first, on miss load DB and populate cache with a TTL. Invalidation is the hard part: stale data until TTL expires unless you update or delete keys on writes.

Why it matters: Expiry creates misses, and synchronized misses can overload the database. Stagger expiration, coalesce refills, and bound fallback traffic according to database capacity.

Primary database (SQL)

Source of truth

Relational database class: schemas plus transactions when money or inventory must add up.

Rows and transactions give you ACID guarantees and expressive queries (joins, constraints). One primary handles writes; read replicas spread read load but return data that may lag slightly behind the primary. When write throughput or disk size exceeds one machine, you move toward sharding or specialized stores.

Why it matters: Transactions and constraints enforce declared rules locally. Read replicas can reduce read pressure, while write capacity and cross-row coordination still require separate planning.

NoSQL / document / wide-column

Scale patterns

When access paths are predictable at huge volume: profiles, timelines, wide events.

These systems optimize for partitionable access patterns (key/value, document by id, wide rows by time). They trade arbitrary ad-hoc joins for horizontal scalability and tunable consistency. You choose partition keys and indexes up front so reads do not scan the cluster.

Why it matters: Choose keys around the reads and writes the application must perform. A query that cannot identify its partition can become an expensive scan across the cluster.

Object / blob storage

Big blobs

Object-store durability for uploads, backups, data lake files, not row-level OLTP.

Objects are addressed by key and stored with replication across disks and availability zones. Great for user uploads, static bundles, and analytics files (Parquet). Not a substitute for transactional row updates; you usually store a pointer (URL or key) in your relational DB.

Why it matters: Store ownership and processing state with the object reference. Downloads, transformations, retention, and deletion need a lifecycle that stays consistent with those metadata records.

Search index

Full-text

Full-text index for fuzzy search and ranking beside your transactional DB.

Inverted indexes map analyzed terms to matching documents and support relevance-ranked search; arbitrary substring matching needs suitable tokenization or specialized indexes. the operational DB stays authoritative for writes. Search clusters are updated asynchronously (CDC, binlog tail, or periodic batch). Expect eventual consistency between saved in DB and visible in search.

Why it matters: A saved record may become searchable later. Track indexing lag and keep a replay or rebuild path so the search copy can recover from missed updates.

Async & background work

Anything that should not block the user spinner.

Message queue

Buffer & retry

Durable work pipe: absorb spikes, retry workers, decouple publishers.

Producers push messages; consumers pull at their own pace, so short traffic bursts do not overwhelm downstream systems. Failed handling can retry with backoff; poison messages go to a dead-letter queue. Ordering may be per-partition only, so design accordingly.

Why it matters: A worker may finish a job and crash before acknowledgment, causing redelivery. Make repeated processing safe and give permanently failed jobs a visible repair path.

Pub/sub

Fan-out

One event reaches many subscribers (notifications, cache busts, audit pipelines).

A single publish can notify multiple independent services without the publisher knowing them all. Each subscriber typically has its own cursor or subscription so one slow consumer does not block others. Different from a task queue where each job is meant for one worker.

Why it matters: Independent subscriptions each react to an event and track their own progress. Several workers inside one subscription may still share its deliveries.

Workers / batch

Offline

Thumbnails, reports, reconciliation: user gets accepted fast, work finishes later.

The HTTP request acknowledges receipt or enqueueing; separate processes drain queues or run on schedules (cron, Spark jobs). This isolates long-running or unpredictable work from request latency caps. Ops watch queue depth and worker CPU to scale out.

Why it matters: A queue absorbs a temporary burst, but sustained arrivals above worker throughput make waiting time grow indefinitely. Bound admission and measure oldest-job age.

Scale & correctness habits

Patterns that show up in follow-up questions.

Replication

Failover

Extra copies across racks, AZs, or regions for reads and survival.

Data is copied asynchronously or synchronously to followers. If the leader dies, a replica can be promoted, often with a small window of lost or conflicting writes unless you use synchronous replication everywhere (costlier latency). Readers on replicas may see slightly old data until replication catches up.

Why it matters: A replica can be healthy but behind. Route freshness-sensitive reads using a known commit position or the authoritative writer, and test promotion against the promised data-loss bound.

Sharding / partitioning

Split data

Partition by key so no single machine owns the whole dataset.

Each shard holds a subset of keys (for example hash(user_id) mod N). Queries that cross shards are harder and slower, so you shard along dominant access paths. Uneven key distribution creates hot shards; one celebrity user can overload a single partition.

Why it matters: Evenly distributed keys do not guarantee evenly distributed requests. Plan for a dominant tenant and for moving partitions while preserving one authoritative write owner.

Idempotency

Safe retries

Retries and queues mean duplicates; design writes so twice is harmless.

Networks and clients retry on timeout; message systems may deliver more than once. Idempotent APIs use natural keys, dedupe tables, or idempotency tokens so a duplicate request updates nothing harmful (no double charge). This pairs tightly with queues and mobile clients.

Why it matters: Record a stable business operation identifier together with its committed result. A retry can then return the original outcome instead of applying the side effect again.

Interview practice questions

After the catalog, rehearse short answers you can say out loud in a mock interview.

Interviewer: “Where’s your first bottleneck if traffic 10× overnight?”Tap to reveal
  • Name the hottest resource: DB writes, cache miss storm, or queue consumers usually melt first.
  • Scale that tier horizontally; more app servers rarely fix a saturated primary database.
“Why add a queue instead of doing it inline in the API?”Tap to reveal
  • Protect user-facing latency when work is slow or flaky, such as email or ML scoring.
  • Smooth spikes and retry safely without tying up HTTP threads on the request path.
“When would you skip the CDN?”Tap to reveal

Mostly private APIs, highly personalized bytes per response, or strict caching bans; otherwise default to a for static assets.

Flashcard: “What breaks first, what metric proves it, and what is the downside?”Tap to reveal
  • Cache: stale data; prove with hit ratio and database read QPS.
  • Queue: delayed work; prove with queue depth and oldest message age.
  • Shard: hot partition; prove with per-shard QPS and latency.

Wrap-up

The cards and practice questions above cover the catalog. Use this section to lock invariants, rehearse failure, drill the math from Numbers to know, and leave with one clean takeaway.

Product shapes (same blocks, different pressure)

  • Social feed: plus plus fan-out plus search; cost and latency track read volume.
  • Ride / maps: high-write location streams, queues, and geo partitioning; latency is the product.
  • Payments: strong consistency, , and audit logs; fancy caches matter less than correctness.

Correctness invariant

An invariant is a rule that must remain true across every execution, including retries and crashes. For a notes product, one useful rule is that an acknowledged save can be retrieved after an application restart; another is that a user cannot read a private note without permission. Caches and replicas may accelerate reads, but the design must state how an authorization change or a just-completed save reaches those copies. A diagram becomes meaningful when each component helps preserve a stated rule.

Every request path has exactly one authoritative owner for each write, and retries never create a second successful side effect for the same client intent. Optional tiers (, , ) may be stale or delayed, but they must not silently replace the source of truth or invent commits the primary never recorded.

Failure under pressure

A good interview answer explains graceful degradation, not only the happy path.

  • Cache miss: fall back to the database, refill the cache, and watch hit ratio plus miss latency.
  • Database slow: serve cached stale data if acceptable, and watch p95/p99 database latency.
  • Queue backlog: accept the request with delayed status; watch depth and oldest message age.
  • Worker retry: use so duplicates stay harmless; watch retries and dead-letter size.
  • Dependency down: degrade the optional feature, keep the core flow alive, and watch error rate plus circuit opens.

Failure timeline: cache miss storm after deploy

Adding a is not free: a cold or flushed tier can stampede the database you meant to protect. Peak QPS math is in Numbers to know.

T0  Deploy flushes feed cache (or TTL expires for a hot key set)
T1  Peak ~1.2K read QPS all miss → DB sees ~1.2K instead of ~120
T2  Primary p99 climbs; replicas lag; API timeouts rise
T3  Clients retry without jitter → effective QPS multiplies
T4  Circuit / shed load / serve stale → DB recovers; refill with stampede control

Exercises

  1. Using the peak read QPS in Numbers to know, pick a cache hit ratio that keeps the primary under 200 QPS. What invalidation rule keeps that ratio after a viral post?
  2. Email send takes 800ms p95. Do you keep it on the request path or queue it? Name the user-visible contract and the .
  3. One shard owns 40% of writes. Do you re-shard now, introduce a queue buffer, or add replicas? What metric proves the choice?

Where to go next in Swadhyai

  • Fundamentals: sharpen interview flow and trade-offs so you pick blocks with reasons.
  • Networking: everything before your services speaks HTTP, WebSockets, and related transports.
  • Deep dives on load balancers, caches, databases, and queues turn each card above into a full chapter.

References

Key takeaway

Interviewers care more about trade-offs and failure handling than how many technologies you name. For major components, describe impact if that tier fails or saturates, cite the capacity figures in Numbers to know, and only add a box when a metric or failure risk demands it.

Practice and further reading

Use these exercises to connect the lesson to a design decision. State your assumptions, trace what happens to one request, and explain the rule that must remain true even during a failure. That rule is the design's correctness invariant.

Focus for this lesson: Translate requirements into a small set of stateful and stateless building blocks, then connect every box to a measurable constraint.

Explain the decision

Why is “draw a cache and a queue” not yet a design?

Read the reasoning

Each component needs an owner, data contract, sizing assumption, failure behavior, and reason it is preferable to a simpler path.

Apply it to a production scenario

A read-heavy product jumps from 500 to 50,000 requests per second in one region. Which blocks change first?

Read one approach

Separate cacheable reads, add a load-balanced stateless tier, protect the database with bounded pools, and validate hit rate and hot-key distribution before sharding.

Work through the numbers

Estimate application replicas for 50,000 requests/s when one replica safely sustains 800 requests/s and you require 30% headroom.

Read the worked calculation

Required capacity is 65,000 requests/s; ceil(65,000 / 800) = 82 replicas. Check downstream connection and network budgets too.

Reason through a failure

State the end-to-end invariant when a queue redelivers work.

Read the recovery reasoning

A logical operation changes durable business state at most once; stable operation IDs, transactional writes, and deduplication evidence preserve it.

Sources and further study

Continue with What is System Design?.