What is System Design?

How to choose where to start

Once you see that growth forces new pieces, pick a starting path instead of memorizing every tool at once.

1. 1

   Requirements

   Requirement

   Users can create and read notes, so the scope stays concrete.

   ↓
2. 2

   Estimates

   Estimate

   Ten lakh reads per day is about twelve average QPS to size with.

   ↓
3. 3

   Simple diagram

   Diagram

   Client to API to database is enough for a first sketch.

   ↓
4. 4

   Bottleneck

   Bottleneck

   Peak reads may overload the database first, so name that bottleneck early.

   ↓
5. 5

   Trade-off

   Trade-off

   Add a cache for speed, and accept briefly stale reads.

Prerequisite map and how to read this page

After the 10-minute drill, use this map so you know what this page owns versus what Building Blocks owns. This page is the why and mental model: requirements, scale drivers, latency, availability, and distributed-systems vocabulary. For the component vocabulary (CDN, cache, queue, load balancer boxes), read Building Blocks first or in parallel.

Requirements: the first design material

Start with a concrete promise: a user can save a private note, close the browser, and open the same content from another device. This implies an authenticated write API, durable storage, and a read path that returns the right user’s data. “Support notes” is too vague to establish those requirements; describing the action makes the necessary behavior visible.

Then make quality measurable. For example, assume 95% of reads should finish within 200 milliseconds, acknowledged writes must survive one server failure, and other devices may see edits up to two seconds late. These are different requirements: the first concerns latency, the second durability, and the third freshness. Improving one does not automatically improve the others.

The thinking loop starts with a product promise. Requirements turn that promise into design material: whether you optimize for correctness, latency, cost, operability, or growth. Split them into functional (what users can do) and non-functional (how well the system must do it).

Functional

Functional requirements name actions and rules. In a notes app, creating, reading, editing, sharing, and deleting a note are distinct flows. Describe who may perform each action and what changes when it succeeds: sharing might create a permission record, while deletion might hide a note immediately and remove stored copies later. These details define the API and data model before any scaling choice.

Include the important failure behavior in the flow. If saving times out, the client needs to discover whether the save completed. If sharing fails, the original private note should remain accessible to its owner. Describing these outcomes makes the boundary of each operation clear.

Non-functional

Quality requirements put measurable limits around those actions. Estimate daily active users, read and write rates, peak bursts, record sizes, and retention so the design has a workload. Then specify latency, availability, durability, and freshness separately. A fast response from stale data may satisfy a latency target while violating the user’s expectation after an edit.

Operational constraints also shape the solution: budget, the team’s ability to run a database cluster, backup recovery time, and access controls. Record assumptions explicitly so a later change in traffic or product requirements leads to a reasoned revision instead of an unexplained architecture rewrite.

Interview phrase: "I will assume the core path is X, p95 latency target is Y, and we can tolerate Z seconds of staleness. If any of those are wrong, I would adjust the storage and caching choices."

How architecture evolves

A monolith describes a deployment boundary, not the number of servers: the same monolithic application can run on many machines behind a load balancer. Microservices separate deployment and ownership boundaries but add network calls, partial failures, and cross-service data coordination. Split a service when independent scaling, team ownership, or release isolation justifies that operational work. Merely adding users does not prove that a service split is needed.

Numbers to know show when pressure rises. This section shows common shapes teams move through as that pressure, team size, and ownership change. Products rarely jump straight to the most complex layout.

monolith

Single deployable unit. All code in one repository and process. Simple to develop, test, and deploy initially.

• Works well up to roughly fifty engineers before release coordination starts to hurt.
• Some large commerce products still run carefully managed monoliths at meaningful scale.
• Usually one shared database and memory space, which simplifies early transactions.
• Scale first by buying a bigger machine, then by running more identical copies behind a balancer.

Microservices

Independent services communicating over the network. Each team owns a service end-to-end.

• Large streaming platforms may run thousands of services with independent deploy cycles.
• Small service teams own interfaces end-to-end, which reduces cross-team merge friction.
• Services can deploy, scale, and fail independently, which shrinks some blast radii.
• You pay with network latency and harder distributed debugging across many hops.

Serverless / FaaS

Event-driven functions. No server management. Auto-scales from zero to millions of invocations.

• Managed function runtimes remove most server patching from the product team's plate.
• Pay per invocation instead of per idle hour, which fits bursty or sparse workloads.
• Cold starts can add about 100ms to 1s, so keep them off the tightest user paths.
• Best for event-driven and bursty workloads where idle capacity would otherwise waste money.

How large platforms evolve

CAP theorem and PACELC

Architecture evolution eventually puts data on multiple machines. That is when consistency, availability, and latency stop being slogans and become daily trade-offs.

Once data lives on multiple machines, networks can fail between them. During those failures, do you prefer to stop some requests until everyone agrees on the latest data (stronger consistency), or keep answering even if some reads are briefly stale (availability + responsiveness)?CAP gives vocabulary for that tension. PACELC adds: even when the network is fine, you still trade low latency vs strong consistency in normal operation.

In any distributed system, you can only guarantee two out of three properties simultaneously (this is a simplified teaching model; in practice, systems expose tunable knobs):

• C - Consistency: every read receives the most recent write so clients do not see conflicting answers.
• A - Availability: every request receives a response even when some replicas are hard to reach.
• P - Partition tolerance: the system keeps operating despite network failures between machines.

PACELC (extends CAP for real-world tradeoffs)

If Partition → choose Availability or Consistency (PAC). Else (normal operation) → choose Latency or Consistency (ELC).

Key insight: Most billion-user systems choose PA/EL (availability + low latency) and handle inconsistency at the application level with eventual consistency.

The trade-offs mindset

• Banking leans consistency so balances stay correct; social feeds often prefer availability with brief staleness.
• Real-time gaming needs low latency; batch pipelines need high throughput instead.
• More regions raise availability and cost; see Numbers to know for how extra "9s" shrink downtime.
• A monolith stays simpler to debug; microservices buy independent scale at the price of network complexity.

Distributed systems reality

The difficult failure is often uncertainty. A client sends a save request and times out; the server may have crashed before committing, or it may have committed and lost only the response. Retrying is necessary for progress but dangerous for non-idempotent work. Carry a stable operation identifier, record the result with the state change where possible, and return that result on a duplicate. This connects failure handling directly to API and data-model choices.

CAP names the tension. The day-to-day reality is messier: no shared memory, unreliable networks, drifting clocks, and failure domains that grow from one server to a whole region.

Three fundamentals

• No shared memory: processes on different machines coordinate only by messages, so shared state needs explicit protocols.
• The network is unreliable: messages can be lost, delayed, duplicated, or reordered, so design for partial failure and idempotency.
• Clocks are not synchronized: machines drift, so prefer logical clocks or specialized global time when order must be correct.

The 8 fallacies of distributed computing

These false assumptions appear when engineers first build distributed systems. At large scale, each one becomes a design constraint.

1. The network is reliable: packets get lost, and tiny failure rates become millions of failed calls at huge RPC volume.
2. Latency is zero: cross-continent RTT is 100-300ms, and a 10-hop service chain can add another 50-200ms.
3. Bandwidth is infinite: payload size matters, and compact binary formats can save petabytes versus chatty JSON.
4. The network is secure: use zero-trust patterns such as mTLS, and encrypt data at rest and in transit.
5. Topology does not change: servers come and go, orchestrators reschedule work, and discovery must track endpoints dynamically.
6. There is one administrator: thousands of engineers may deploy independently across platform, reliability, and product layers.
7. Transport cost is zero: serialization, compression, and encryption cost CPU, and one saved byte compounds daily.
8. The network is homogeneous: different protocols, hardware, clouds, and hybrid deployments are normal in large enterprises.

Classic list from Deutsch, Gosling, and others (often called the "Fallacies of Distributed Computing"). Kleppmann's Designing Data-Intensive Applications expands partial failures, timeouts, and idempotent retries in production.

Infrastructure: regions, AZs, and blast radius

• Region: geographic area with multiple data centers, used for sovereignty and user-local latency.
• Availability Zone (AZ): independent data centers in a region; deploy across three AZs for high availability.
• Data center: often 50,000-100,000 servers with redundant power, cooling, and network paths.

Large global fleets may span 40+ DC locations across continents, 100K+ miles of private fiber, software-defined routing between sites, and 1M+ running servers.

Key takeaway

• System design means choosing pieces and data flow so the product stays useful as load and failures grow.
• Scale changes the architecture; use Numbers to know instead of memorizing every row twice.
• Track p50/p95/p99 latency, not averages, and tie uptime goals to an explicit SLA.
• Evolve from monolith toward services or serverless when traffic and team size force it.
• Most large systems lean PACELC PA/EL and handle inconsistency in the application carefully.
• Every choice is a trade-off: name what improves and what gets worse before you draw more boxes.

References

• Brewer: CAP twelve years later (partitions force consistency vs availability per operation)
• Abadi: PACELC (extends CAP with latency vs consistency in normal operation)
• Kleppmann: Designing Data-Intensive Applications (replication, consistency models, practical trade-offs)
• RFC 2119 (normative MUST/SHOULD language used in many distributed specs)