Trade-Offs in System Design

How to choose under pressure

Trade-off decision loop

A useful comparison holds the workload and required guarantees constant. Compare two ways to serve the same profile page: reading the database on every request, or caching the result for 30 seconds. The cache saves repeated queries but creates a period during which an edit may not appear. Decide whether that delay is acceptable, how the editing user sees their own change, and what happens when the cache is empty. Without a shared scenario, “fast” and “consistent” remain vague labels.

The mistake is treating trade-offs as a list to memorize. Start from a requirement, compare at least two options, choose one for this scenario, then name the risk and mitigation.

Reusable sentence pattern

"Given requirement X, I am choosing A over B. We gain Y, but accept Z; I would monitor M and mitigate with N."

How to use this pattern

• Interview: When you pick a store or cache policy, name one clear downside you accept.
• On the job: Document TTL assumptions so teammates know the freshness bar.

Same problem, different constraint

A notification system can look very different under each constraint. Scan the matrix below when you need a quick "improves / worsens / metric" template for cache or queue choices.

Trade-off matrix template

Vocabulary that sounds precise

Instead of saying "this scales," name which quality improves and which gets worse. Keep these words near every architecture choice.

• freshness: how old returned data can be before users suffer.
• durability: can acknowledged writes survive disk or zone loss?
• Blast radius: how many users fail when one component breaks?
• Saturation: which resource hits its limit first under load?
• back-pressure: how producers slow when consumers fall behind.
• Operational cost: who runs, debugs, pages, migrates, and pays?

Interview sentence: "The bottleneck is not API CPU; it is write amplification in the storage layer. I would reduce fanout, batch work, and monitor queue age plus p95 write latency."

Consistency vs availability

Be precise about which operation needs which guarantee. A checkout service may serve an old product description during a network problem but must coordinate the last unit of stock before confirming its sale. One product can therefore use different consistency policies on different paths. Strong consistency here means a defined ordering guarantee such as linearizability, not simply that replicas normally catch up quickly.

consistency (in interview speak) often means every reader sees the latest write quickly. availability means the system keeps answering even when parts fail. When networks glitch between replicas, designs force a choice: wait for agreement (stronger consistency, risk timeouts) or respond from the best local copy (more availability, risk staleness). Neither is morally superior; it depends on whether wrong data is dangerous or merely annoying (see the hero checkout vs likes framing).

Latency vs throughput

Suppose each database batch has 5 milliseconds of fixed overhead plus 1 millisecond per item. Processing one item at a time takes 6 milliseconds per item, about 167 items per second for one serial worker. A batch of ten takes 15 milliseconds, about 667 items per second, but the first item may wait for nine more to arrive. A maximum batch age bounds that waiting time while a maximum batch size bounds memory and transaction work.

latency is how long one unit of work waits; throughput is how many units finish per second. Optimizing for one often hurts the other. Batching increases throughput but adds latency because items wait to fill a batch. Highly parallel pipelines raise throughput but add coordination overhead and can hurt tail latency if one stage stalls.

Concrete linger timings and the cost/latency ladders live in Numbers to know below.

linger.ms=0   → 1 msg/request, lowest latency, poor throughput
linger.ms=100 → batch 100ms of msgs, 10x throughput, +100ms latency

CAP and PACELC theorems

CAP is a teaching lens for distributed databases, not a literal dipstick you hold against every microservice. Most real systems blend patterns: sync writes to a primary, async replicas, caching layers with TTL. Still, interviewers expect CAP/PACELC language when comparing highly available stores vs strongly consistent SQL.

CAP THEOREM:
In a distributed system during a network partition (P), you must choose between Consistency (C) and Availability (A).

CP Systems: Refuse requests if unsure of consistency → coordination stores, strongly consistent databases
AP Systems: Always respond, even with potentially stale data → wide-column stores, key-value stores, DNS

PACELC extends CAP:
If Partition → choose A or C. Else (normal operation) → choose Latency or Consistency.

• CP coordination store: refuses writes without quorum so cluster metadata stays correct.
• AP wide-column store: stays writable with tunable ONE/QUORUM/ALL consistency levels.
• PA/EC document store: serves secondary reads in partitions; primary reads stay consistent.

Design levers: reads, writes, caches, and stage

Trade-off triangles

Many choices pull three qualities against each other. Pick the two that matter most and explain the third cost.

Read vs write optimization

An index is a concrete example of moving cost between paths. It lets a read jump to matching rows, but every insert or indexed-field update must also maintain the index. A precomputed feed goes further: each new post may update hundreds of followers’ feed entries, making later feed reads cheap. If most followers never open the app, much of that write work is wasted. Estimate both useful reads and amplified writes before choosing precomputation.

Caching strategy comparison

TTL extremes and hit-ratio math are in Numbers to know; this table compares freshness versus complexity.

Simplicity vs scalability

Wrap-up

Every design decision is a trade-off

At billion-user scale, there are no perfect solutions, only choices with side effects. Large marketplaces have measured that extra latency hurts revenue; large search products have reported latency hurting usage. Those stories are reminders that engineering decisions affect users and dollars, not arguments that your weekend project needs global CDN edge POPs on day one. Pick trade-offs that match your stage: MVP, growth, or hyperscale (cost ladders in Numbers to know).

Lightweight ADR pattern for interviews

Record the condition that would make you revisit the decision. “Use a cache” is incomplete; “cache public profiles for 30 seconds while the database can survive the measured miss rate; revisit if edit visibility or miss traffic breaches its target” is reviewable. This prevents a reasonable early choice from becoming an unexplained permanent constraint. The same record should include the fallback if the new component fails.

When the discussion gets fuzzy, state the decision as a mini ADR (Architecture Decision Record) so the trade-off is explicit.

Decision:
Use cache-aside with 60s TTL for product reads.

Context:
Reads are 20x writes and p95 latency matters more than perfectly fresh inventory counts.

Alternatives considered:
1. Direct DB reads only
2. Write-through cache

Why this choice:
Cache-aside cuts read latency and DB load immediately.

Accepted cost:
Users may see stale data for up to 60s.

Mitigation and metric:
Invalidate on critical writes; monitor hit ratio, stale-read complaints, and DB read QPS.

Common interview trade-off questions

1. State both sides clearly so the interviewer hears a real comparison.
2. Explain when each option is preferred for the stated requirements.
3. Give one concrete example for each side without inventing new products.
4. Recommend one choice for this scenario and name the accepted cost.
5. Mention the metric you would monitor to validate the choice still fits.

Design decision drills: why not the alternative?

• Storage: Why relational database over key-value store for orders?
• Cache: Why cache-aside instead of write-through for profiles?
• Queue: Why async queue instead of synchronous provider call?
• API: Why cursor pagination instead of offset pagination?
• Consistency: Why eventual for likes but strong for balances?

Interview tip

Never present a single solution. Always discuss at least two approaches with their trade-offs, using the reusable sentence in the decision loop above. Reason about trade-offs at every layer, from database choice to caching strategy to API design.

Primary sources

• Brewer: CAP Twelve Years Later: trade-offs under partition.
• Abadi: PACELC: latency vs consistency when healthy.
• Google SRE: Embracing Risk: error budgets as explicit trade-offs.