SwadhyAI Notes

All System Design notes

System Design Foundations

Regionalisation & Edge

Tokyo needs an Uber match; the product's main data center sits in Virginia. Crossing the Pacific for every dynamic call can add roughly 150–200 ms before the app even thinks, and users feel that lag.

Regionalisation is the answer to that lag: it places app servers, caches, and data copies closer to people. A (a network of caches in many cities) can serve static bytes at the (the cache nearest the user), but dynamic work, such as matching a rider to a driver, still needs compute sitting next to data.

One global request

1

/

Location-aware DNS or a shared address answers from nearby, so the trip starts short.

2

Cacheable bytes such as images and scripts return from the edge cache in milliseconds.

3

Closest healthy region

Dynamic requests continue to an app tier that sits beside a local data copy.

4

Primary when needed

Writes that need one authority go to the primary region or follow conflict rules.

What actually happens

The Tokyo story is not a quirk of one product. You can buy faster CPUs, but you cannot buy faster physics: light in a fiber cable still needs tens of milliseconds to cross an ocean, and every request goes there and back. A server that feels snappy to users in its own city therefore feels slow to users far away, and the gap grows with every extra round trip a single click triggers.

A global product usually makes two decisions to close that gap. First, it gets the user onto a nearby network: (DNS is the lookup that turns a hostname into an address; GeoDNS answers differently by location) or (one address announced from many places) steers the user to a nearby , a point of presence where the provider has equipment. Second, it decides what can be answered right there. Static files stop at the via a . Dynamic requests continue to a regional app tier that should sit next to a regional data copy, not an ocean away from the database.

That phrase, "next to a regional data copy," is doing a lot of work. The tempting shortcut is to put an app server in London and leave the database in New York. That often makes things worse, because one user click can become many database queries, and each query then pays the ocean round-trip time (RTT) on its own. The worked example below puts numbers on it, including what happens to the (the latency that the slowest 5% of requests still beat) you thought you had improved.

Background reading guide

Networking basics · start here

Beating the speed of light with geography

You can buy faster CPUs and more RAM, but you cannot buy faster physics. Data traveling across the globe takes time. Regionalisation places servers, databases, and content closer to users to reduce latency.

Helpful if you already know: Basic understanding of Client-Server architecture and Latency.

How to read this page

First-time readers

Read the definition above first, then follow sections top to bottom. Under each title, the In simpler terms box restates the idea before diagrams and tables.

Mental model

Ask: who opens the connection, who speaks first, and what happens if packets arrive late or twice?

When explaining

Lead with a simple definition, then add trade-offs, failure modes, and one small calculation.

Try it: see when geography beats bigger CPUs

Pick a user city and a deployment shape, then send requests. The model adds one ocean round trip per database query when compute stays in the US, and collapses to local RTT when reads are regional or static assets sit at the edge.

User location
Topology

User

Tokyo

~160 ms RTT to US primary

Path

App and database both in US East; every query crosses the ocean.

End-to-end latency

tail risk

800 ms

Requests sent
0
DB round trips
4
Local RTT (replica)
18 ms
Edge static RTT
8 ms

Tokyo · 800 ms modeled latency · 0 requests sent. Tokyo is asking a US-only API. Each of four database round trips pays the full Pacific RTT, so p95 latency explodes even when CPU is idle.

Do not multiply RTT blindly. One page with four sequential queries to a distant primary can cost four times the ocean delay. Regional replicas and edge caches remove different parts of that chain.

Where the milliseconds go

Here are the numbers behind the paragraphs above: first one server versus three, then the London shortcut slowed down step by step. Regionalising means deploying servers, databases, and networks into specific geographies (US East, Europe West, Asia Pacific) instead of one central location.

One global server vs three regional ones

Single global server in New York

  • A New York user sees roughly 10 ms of network time and notices nothing.
  • A London user pays about 80 ms per round trip before any work even starts.
  • A Tokyo user pays about 200 ms per round trip, and feels every one of them.

Servers in New York, London, and Tokyo

  • Each user is routed to the closest server: roughly 10–15 ms of network time.
  • Everyone stays fast for dynamic work, provided the data is copied nearby as well.

The shortcut that backfires: compute far from data

Count sequential remote dependencies, not only the user-to-server distance. If a regional handler performs four dependent database round trips to another continent at 80 milliseconds each, those calls add roughly 320 milliseconds before query execution. Independent reads can overlap, but transactions and data-dependent queries often cannot. Move the relevant data closer, combine queries, or route the whole operation to its data owner.

That last bullet is the catch. Imagine you put an app server in London for local users but leave the database in New York to save money. The setup is a user in London, an app server in London, and a database in New York about 80 ms of round-trip time (RTT) away. Loading a profile then turns into an N+1 pattern: one query for the user, then one more query per order.

The disaster (N+1 amplified)

  1. The user clicks "Load Profile"; the request reaches the London server in about 10 ms.
  2. The server asks the database for user info, crossing the ocean once: about 80 ms.
  3. It then fetches Order 1, Order 2, and Order 3 separately, paying 80 ms each time.
  4. The finished HTML travels back to the user in roughly another 10 ms.

Total wait: ~340 ms. Keeping compute and database together in New York would have been ~90 ms.

Takeaway: if you regionalise servers, replicate data too, for example with a (a regional copy that serves reads). A single HTTP request often becomes 10–20 database queries; those queries need a local network (<1 ms), not an ocean.

When is lag acceptable?

Before spending on any of the above, check whether your users would notice. Not every product needs expensive global infrastructure; match geography spend to how sensitive users are to delay.

High sensitivity

  • Multiplayer gaming: even 50 ms can break fairness, so servers must be spread across many regions.
  • High-frequency trading: milliseconds decide trades, so servers sit physically next to the exchange.
  • Voice / WebRTC (real-time calls in the browser): delay above ~200 ms makes people talk over each other.

Lower sensitivity

  • Batch jobs: nightly reports can wait seconds without anyone noticing the delay.
  • Async chat / email: a few hundred milliseconds of delivery delay is usually invisible.
  • Buffered video: the player downloads ahead, so startup and rebuffering matter more than path latency.

How to choose a regional shape

If your users sit in the high-sensitivity column, three questions decide which shape to build.

1. What problem are you solving? Lower (tail latency), higher availability, data residency (a legal requirement that certain data stay inside a geography), or local writes each point to a different design.

2. Can static bytes stop at the edge? Prefer a single region plus a when dynamic latency is acceptable and most bytes are cacheable.

3. Who accepts writes? keeps one writer. with several writers needs a conflict strategy: home regions (each user's writes owned by one region) or CRDTs (conflict-free replicated data types, structures that merge concurrent updates automatically).

The four shapes, simplest first

Separate active traffic from write ownership. Two regions can both serve users while each record has exactly one home region that accepts writes. Reads may use local replicas under an explicit freshness policy. This avoids concurrent ownership of the same record, but a user far from their home region may pay remote-write latency. Moving ownership requires a controlled handoff so the old and new owners never both believe they can commit authoritative writes.

Single region + CDN

Simplest. Great when static bytes can sit at the edge and dynamic latency is acceptable.

Active-passive

One region takes writes; a standby waits. Simpler data, but failover time and data loss need honest numbers.

Active-active read-local/write-primary

Users read nearby replicas, but writes go to one authority to avoid conflicts.

Active-active multi-writer

Lowest local write latency, but the highest conflict-handling and product-rule complexity.

check

Whichever active-active shape you pick, ask this: if a user posts in Region A and immediately reads from Region B, can they see their own update? If replica lag is 2 seconds during a spike, either route that user to the write region, read from the primary for a short window, or show a clear pending state.

Simple rule of thumb

Put static content on a CDN. For dynamic APIs, keep compute and data in the same region (the worked example above shows why). Start with one region until geography, residency, or availability forces a second; then prefer read-local / write-primary before multi-writer complexity.

Edge-to-region request path

The diagram traces one request through the shape the rule of thumb describes: steering first, then the edge, then a region only when the answer cannot be cached.

Static or cacheable?

Yes

No

CDNs, edge, and origin

The "Yes" branch of the diagram is the . For static files you do not need full application servers everywhere; you use a CDN with many in front of one .

: source of truth

Your compute or storage endpoint where master files and dynamic logic live. It sits in a specific region next to databases.

: the cache

Mini data centers operated by CDN providers, scattered in many cities at the edge of the network.

How a Paris logo request works

  1. The request for the logo reaches the Paris edge location first.
  2. On a cache hit, the edge returns the file instantly, in about 5 ms.
  3. On a miss, it fetches from the New York origin (~100 ms), stores a copy, then returns it.

This edge-only shape is best for assets, pages, and read-mostly APIs. It does not remove the need for origin consistency on writes, which is why the "No" branch of the diagram exists.

Regional failover: capacity math and runbook

Failover has two independent questions: where can traffic run, and which data is safe to serve there? A healthy spare application fleet is insufficient if its replica is behind or cannot become authoritative safely. State the recovery time objective, the tolerated loss of acknowledged writes, the promotion rule, and how the old writer is fenced. Then test the routing and data changes together.

Multi-region is not only about latency. Failover means moving a failed region's users onto the survivors, and those survivors need warm (already running) spare capacity and database headroom to absorb the displaced traffic. The example below measures load in QPS, queries per second.

1.2K peak QPS

Region A 540 QPS

45%

Region B 420 QPS

35%

Region C 240 QPS

20%

A fails → 540 QPS displaced

70% of failover

Region B now 798 QPS

30% of failover

Region C now 402 QPS

Check app replicas, cache nodes, DB, and network

Numbers to know

  • Normal split: Region A takes 45%, B 35%, and C 20% of 1.2K peak QPS.
  • Lose A: B and C absorb 540 extra QPS; at 70%, B needs about 798 QPS.
  • C takes the remaining 30% of the failover, so it needs about 402 QPS.
  • Warm reserve: run each survivor at or below ~60% normal utilization so failover cannot saturate it.

When you present this math, also quote (how long recovery may take) and (how much recent data you may lose), the DNS or traffic-steering delay, the cache trap described below, and whether writes stay single-region or become with conflict handling.

Runbook

The numbers only help if someone can act on them. A failover plan is credible when it names the traffic split, spare capacity, data-loss window, and the degraded mode users will see, in four steps.

  1. Detect and steer. Health checks, synthetic journeys (scripted fake-user checks), and operator confirmation decide when Region A is gone. Traffic steering must allow for DNS and client cache delay (resolvers keep serving the old answer for a while), not assume instant movement.
  2. Absorb load. Keep app, cache, database, and network below normal saturation so the survivors can take the displaced traffic (see Numbers to know above).
  3. Degrade deliberately. Disable expensive personalization, serve stale read models (precomputed views that may be slightly behind), queue non-critical writes, and route recent writers to their home region when matters.
  4. Prove recovery. Run quarterly region evacuation tests: fail traffic over, measure RTO and RPO, watch the cache-miss multiplier, validate data catch-up, then fail back without split-brain writes (two regions both believing they are the primary).

What still breaks: the cache warmup trap

Assume two regions each receive 10,000 reads per second and normally miss their local cache on 5% of reads. Each origin handles about 500 reads per second. If one region fails, the survivor receives 20,000 reads per second; at the same hit rate, origin demand doubles to 1,000. If unfamiliar keys raise misses to 30%, origin demand becomes 6,000 instead. Reserve capacity based on the recovery workload and use warming, coalescing, or shedding to control that surge.

Even with the capacity above, when a failed region's users move, surviving regions may receive both the extra request QPS and a cold-cache miss surge at the same time. Warm critical keys and throttle low-priority reads so database failover capacity is not consumed by refills.

Wrap-up

You have seen the physics, the shapes, and the failover math. Here is how to say it out loud, what to keep, and where the primary sources are.

How to explain this in an interview

  • A CDN serves static, cacheable bytes; regional compute beside nearby data serves dynamic requests.
  • Compute far from its data can be slower than centralizing both, because every query pays the trip.
  • Active-active needs a conflict strategy: last-write-wins (newest timestamp wins), CRDTs, a home region, or one writer.

Likely follow-ups

  • Where do a user's writes go if they travel between regions mid-session?
  • What and (data-loss window and recovery time) do we promise during an outage?
  • Which data is legally required to stay inside a particular geography?
  • How do we warm a new region without overloading the primary region?

What to remember

Static content belongs on a . Dynamic APIs need regional app servers and nearby data copies so compute stays close to data. Choose or read-local/write-primary before multi-writer , and plan failover capacity, not only happy-path latency.

Primary sources

Practice and further reading

Use these exercises to connect the lesson to a design decision. State your assumptions, trace what happens to one request, and explain the rule that must remain true even during a failure. That rule is the design's correctness invariant.

Focus for this lesson: Place compute and data by latency, residency, write ownership, replication lag, conflict policy, and regional failure requirements.

Explain the decision

Why is active-active compute easier than active-active writes?

Read the reasoning

Stateless traffic can shift freely, but concurrent writes require ownership, ordering, conflict resolution, uniqueness, and failover fencing.

Apply it to a production scenario

DNS still sends 15% of clients to a failed region after failover. What protects users?

Read one approach

Health-aware edge routing, client retry to alternate endpoints, drained origin paths, short but realistic TTLs, and capacity reserved in surviving regions.

Work through the numbers

At 80,000 writes/s with 1.5 KiB records replicated to two remote regions, estimate logical inter-region egress.

Read the worked calculation

80,000 × 1.5 KiB × 2 ≈ 234 MiB/s before log, protocol, compression, and retransmission overhead.

Reason through a failure

What invariant prevents two regions from both becoming the writer after partition?

Read the recovery reasoning

A quorum-backed epoch or fencing token must be acquired and enforced by storage before a region accepts authoritative writes.

Sources and further study

Continue with Load Balancing.