System Design Delivery

Content Delivery Networks

In Tokyo at lunch, an Instagram carousel snaps into place. The images and JavaScript did not travel all the way from a US (the server that holds the authoritative copy) on every scroll. A nearby edge, a cache machine in the same city, answered most of those bytes in a handful of milliseconds.

That is the whole idea of a CDN (Content Delivery Network): keep safe copies of the same bytes near users, let those copies absorb traffic spikes so the origin stays calm, and only walk inward to the origin when a nearby copy is missing. The rest of this page is about the decisions that make that work: what to cache, for how long, how to name it, and how to survive the day the caches go cold.

Photo load journey

The Tokyo edge answers from its local copy in about 5 ms when fresh.

Regional hub

A regional hub (the second cache layer) fills the edge when it is cold.

Origin shield merges duplicate misses so origin sees one request, not hundreds.

Origin, the source of truth, answers only when every cache layer missed.

What actually happens

Slow the Tokyo scroll down and it looks like this. The browser asks for /photo.abc123.jpg. DNS (the lookup that turns a hostname into an address) steers it to a nearby (Point of Presence, the CDN's cluster of cache servers in that city). If the edge holds a fresh copy, that is a : the edge returns the bytes and origin never hears about it. If not, a makes the request walk inward: edge → regional cache → → . Each layer may store the object for the next viewer, so the second person in Tokyo asking for the same photo usually stops at the edge.

Day-to-day engineering is not "buy a CDN." It is choosing (the rule that decides when two requests are the same object), the (Time To Live, how long a copy may be served as fresh), how you invalidate copies you no longer want, and how you protect origin so a deploy or purge does not suddenly dump peak traffic onto your app tier. Every later section is one of those four decisions seen up close.

Try it: translate hit ratio into origin offload

Send traffic to one popular object. After the first miss, the edge serves copies until you purge. Longer TTL raises hit ratio for immutable assets; purges and deploys recreate miss storms.

Cache TTL for this object: 300 s

Edge PoP

Cold edge; the next request walks to origin.

0 hits

Origin

Origin answers misses and refills the edge.

0 origin fetches

Hit ratio: No traffic yet
Edge misses: 0
Origin fetches: 0
Total requests: 0

0 requests · 0% edge hit ratio · 0% reached origin · TTL 300s. The edge is cold. The first viewer for an object walks inward to origin; later viewers hit the edge if TTL has not expired.

Origin protection. At 0% hits, origin sees about 100% of viewer requests. A 10× traffic spike with 95% hits is roughly half a million origin requests per million viewers; at 80% hits it is two hundred thousand more than that.

How to choose CDN caching

Those four decisions collapse into three questions you can ask about any URL before it goes anywhere near the edge.

1. Can every user see the same response? Immutable assets (files whose content never changes once published) and public GETs are great at the edge. Personalized or auth-sensitive bodies need private caches, a careful design (Vary is the header that names which request headers change the response), or no edge cache at all.

2. How fresh must it be? Hashed filenames can live for months, because a new build gets a new name. Product pages may use a short plus (serve the old copy now, refresh it in the background). Mutations, meaning writes, should reach origin.

3. What happens on a cold day? Size origin for the miss percentage plus purge bursts, not for the happy path. Add before you blame the CDN vendor for an origin outage.

Simple rule of thumb

Ship immutable assets with hashed filenames and a long TTL. Use a short TTL and revalidation for semi-dynamic public reads. Add origin shield before debating a second CDN. Learn your vendor's and surrogate-key model (tagging objects so you can drop a whole group at once) early, because the failure section below is mostly about invalidation gone wrong.

PoPs, edges, and origin shield

Question 3 above assumed you know what sits between the browser and origin, so here is the path laid out as nested caches. A miss walks inward one layer at a time, and each layer answers in roughly the time shown (L1, L2, and L3 simply mean first, second, and third cache layer).

User (Tokyo)

↓ missEdge PoP (Tokyo)[L1 cache: ~5 ms]

↓ missRegional hub (Singapore)[L2 cache: ~30 ms]

↓ missOrigin shield (US-West)[L3 coalescing: ~80 ms]

↓ missOrigin server (US-East)[Source of truth: ~200 ms]

A is a metro footprint of edge caches. Vendors advertise city counts, but those change, so design from measured latency and hit ratio instead. The inside a PoP terminates TLS (the encrypted connection), serves cached bytes, and may run small edge functions. The regional hub is a second, larger cache that many edges share, so a miss at one edge can still be a hit for the region.

is the last cache before and acts as the consolidation layer: when hundreds of edges miss the same object at once, shield coalesces them into a single upstream fetch instead of hundreds of identical origin requests. Without it, one cold event can stampede the origin. The sequence diagram below shows the same walk for a single JavaScript file.

Diagram text: the exact Mermaid definition below preserves the nodes, labels, and connections. Enable JavaScript to view the rendered diagram.

Read the diagram definition

sequenceDiagram
    autonumber
    participant B as Browser
    participant E as Edge PoP
    participant S as Origin Shield
    participant O as Origin

    B->>E: GET /app.abc123.js
    alt Cache hit
        E-->>B: 200 from edge, Age header increases
    else Cache miss
        E->>S: Fetch object
        S->>O: One coalesced origin request
        O-->>S: 200 with Cache-Control
        S-->>E: Store and return
        E-->>B: Store and return
    end
    Note over E,O: Purge or TTL expiry makes edges revalidate later

Three outcomes cover everything in the diagram. On a , the user receives bytes from the local PoP, the Age header (seconds since the copy was fetched) keeps counting up, and origin sees no request. On a , the edge fetches from shield or origin, stores the object according to its cache headers, then serves the user. Revalidation is the middle case: when the expires or a marks the copy stale, the edge checks freshness with ETag and If-None-Match (a fingerprint of the content, and the header that asks whether it changed) or fetches a replacement.

Numbers to know: origin offload math

Request hit ratio and byte hit ratio answer different questions. If 99 small icons hit cache but one large video misses, request hit ratio can be 99% while most bytes still come from origin. Use request misses to estimate origin handler load and byte misses to estimate origin bandwidth. Shield caching and request coalescing can reduce origin work further, so measure the edge-to-shield and shield-to-origin boundaries separately.

The hit and miss walk becomes concrete when you translate (the share of bytes served from cache rather than from ) into origin QPS (queries per second) and bandwidth. A small miss percentage can still be a large number during peak traffic, which is why the worksheet works from peak rather than average.

Step	Calculation	Result
Asset requests	50M requests/day ÷ 86,400 × 6 peak	~3.5K peak QPS
Edge bandwidth	3.5K QPS × 250KB object	~875MB/s, about 7Gbps
95% byte hit	875MB/s × 5% origin bytes	~44MB/s origin bandwidth
99% byte hit	875MB/s × 1% origin bytes	~8.8MB/s origin bandwidth
Shared scenario	92% CDN byte hit on user-facing bytes	Origin handles 8% plus miss bursts

Read the table top to bottom. 50M asset requests a day is about 579 QPS on average, and a 6× peak factor lifts that to roughly 3.5K QPS; at 250KB per object the edge is moving about 875MB/s, close to 7Gbps. A 95% byte hit leaves origin with ~44MB/s, and 99% drops it to ~8.8MB/s. Those last savings come mostly from immutable URLs and shielded misses, and origin must still handle concentrated misses after a purge. The shared scenario in the last row is the one the failure timeline later on the page uses.

Two ratios matter, and they can disagree. Request hit ratio counts objects; byte hit ratio counts data volume. Large images and videos hitting the edge can push byte hit high while many tiny personalized API requests still reach origin (high byte hit, lower request hit). The reverse also happens: many small cached objects hit, yet one large uncacheable download dominates origin bandwidth (high request hit, lower byte hit). Track both during incidents. Low request hit with high byte hit may be fine for tiny API misses, but low byte hit means origin egress (the outbound bandwidth you pay for) and user latency can explode even when request counts look modest.

What is safe to cache, and how to key it

A cache key is a claim that two requests may share one response. If language changes the content, include language in that decision. If an authorization cookie changes it, either separate the cache safely or avoid a shared cache for that response. Ignoring a query parameter improves hit rate only when the parameter truly cannot change content or permissions. Test variants as pairs of requests to prove that key normalization preserves meaning.

The numbers above assume the edge is allowed to share a copy between users. Before putting anything at the edge, decide whether all users can see the same response and which headers or query parameters change it. The matrix covers the four common cases.

Content type	Policy	Why
Immutable static assets	Long TTL, versioned URL	Safest and highest byte hit ratio.
Public API GET	Short TTL, revalidation, stale-while-revalidate	Good for popular read-mostly data.
Personalized GET	Private cache or carefully varied key	Risk of data leak if auth/user is missing from key.
Authenticated mutation	Do not cache response unless explicitly safe	Writes should reach origin services.

Once a response is cacheable, the decides which requests share it. CDNs key responses much like HTTP caches: scheme, host, path, and query string matter. Two URLs that differ only by parameter order can accidentally cache as two entries for the same payload unless you normalize.

Key design (hit rate)

The default key is often scheme + host + path + full query string. The gotcha: /api?a=1&b=2 and /api?b=2&a=1 may be different keys for identical bodies, so the same payload is fetched and stored twice. Normalization fixes it:

Sort query parameters deterministically so reordered URLs collapse into one cache entry.
Lowercase the path and strip tracking parameters that never affect the response body.
Include only parameters that actually change the response; anything else fragments the cache.

Invalidation patterns

For immutable assets, publish a new content-hashed URL whenever bytes change and keep older versions available while clients still reference them. The new HTML points to the new asset, so a global purge is unnecessary. Mutable pages require a different policy: short freshness windows, validation, or targeted invalidation. A successful purge request is not necessarily simultaneous removal from every edge, so emergency removal and ordinary refresh need explicit propagation expectations.

Keys decide what is stored; invalidation decides how you get rid of it. Vendors offer four tools, and the last one is the one to lean on.

Purge URL: drop a specific path from edges (vendor APIs differ in propagation time).

Surrogate-key / tag: everything tagged product-123 without listing URLs.

Soft purge: mark stale but serve immediately while revalidation runs in the background.

Versioned URLs: /styles.a1b2c3d4.css is an immutable asset with a long , so a deploy needs no purge and cannot start a purge storm.

Pull, push, video, and edge compute

Everything so far assumed the edge fills itself on demand. That is one of two fill models, and the choice shapes how video, APIs, and code at the edge behave.

Pull vs push, and dynamic acceleration

A fetches on first miss: the first request may reach , later requests hit the edge until or , and there is little operational upload overhead. Most SaaS setups are pull. A uploads the catalog ahead of demand, pre-positioning popular objects during quiet windows so the first viewer still hits local disk; it costs more storage planning and is common for large media rollouts. Even "uncacheable" APIs gain from dynamic acceleration: TLS terminated at the edge, HTTP/2 multiplexing (many requests sharing one connection), persistent connections, and optimized backbone paths between PoPs cut latency even when responses are not cached byte-for-byte.

Video delivery and signed URLs

A signed URL proves that a request satisfies a time-limited access policy, but the CDN must validate that policy before serving a cached object. Decide whether signatures affect the cache key after validation; including every unique token can destroy reuse, while removing tokens before validation can expose private media. Segment-based playback also needs cache lifetimes suited to live manifests, which change frequently, and completed media segments, which can often remain immutable.

Streaming stacks split video into short segments referenced by a manifest (an index file listing the available bitrates and segment URLs), and CDNs cache those segments aggressively. The typical flow: the client loads the manifest (HLS or DASH, the two common formats), the manifest lists bitrates and segment URLs, and the player adapts quality to bandwidth while pulling segments from the nearest PoP. let edges enforce expiry without a round trip to your auth service for every segment, as long as the edge still validates the cryptographic signature.

http

https://cdn.example.com/video/episode1.mp4
  ?expires=1735689600
  &signature=<base64-encoded-signature>
  &key_id=public-key-1

Expiry plus the cryptographic check limits hot-linking (other sites embedding your video) and unpaid replay.

Edge compute and API caching

Running small handlers at the edge (provider edge functions and lightweight runtime hooks) can authenticate, rewrite, or personalize without a full origin hop. The trade-offs teams evaluate are CPU limits, regional data rules, and cold starts where applicable. For plain GET APIs, caching with a short TTL and trims repeated identical reads:

Cache-Control: max-age=10, stale-while-revalidate=60: fresh for ten seconds, then served stale while refreshing.
Send Vary when language or compression differs, so users never receive the wrong variant.
Use surrogate keys for entity-level purge where supported, so one product change invalidates cleanly.

Multi-CDN, media edges, security, and cost

Once one CDN works, the remaining questions are about resilience, extreme scale, and the bill. Each answer reuses the same primitives: hit ratio, shield, and fewer bytes.

Multi-CDN and failover

A second CDN can reduce dependence on one provider, but it introduces another cache population and configuration surface. A traffic shift may send users into cold caches and sharply raise origin demand. Before failover, verify equivalent cache keys, authorization behavior, origin access, and purge rules. Test recovery under the expected miss rate instead of assuming two providers automatically double resilience.

Splitting traffic across CDN providers improves resilience and pricing leverage; DNS or client-side routing chooses who serves each region. Very large consumer systems often run active-active (both CDNs live at once), steering per region or ISP (Internet service provider) based on live latency and error rates. Active-passive keeps a secondary CDN on DNS or health-check failover; watch DNS TTL versus detection time, because a long TTL delays the switch. Either way, keep as the central cache in front of , since several CDN footprints would otherwise multiply cold misses against your backend.

Dedicated media edge pattern

At extreme scale, renting generic CDN alone can dominate unit economics (the cost of serving each viewer), so some media platforms place cache appliances inside access networks (the ISP networks subscribers connect through) and optimize fill algorithms for their catalog. Edge appliances serve encoded media libraries close to subscribers, and much traffic stays inside the access network backbone. Treat it as a case study in design, not a default startup requirement.

18,000+

edge appliances

6,000+

access-network sites

Custom stack

custom cache appliances

Pre-position

Off-peak catalog sync

Security at the edge

CDNs absorb volumetric noise, terminate TLS, and run WAF rules (a web application firewall that blocks known attack patterns) before traffic reaches your origin. Combine them with rate limits, bot challenges, and hiding origin IPs behind allowlists.

DDoS absorption across a global anycast footprint, where many PoPs answer for one address.
WAF rules for OWASP-class patterns (the common web vulnerability list), plus per-route rate limits.
Origin cloaking, so clients never dial your datacenter directly and cannot bypass the edge.
Modern TLS and HTTP/2 or HTTP/3 where offered, for faster and safer connections.

Cost levers

You pay for egress bytes and requests, so smaller payloads and higher hit ratios beat heroic tuning of marginal CPU on origin. Measure both sides of the ledger, the CDN bill and the origin savings it buys: without offload origin egress is high, and after tuning most bytes are edge served.

Fewer bytes

Compress text with Brotli or gzip so every HTML and script response carries fewer bytes.
Serve modern image codecs (WebP, AVIF), which pack the same picture into fewer bytes.
Use adaptive bitrate video so players download only the quality their connection supports.

Higher hit rate

Fix the two hit-rate leaks from the keying section: unnormalized keys and unversioned filenames.
Keep origin shield in place; the mechanism section explains why misses then cost one fetch.
Give semi-dynamic APIs a short TTL plus stale-while-revalidate instead of skipping the cache.

Failure timeline, what still breaks, and the invariant

CDN failures are often self-inflicted: a broad , a new , or a traffic shift makes edges cold and suddenly sees traffic it was not sized for. Here is how that plays out minute by minute, what protects you, and the rule that must hold throughout.

Timeline: full-site purge

t=0      Steady state per Numbers to know: origin sized for the miss share plus small bursts
t=1m     Deploy runs broad purge (or new cache key fragments every object)
t=2m     PoPs go cold; miss traffic approaches peak edge bandwidth (see Numbers to know)
t=3m     Origin CPU/egress saturates; p95 for dynamic pages collapses
t=4m     Without shield: N edges × same miss = N identical origin GETs
t=5m     With origin shield + versioned URLs + staged purge: herd coalesces
Outcome  Purge is an origin-capacity event: design keys and invalidation to avoid it

What protects origin

Cold-cache event. In the shared scenario (see Numbers to know), origin is sized for 8% of bytes plus bursts. A cold rollout can temporarily multiply origin bandwidth by 5-10x unless and warmup collapse the duplicate misses.

Purge safety. Prefer versioned asset URLs and entity-level purge tags over full-site invalidation. Rate-limit purge jobs, and warm the homepage, app shell, and top media before shifting all traffic.

Degraded mode test. Simulate 10% miss traffic for five minutes and verify origin CPU, bandwidth, queue depth, and p95/p99 latency (the slowest 5% and 1% of requests). Confirm serves old safe content when origin slows.

What must never become false

A shared edge cache key must never serve another user's private response. Public objects may be slightly stale within the declared / revalidation contract. Origin capacity must survive the planned miss percentage plus a purge or cold-cache burst, not only the happy-path .

Wrap-up

You have seen the path, the numbers, the keying rules, and the purge that goes wrong. Use this section to rehearse the interview version, test yourself, and find the primary sources.

How to explain this in an interview

Start with the capacity hooks, then expect the follow-up questions.

Estimate bytes per second as requests per second times average object size.
Estimate origin offload as edge hit ratio times total traffic, then size origin for the remainder.
Protect origin from miss bursts with shield, request coalescing, and warmup before launch.

Follow-up questions to expect

Are URLs immutable and versioned, or will deploys need purges to refresh content?
Which headers vary the response (auth, language, device, compression), and does the key include them?
Can private API responses be cached safely at the edge, or only with private caching?
What is our fallback if the CDN or one PoP is degraded during peak?

Exercises

Using the Numbers to know scenario, size origin bandwidth at 95% vs 99% byte hit and decide whether origin shield is mandatory before launch.
Design cache keys for a product page that varies by logged-in vs anonymous and by language. How do you prevent leaking private JSON at the edge?
Choose purge-by-URL vs surrogate tags vs versioned filenames for a CSS deploy and a price update. Which one can cold-start the whole CDN?

References

RFC 9111 (HTTP Caching) and RFC 5861 (Cache-Control extensions: stale-while-revalidate and stale-if-error), the standards behind TTL and revalidation.
CDN vendor architecture guides (Cloudflare, Fastly, CloudFront) on origin shield, cache keys, and purge models.
OWASP guidance on caching and CDN considerations for sensitive content, plus the Cache Poisoning cheat sheet.

What to remember

Put identical bytes near users. Version static assets so you rarely need a global purge. Treat miss storms as an origin-capacity problem: shield, coalesce, warm, and measure both request and byte hit ratios. The best CDN design is the one that keeps the Instagram carousel, or your homepage, fast without waking your origin for every scroll.

Practice and further reading

Use these exercises to connect the lesson to a design decision. State your assumptions, trace what happens to one request, and explain the rule that must remain true even during a failure. That rule is the design's correctness invariant.

Focus for this lesson: Design edge caching around cache keys, Vary, signed access, origin shielding, purge consistency, range requests, and multi-CDN steering.

Explain the decision

How can an incomplete CDN cache key leak data?

Read the reasoning

If responses vary by authorization, tenant, cookie, locale, or encoding but the key omits that dimension, one user can receive another variant.

Apply it to a production scenario

A purge succeeds in most points of presence but stale private content remains in one. What is the safety path?

Read one approach

Use short authorization grants, origin-side access checks or uncacheable private responses, purge observability, and versioned object URLs for immutable assets.

Work through the numbers

A 10 TiB/day catalog has 96% CDN offload. How much logical traffic reaches origin?

Read the worked calculation

0.4 TiB/day, excluding revalidation, fills, multi-CDN duplication, and request overhead.

Reason through a failure

What must origin shielding prevent during an edge miss wave?

Read the recovery reasoning

Many edge misses for the same object must coalesce into bounded origin work rather than amplify into a thundering herd.

Sources and further study

Continue with Auto-Scaling.