Case study
Design a URL shortener
A bit.ly or tinyurl-style link lands in Slack. A teammate taps it on a phone. To them it is one click. Under the hood the system must find the long destination in a few milliseconds, issue a , and still count the click without making the user wait for analytics.
Creating links is relatively rare. Following them is constant. The design job is a tiny map that stays fast when redirects dominate, while expiry, abuse checks, and dashboards stay off the hot path.
What happens on a click?
Look up the short code near the user so most clicks skip the database.
Send the browser to the long URL while still allowing click counts.
Click event
Publish analytics asynchronously so the user never waits on dashboards.
On a miss, load the database once, then warm the cache for the next click.
Try it: redirect and create paths for a URL shortener
Redirects should stop at cache when possible. Creates draw from a pre-generated code pool so the write path avoids contending on random code generation.
- Client
- Edge cache
- Redirect API
- Cassandra
- Analytics queue
- Cache hits
- 0
- DB reads
- 0
- KGS pool codes
- 50
- Mode
- redirect
Read capacity
99% targetHit ratio 0% on simulated redirects.
Write path
KGSPool remaining: 50. Low pool triggers backfill workers.
Consistency
cache-asideStale cache entries expire via TTL; viral codes use request coalescing on miss.
Run a redirect (read-heavy) or create (write). Watch cache-aside vs KGS pool on the hot path.
Related building blocks
These notes explain pieces of the HLD in depth.
What actually happens
Those related notes are the building blocks. Here is the end-to-end story for create and redirect.
On create, the API validates the destination, draws a code from the , or checks a custom alias, then stores the mapping. An keeps retries from minting duplicates. Optional cache warm helps the first click, but create never waits for the analytics warehouse (ClickHouse).
On redirect, on the is the default path. A viral campaign can create a ; that is why hit rate, coalescing, and partition design matter as much as the schema. Analytics lag shows up on the dashboard, not on the user's click.
How to choose the redirect design
Once create and redirect are clear, three product choices shape the design.
1. Do you need accurate click counts? Prefer so browsers keep hitting your edge. Use 301 only when permanent SEO-style redirects matter more than analytics.
2. Can analytics wait? Always. Publish click events asynchronously so a broker or warehouse outage (Kafka or ClickHouse) never slows redirects.
3. Will one link go viral? Plan for a with local caches, replication, and coalescing, not only average math.
Simple rule of thumb
Optimize the read path first: → redirect → async click event. Keep create correct with validation, , and an on writes. Exact QPS and hit-rate targets live in Numbers to know.
With those choices set, size the system for millions of redirects and billions of mappings. Exact capacity math lives in Numbers to know; the diagram below is the map, then architecture layers unpack each box.
High-level design diagram
Scalable to millions of RPS and billions of links. Match the drawing to the layer sections next.
Treat this drawing as the map for everything below. The legend shows sync versus async edges; highlights call out the decisions the boxes encode.
User or API waits for a response
Published to Kafka; redirect does not wait
Redirect lookup (cache → DB)
Create / update mapping
Multi-AZ / multi-region copy
Design highlights (from the diagram)
Hard-to-guess codes
Prefer Base62 random codes from a KGS pool, not sequential IDs that invite scraping.
Read path first
CDN plus edge Redis with cache-aside keeps redirects in the milliseconds.
Write path
Validate, persist the mapping, then emit async lifecycle events.
Right stores
Cassandra (or similar) for mappings; ClickHouse for click analytics.
Stateless services
Scale redirect and create fleets independently behind health-checked LBs.
Availability target
Design for 99.99%+ global availability: redirects should survive analytics outages.
Walk the same left-to-right order as the HLD so each layer below maps to a box you already saw in the drawing.
Architecture layer by layer
Same order as the HLD: entry → gateway → services → async → storage → ops.
01Entry and routingGet the user to the nearest healthy region fast.
Get the user to the nearest healthy region fast.
Think of it like: Like airport signs that send you to the closest baggage claim, not the main hub every time.
Browser redirects, mobile clicks, and API-based link creation reach a healthy service region through DNS or anycast routing. A CDN serves dashboard assets and can cache redirect responses when the link’s update and expiry policy permits it. Regional load balancers distribute application requests; routing users closer helps only if the mapping and revocation information are available there as well.
Edge rate limits and filtering reduce automated link creation, abusive requests, and traffic floods before they exhaust the mapping store. Distinguish public redirect traffic from authenticated management APIs, and measure each separately. A popular legitimate code may receive far more traffic than its creator usually generates, so a creation quota should not accidentally disable all reads of that link.
02API gateway and edge cacheAuth, quotas, and a hot redirect lane.
Auth, quotas, and a hot redirect lane.
Think of it like: The front desk checks your badge, then looks up a sticky note on the wall before opening the filing room.
The gateway authenticates create, update, and delete operations using the account’s chosen credential scheme and enforces per-tenant quotas. Redirect requests usually have a simpler public lookup contract: resolve a code only if its mapping is active and unexpired. Administrative authorization belongs to the creator’s account, not to anyone who happens to know the short code.
A regional Redis or Valkey cache can hold hot mappings and separate rate-limit state. A cache entry includes status, version, and expiry rather than only the destination URL. On a miss, load the authoritative record and fill with an appropriate lifetime. Destination edits and revocations require invalidation or version checks; a high hit rate is not useful if cached redirects continue serving a disabled link.
03Core services (stateless)Split write and read so each path can scale differently.
Split write and read so each path can scale differently.
Think of it like: One clerk registers new nicknames; another clerk only answers “where does this nickname go?”
The create service validates the destination scheme, checks account policy, allocates a code, and conditionally inserts the mapping under a uniqueness constraint. A stable idempotency key lets a client recover the same code after a timeout. Custom aliases compete for the same published namespace and therefore need atomic uniqueness checks, not only an earlier “is this available?” query.
The redirect service reads the mapping and its policy state, chooses a redirect status and cache headers, and returns the destination. It emits click telemetry without waiting for analytical aggregation. Both paths can run as stateless fleets with health-checked routing, but they need different capacity plans: reads may be much more frequent, while writes require code ownership and durable state transitions.
Size the two paths separately: a read-to-create ratio such as 100:1 is an explicit example assumption, not a universal shortener constant. Keep analytical work off the redirect path, while checking the current allow/block decision needed to serve the link safely. Expensive scanning can run asynchronously before publication or update a cached decision according to the product’s policy.
04Async backbone (Kafka / Pulsar)Decouple clicks and housekeeping from milliseconds-critical redirects.
Decouple clicks and housekeeping from milliseconds-critical redirects.
Think of it like: The redirect clerk drops a ticket in a tray; warehouse workers process tickets later.
Publish committed link-created, updated, deleted, and expired events through a durable outbox or change log so cache and analytics consumers can replay missed work. Click events carry stable identifiers when deduplication matters; workers enrich permitted device, referral, and geographic information before storing aggregates. Partition lifecycle events by code to preserve the ordering of changes for one mapping.
Expiration changes the mapping’s status and invalidates its cached copies. Keep a tombstone or an equivalent permanently allocated-code record instead of casually returning a published code to an unused pool: old bookmarks, printed links, messages, and browser caches can outlive the owner’s account. Hourly and daily analytics can run later, with retention and loss policies separate from link identity.
05Data storage (polyglot)Right database for mapping vs analytics vs cache.
Right database for mapping vs analytics vs cache.
Think of it like: Phone book for lookups, spreadsheet warehouse for stats, sticky notes for what is hot today.
The mapping store owns code uniqueness, destination, creator, status, version, and expiry. A relational database or a distributed keyed store can serve this access pattern if its configuration supplies the required conditional writes and read behavior. Multi-region copies may lag, so specify how a just-created or revoked mapping is resolved before routing all reads to the nearest replica.
Use a columnar analytical store such as ClickHouse when large click-history scans justify it, and object storage for report exports, backups, and retained raw events. Redis accelerates repeated lookups but never becomes the only record of a published link. Keep code tombstones small if old analytics and owner metadata expire, so data-retention cleanup does not accidentally reassign historical public URLs.
06Observability and operationsYou cannot fix p99 redirect latency without tracing the read path.
You cannot fix p99 redirect latency without tracing the read path.
Think of it like: Speed cameras on the highway plus a control room watching every exit ramp.
Measure redirect latency distributions, miss traffic, database fallback latency, error rate, and the busiest individual codes. Trace a request identifier through gateway, cache, and mapping store to distinguish network delay from an expensive miss. If using a preallocated key pool, monitor both refill capacity and safely reserved inventory; a large total pool does not help if ownership records are unavailable.
Alert on service-level objective consumption, delayed revocation propagation, dead-letter queues, and scanner backlog. Run a cache-loss drill to verify bounded database fallbacks and an analytics-outage drill to verify the declared telemetry loss policy. Recovery must preserve mapping identities and disabled states. A fast redirect to the wrong destination is a correctness failure even when latency and uptime look healthy.
Interviewers expect you to split redirect traffic from create traffic. Capacity figures live in Numbers to know below; here focus on caches, correctness, and what must stay off the hot path.
Read path vs write path
Different load shapes, caches, and failure modes. Keep redirect fast; keep create correct.
Read path (redirect)
The illustrative redirect target is p99 below 100 ms, measured at the chosen service boundary. A 302 describes a temporary redirect, while 301 communicates a permanent move. Neither status alone guarantees that every click reaches the server: cache-control headers and client behavior matter. Choose redirect and caching policy together, and document whether analytics count edge-observed requests or all user navigations.
Read the diagram definition
sequenceDiagram
autonumber
participant C as Client browser
participant CDN as CDN edge
participant LB as Load balancer
participant GW as API gateway
participant EC as Edge Redis
participant R as Redirect service
participant DB as Cassandra
participant K as Kafka
C->>CDN: GET /x7K9m
CDN-->>C: cache miss (dynamic)
C->>LB: GET /x7K9m
LB->>GW: route
GW->>EC: GET short_code
alt cache hit
EC-->>GW: original URL
else cache miss
GW->>R: resolve
R->>DB: lookup by short_code
DB-->>R: long URL
R->>EC: SET with TTL
end
R-->>K: click-event (async)
GW-->>C: 302 redirect- Use on the so most clicks never touch Cassandra.
- Coalesce concurrent misses for one so a cold viral code does not stampede the database.
- Return 404 for unknown codes and 410 when expired so clients fail clearly.
- Never block the redirect on ClickHouse or malware scan; publish clicks after the hop.
301 vs 302
A browser or intermediary can reuse a cached redirect without contacting the shortener, so server-side analytics cannot observe that navigation. Permanent redirects encourage long-lived reuse; temporary redirects still need an explicit caching policy when fresh resolution matters. Balance load reduction against destination changes, revocation speed, and measurement accuracy rather than treating 302 as automatic revalidation.
Write path (shorten)
Creates are lower volume than redirects but stricter: blocklist checks, open-redirect guards, custom alias conflicts, and safe retries all happen before you store a mapping.
Read the diagram definition
sequenceDiagram
autonumber
participant App as API client
participant GW as API gateway
participant S as Short URL service
participant KGS as Key pool
participant DB as Cassandra
participant Cache as Redis
participant K as Kafka
App->>GW: POST /api/v1/urls
GW->>S: create (auth + rate limit)
S->>S: validate URL + blocklist
alt custom alias
S->>DB: check uniqueness
else
S->>KGS: next available code
end
S->>DB: INSERT mapping
S->>Cache: optional warm
S-->>K: link-created (async)
S-->>App: 201 + short URL- Require an on POST so network retries do not mint duplicate links.
- Buffer codes from the so a brief generator stall does not block creates.
- Emit link-created asynchronously so search indexing and cache warming never slow the create response.
Analytics path (async)
The redirect never waits for ClickHouse or rollups. Stale dashboards are acceptable; a slow hop is not.
Read the diagram definition
sequenceDiagram
autonumber
participant R as Redirect service
participant K as Kafka
participant W as Analytics worker
participant CH as Columnar store
participant AU as Rollup job
participant D as Dashboard
R->>K: click-event (async)
Note over R: 302 already returned: non-blocking
K->>W: consume event
W->>CH: append click row
W->>AU: trigger hourly rollup
AU->>CH: aggregate by short_code
D->>CH: query rollups
CH-->>D: stats for dashboardExpiration and retirement
Expiry should be enforced when resolving a mapping, with background cleanup reclaiming storage later. Invalidate cached mappings when a code is revoked or blocked, and bound any stale-serving policy. Avoid recycling a published short code merely after a cooldown: old bookmarks, emails, and caches can survive indefinitely and would then send users to an unrelated new destination. Keeping codes unique over time is the simpler safety contract.
Read the diagram definition
sequenceDiagram
autonumber
participant TW as TTL worker
participant DB as Cassandra
participant Cache as Redis
participant K as Kafka
participant KGS as Key pool
TW->>DB: scan expires_at < now
TW->>DB: UPDATE status=revoked
TW->>K: link-expired (async)
TW->>Cache: DEL short_code
opt retire allocated code
TW->>KGS: keep code retired; do not reassign
endBefore naming databases, tell the user story: create a link, open it, then read stats later.
What users experience
Tell this story in the interview before naming databases.
Create a short link
You: You paste a long URL and get back short.example/abc12.
System: Validate the URL, take a free code from the KGS pool so creates stay fast, store the mapping, and warm cache optionally. Analytics and scanning stay async so create does not wait.
A client opens the link
You: Browser opens the short URL and lands on the original site quickly.
System: Look up the active mapping near the user and return the chosen redirect status with an explicit cache policy. Count requests observed at the edge, recognizing that a cached redirect can bypass that observation. Publish analytics asynchronously so delivery of the redirect does not wait for reporting.
View stats (later)
You: Dashboard shows clicks, countries, referrers.
System: ClickHouse aggregates click events later so dashboards stay off the redirect hot path and never slow the user.
Lock scope with the interviewer before capacity math. Functional needs define product; non-functional needs define what must stay fast when traffic spikes.
Requirements
Clarify scope before capacity math.
Functional
- Shorten a long URL via API or UI so users can share compact links.
- Redirect a to the original URL on every click.
- Support optional custom aliases and expiry TTLs when product needs branded or time-bound links.
- Capture click analytics such as geo, referrer, and device for later dashboards.
- Allow revoke or delete so owners can kill leaked or abused links.
Non-functional
- Keep redirect latency low enough that users feel an instant hop.
- Target high availability (99.99%+) because broken redirects look like total outage.
- Persist mappings durably with no silent loss across retries and failover.
- Make short codes hard to guess so private destinations are not easily enumerated.
- Accept analytics lag on dashboards; never let analytics delay the redirect.
Numbers to know
Requirements set latency and durability bars; these rates are the traffic shape you design against.
Writes stay rare versus click traffic
Read bias forces cache-first design
Size for viral spikes, not averages
Keeps database lookups rare at peak
Plan from 100M creates per month and a 100:1 read:write ratio. That yields the card rates above (~39 creates/s, ~3.9K redirects/s average, ~39K/s at a 10× peak). A 99% cache hit target keeps peak database load near ~400 QPS instead of tens of thousands. Later sections should cross-reference this block rather than restating these rates. Mapping rows are about 264 bytes; see the capacity panel below for 1M / 100M / 1B storage tiers.
Case-study upgrade
URL Shortener Scaling Math
A URL shortener maps a small code to a long link. The redirect path must stay fast even if analytics, abuse checks, or dashboards are slow.
Assumptions and formulas
- Average original URL: 100 bytes lean, 500 bytes conservative, with a 2,048-byte accepted maximum for validation.
- Short code: 7 Base62 characters gives about 3.52T codes; 8 characters gives about 218T codes.
- TTL (Time To Live) is optional per link; expired links should not remain in hot cache.
Concrete math
- 100M URLs/month = 3.33M/day = 38.6 write QPS. With 100:1 reads, redirect average is about 3.9K QPS; 10x peak is 39K QPS.
- Lean row: id 8 + code 8 + URL 100 + user 8 + timestamps 24 + flags/hash/overhead about 116 = about 264 bytes. Conservative row with 500-byte URL is about 734 bytes.
- 1M URLs lean: about 264 MB raw, about 528 MB indexed, about 1.6 GB with 3x replication.
- 100M URLs lean: about 26.4 GB raw, about 52.8 GB indexed, about 158 GB with 3x replication.
- 1B URLs lean: about 264 GB raw, about 528 GB indexed, about 1.6 TB with 3x replication.
- Cache footprint: ~250 bytes lean / ~800 bytes conservative per cached mapping (URL + status + expiry + flags + overhead).
- Hot cache tiers: 1M URLs = 250-800 MB raw (~1-5 GB with overhead/replicas); 100M = 25-80 GB raw (~100-500 GB); 200M hot = 50-160 GB raw (~200 GB-1 TB).
- Viral single-key risk: one short code can receive 10K-100K QPS alone; needs local cache, replicated entries, and request coalescing.
- Analytics at 3.9K redirects/sec average and 100 bytes/event = 34.6 GB/day raw; 30 days is about 1 TB before replication.
- Analytics peak-day: 40K/s sustained = 3.46B events/day; at 100 bytes/event = 346 GB/day raw. Use hourly rollups for dashboards; tier or limit raw event retention.
- Backend QPS = redirect QPS x (1 - cache hit rate). At 39K peak and 99% hit rate, the database still sees about 390 QPS.
Design decisions forced by numbers
- Keep redirect mapping storage separate from raw click-event analytics.
- Redirect should publish analytics asynchronously; analytics outage must not break redirects.
- Hot links need local cache, replicated cache entries, stale-while-refresh, and request coalescing.
Scaling to millions
- At 1M URLs: one primary database, cache-aside, and async analytics are fine.
- At 100M URLs: shard by short code, keep rollups for dashboards, and limit raw event retention.
- At 1B URLs: handle viral keys, regional redirect caches, and background expiry cleanup.
Readable flow
Interview follow-ups to expect
- Do we need exact per-click analytics, or are hourly rollups enough?
- Should custom aliases be reserved strongly before returning success?
- What happens when one short code receives 100K QPS alone?
Operational readiness checklist
After the math, explain how this capacity plan is measured, protected, launched, and evolved.
Observability
- Attach each capacity estimate to a dashboard metric: p99 latency, queue age, cache hit rate, shard skew, error rate, and freshness lag.
- Alert on rate-of-change and saturation, not only hard failures; backlogs and hot keys usually warn before an outage.
Security and abuse
- Budget for auth checks, rate limits, quota isolation, audit logging, privacy filters, and abuse-review side paths.
- Separate trusted internal events from user-controlled input so retries and backfills cannot bypass policy checks.
Degraded modes
- Define the fallback when a dependency is slow: stale cache, partial results, read-only mode, delayed async work, or admission control.
- Protect the invariant first; reduce freshness, personalization, or analytics quality before risking duplicate or corrupt state.
Testing and rollout
- Test average, peak, and hot-object traffic; include retry storms, cold caches, lagging workers, and regional dependency loss.
- Use canaries, shadow reads, dual-write validation, feature flags, and explicit rollback criteria for risky path changes.
Rollout, backfill, and runbooks
- Make migrations and backfills idempotent, resumable, rate-limited, and visible through progress and error metrics.
- Write the runbook before launch: owner, dashboards, paging thresholds, mitigation switches, replay steps, and repair commands.
10x evolution
- For url shortener scaling math, state which number forces the next move: cache memory, shard count, write amplification, queue lag, egress, or failover reserve.
- Add the next layer only when the metric forces it: sharding, regional reads, async replay, hot-key isolation, or stricter backpressure.
Interview hardening checklist
After capacity math, show how you operate, degrade, and grow without breaking redirects.
With the rates in Numbers to know and the storage tiers above, this checklist ties planning numbers to SLOs, failure modes, tests, and a viral-abuse drill interviewers often push on.
SLOs and metrics
Hold redirect p99 under 100ms, create p99 under 300ms, and 99.99% redirect availability so clicks stay the product. Watch cache hit rate, database fallback QPS, hot-code skew, pool depth, blocked-destination rate, and analytics lag.
Degraded modes
If analytics, rollups, or abuse enrichment lag, keep redirects working and mark dashboards stale. If the is cold, coalesce misses and rate-limit viral keys before the database saturates.
Testing
- Load-test redirects with a Zipf hot-key mix so skew shows up early.
- Chaos-test cache loss and broker lag to prove redirects still return.
- Replay click events into shadow rollups so analytics rebuild safely.
- Canary key-generation changes by tenant before a full KGS cutover.
10x evolution
Move from one regional cache to edge-local caches, replicate hot mappings, run regional KGS pools, keep analytics async with backfill, and add per-destination abuse budgets with emergency block propagation.
Hard-part drill
A newly shared jumps to 100K RPS while abuse scanners are delayed. Treat it as a : serve redirects from replicated local cache, pin the mapping until traffic cools, sample click analytics, and let abuse flip a signed blocklist entry that invalidates cache globally within seconds.
With ops and 10x growth sketched, name the HTTP contracts interviewers expect you to write on the whiteboard.
APIs worth naming
/api/v1/urlsCreate a mapping with an API key and an so retries do not mint duplicates./{short_code}Public by ; the click path stays unauthenticated./api/v1/urls/{code}/statsReturn dashboard analytics for one code without touching the redirect hot path./api/v1/urls/{code}Revoke the link and invalidate cache so the next click cannot land.Creates and redirects stay synchronous; everything else (clicks, scans, expiry) must leave that critical path.
Async pipeline
Everything that must not slow down a redirect.
Async topics (Kafka / Pulsar)
Redirects stay sync; clicks and link lifecycle events flow async. Partition by short_code or user_id depending on ordering needs.
Producers
Services → broker
click-eventstopicRedirection service·Buffer with an explicit loss policy; durable append is needed for loss-sensitive analytics
link-createdtopicShort URL service·New mapping persisted
link-deletedtopicShort URL service·Revoke or expiry
Consumers
Broker → workers
click-eventstopicClick worker·Insert into ClickHouse, rollups
link-createdtopicSearch index / cache warmer·Optional side effects
link-deletedtopicExpire worker·Invalidate cached mapping and retain the published code as a tombstone
Outage contract: keep serving valid mappings when analytics is unavailable, but state whether click events remain in a durable spool, are sampled, or are dropped when a bounded buffer fills. Show incomplete or delayed analytics honestly. Revocation and expiration remain part of the redirect correctness path even when reporting is degraded.
Once the write path is clear, show how you mint unique codes without locking the create request.
Key generation
Base62 + ID
A unique numeric identifier encodes compactly without a hash collision. Keep identifier allocation unique across all writers.
Sequential identifiers remain predictable after Base62 encoding, so this choice does not provide an unguessable capability.
Use when public enumeration is acceptable or a separate authorization check protects access; scale is not the only deciding factor.
Hash + truncate
Hashing a normalized destination produces a repeatable candidate code, which can support intentional deduplication.
Truncation can collide, and two users may need different expiry, ownership, or analytics for the same destination. Resolve collisions with an atomic insert and an alternate candidate.
Choose only after defining whether deduplication is global, per account, or disabled; the URL text alone may not identify the product resource.
KGS pool
Pre-generated random codes move generation off the create path when every reservation is durably exclusive.
Pool refill, atomic batch reservation, crash recovery, and unavailable inventory become operating concerns. A plain SELECT does not reserve anything.
Use when measured creation volume or allocation policy justifies a pool. Random generation plus a unique constraint is often sufficient without another service.
7-character Base62 ≈ 3.5 trillion codes. At 40 creates/sec, exhaustion is not your first problem; hot redirects and storage cost are. Use on the read path so popular codes stay in edge Redis.
Prefer when you may change destinations later; reserve for mappings you are sure will never move. still helps either way by landing the first hop nearby.
// Illustrative allocation contract, not executable SQLappserver → KGS: reserve 1000 keys using stable reservation_idKGS → DB transaction: select available rows with exclusive ownershipKGS → DB transaction: mark selected rows reserved for reservation_idKGS → appserver: return the committed batch; retries return the same batch// Publish a mapping using a conditional insert tied to its reserved code.// Recover uncertain allocations before releasing any unpublished reservation.// Never return a previously published code to the ordinary unused pool.
Codes and keys only matter if the tables behind them match the read/write split you already described.
Data model (core tables)
url_mappings
Cassandra partitions on so each redirect hits one shard.
short_code PK
original_url
user_id, status
expires_at, created_atusers
Owns email and API key for authenticated create and delete calls.
id PK
email, api_key
created_atclick_events
Append-only ClickHouse rows keep click analytics off redirect latency.
short_code, ts
ip, referrer
country, devicekey_pool
Holds pre-generated codes so creates draw without minting under lock.
code PK
status available|usedSQL vs NoSQL: Redirects are point lookups by short code, so both indexed relational storage and a partitioned key-value or wide-column design can fit. Choose from measured write rate, mapping count, regional behavior, and operational needs. Enforce unique code allocation atomically, even for custom aliases. Billion-link storage volume alone does not determine the engine, and analytics should use a separate query path.
Schema alone will not meet the hit-rate and abuse goals from Numbers to know above; cache layout and security checks close that gap.
Caching, partitioning, and security
Caching and partitioning
Use on an (Redis): about 20% of codes drive about 80% of traffic. TTL 24h to 7d, LRU eviction, optional warm of top-N codes. A viral replicates the entry or uses a local pod cache. Hit-rate targets and peak redirect math live in Numbers to know above. See caching note.
Partition with hash() for even shards. Reads and writes for one code hit one shard. Multi-region async replication covers DR; the redirect path tolerates brief eventual consistency. Newly created mappings need a read-after-create plan if the redirect lands in another region before replication catches up. Route a miss to the mapping’s authority, carry a creation-region hint, or delay global success until the required replication boundary is met. “Eventual consistency” is not enough to explain a fresh link returning not found immediately after creation.
Security and expiry
- Rate-limit creates per API key and IP so abuse cannot flood writes.
- Block javascript:, data:, and internal IPs to stop open-redirect and SSRF.
- Scan destinations against malware and phishing lists so creates stay safe.
- Reserve aliases like api, admin, login, and www so routes never collide.
On access, compare the mapping expiry with the authoritative policy and return an expired response instead of redirecting. A background worker later removes obsolete storage and cache entries. Cleanup delay must not extend the link’s validity, and a cached destination must not bypass emergency block policy. Retain the code as retired rather than reassigning it to a different destination.
When traffic skews or a dependency stalls, say which failure stays user-visible and which you degrade safely.
Bottlenecks and degraded modes
Viral short code
Risk: One viral can absorb 100K+ RPS alone.
Mitigation: Replicate the mapping, coalesce misses, and keep a local LRU.
Cache stampede
Risk: Mass TTL expiry can stampede the database on one cold wave.
Mitigation: Refresh early with jitter and serve stale while revalidating.
KGS empty pool
Risk: Creates fail globally when the pre-generated pool runs dry.
Mitigation: Alert under 10% depth and pre-generate millions of spare keys.
Kafka lag
Risk: Dashboard analytics go stale while the broker lags behind.
Mitigation: Keep redirects online and scale consumers until lag drains.
Close the design by showing both the five-minute sketch and the deep-dive stack without inventing new components.
Beginner vs advanced answers
Beginner (5 min)
One table maps to long URL, cache redirects, mint random codes, and return a to the browser.
Advanced (deep dive)
Split read/write services, use a , Cassandra plus ClickHouse, a Kafka click pipeline, CDN plus , hash partitioning, replication, multi-region, and observability SLOs.
Choosing and explaining URL shortener
A complete explanation connects the mechanism to a requirement. Describe what improves when you introduce it, which new failure cases become possible, and how you would know that the trade-off is worthwhile.
Explain the core idea
Separate redirect (read) from shorten (write); optimize read first.
Cache-aside + edge Redis; Cassandra (or sharded SQL) only on miss.
Analytics and expiry async via Kafka; redirects must survive broker lag.
When it belongs
Classic FAANG-style design question with clear capacity math.
They probe how you handle viral links and cache misses.
They ask SQL vs NoSQL and 301 vs 302 tradeoffs.
How it fails
Viral code melts one Redis shard.
KGS pool exhausted; creates fail globally.
Open redirect to internal network via malicious long URL.
Trade-offs to weigh
302 vs 301: analytics accuracy vs fewer redirect hits.
KGS vs hash: randomness vs deduplication.
Strong consistency on create vs availability during partition.
Common misunderstandings
Storing click counts on the redirect critical path.
Single global SQL primary for billion-row table.
Skipping URL validation and reserved aliases.
Deeper walkthrough
URL shortener deeper walkthrough
A latency-critical redirect service with a separate write path and analytics pipeline. Redirects must work when analytics or scanning lags.
Requirement refinements
Custom aliases, expiry, private links, and QR codes in or out of scope?
Invariant: active code resolves quickly; expired/revoked codes never redirect.
API design details
Idempotent create; explicit alias_taken, blocked_destination, expired_link errors.
Public redirect endpoint separate from authenticated admin APIs.
Data model and storage
Partition by short_code; click events append-only in analytics store.
Status + expires_at + abuse_state instead of hard delete for audit.
Scaling strategy
Scale reads with CDN, edge cache, coalescing; scale writes with KGS batches and hash sharding.
Replicate hot keys; monitor cache hit rate and DB fallback QPS.
Reliability, recovery, and operations
Kafka down → redirects OK, analytics delayed with visible lag.
Alert: KGS depth, p99 redirect, cache hit %, consumer lag.
Deepening reference
URL Shortener: realistic production notes
Beginner mental model: A read-heavy key-value lookup with a very hot redirect path and an async analytics side path.
Production scope: At production scale, the redirect path must stay tiny: validate code, read cache or store, emit analytics asynchronously, and return 302. Abuse prevention, bot traffic, malware links, and hot viral links matter as much as key generation.
Choose what the first version must guarantee
Start by deciding which url shortener capabilities are essential. Describe the main read and write operations before choosing infrastructure. For each operation, identify the stored records it uses and the condition that makes its result correct. A fast response is useful only if it satisfies that condition.
Next, separate work the user must wait for from work that can finish later. Estimate requests per second, storage growth, and the busiest object or customer. Use those estimates to choose one bottleneck to investigate. The calculations below are example workloads with stated assumptions; they are not measured limits of a particular product.
Concrete capacity calculations
Row size: lean mapping row about 264 bytes (100-byte avg URL); conservative row about 734 bytes (500-byte avg URL). Use lean for capacity planning, conservative for worst-case storage.
Storage tiers: 1M URLs lean about 528 MB indexed, about 1.6 GB with 3x replication. 100M URLs lean about 52.8 GB indexed, about 158 GB replicated. 1B URLs lean about 528 GB indexed, about 1.6 TB replicated.
Keyspace: 7 Base62 chars gives about 3.52T codes; 8 chars gives about 218T. Collision handling matters more than exhaustion at interview scale.
Traffic: 100M links/month = 3.33M/day = 38.6 write QPS average. With 100:1 read:write, redirect average is about 3,860 QPS; 10x peak is about 38.6K QPS.
Cache math: backend QPS = redirect QPS x (1 - hit rate). At 38.6K peak and 99% hit rate, the mapping store sees about 400 QPS.
Analytics: 4K redirects/sec x 100 bytes/event = 34.6 GB/day raw; 30 days is about 1 TB before replication. Rollups serve dashboards; raw events need tiered retention.
Scaling to millions of users
Bottleneck is the redirect path, not link creation. Keep code lookup, expiry/status check, and redirect synchronous; move analytics, scanning, and dashboards to async workers.
Partition mappings by hash(short_code) so reads hit one shard. Avoid range partitioning by prefix because generated codes can cluster and viral links can create prefix hotspots.
Use a distributed cache plus small local hot-key cache on redirect servers. Add request coalescing so one cache miss loads a code while concurrent requests wait.
Replicate hot mappings across cache nodes when one link becomes viral. Store should be protected by rate limits, circuit breakers, and fallback read replicas.
At multi-region scale, route users to a nearby read region and replicate mapping changes asynchronously. Revocation and abuse blocks need faster propagation than analytics aggregates.
Sequence and data-flow diagram
Walk this path after the high-level design so it is clear what is synchronous, what is asynchronous, and where the system can degrade safely.
Read the diagram definition
sequenceDiagram
participant U as Browser
participant G as Edge or gateway
participant R as Redirect service
participant C as Cache
participant S as Mapping store
participant Q as Event pipeline
U->>G: GET /abc123
G->>R: Route short code
R->>C: Lookup code
alt Cache hit
C-->>R: Destination and status
else Cache miss
R->>S: Read mapping by short_code
S-->>R: Destination and status
R->>C: Fill cache with TTL
end
R->>Q: Emit click event async
R-->>U: 302 redirectCore flow to walk through
Create: validate long URL, apply abuse checks, allocate code, persist mapping, optionally warm cache.
Redirect: parse code, cache lookup, fallback to store, emit click event, return redirect.
Analytics: consume click events, dedup bots if required, aggregate by time bucket, user agent, referrer, and geography.
Production readiness
URL Shortener: ship, operate, and evolve
Now examine the same request flow during failures and deployments. Tie each operational control to the user behavior it protects.
Observability
Track url shortener p95/p99 latency, error rate, saturation, and user-visible freshness separately for read, write, and async paths.
Alert on leading indicators such as queue age, hot-key skew, cache miss storms, reconciliation mismatches, and error-budget burn.
Security and abuse
Enforce authentication, authorization, rate limits, input validation, audit trails, and privacy checks at the service boundary.
Treat abuse controls as a first-class path: throttling, quota isolation, moderation hooks, revocation, and safe handling of sensitive data.
Degraded modes
Name what still works if cache, search, ranking, stream processing, object storage, or a regional dependency is slow.
Prefer stale-but-safe reads, partial results, async retry, admission control, and feature disablement over corrupting core state.
Testing and rollout
Load test average, peak, and hot-object traffic; add failure-injection for retries, timeouts, backlog growth, and dependency outages.
Ship with canaries, shadow reads or dual writes when useful, rollback switches, schema compatibility checks, and replay validation.
Runbooks
Document dashboards, paging thresholds, owner actions, rollback steps, data repair commands, and customer-impact language.
Keep backfills and replays idempotent, resumable, rate-limited, and observable so repair work does not create a second incident.
10x evolution
Say which metric forces the next architecture change: shard count, cache footprint, write amplification, queue lag, p99 latency, or regional failover reserve.
Evolve one bottleneck at a time: partition by access pattern, isolate hot tenants, regionalize reads, add replayable async pipelines, then harden multi-region behavior.
Hard parts worth deepening
Avoid making analytics synchronous on the redirect path.
Prevent cache stampedes for viral links with request coalescing and hot-key replication.
Choose 302 when analytics accuracy matters; choose 301 only when permanent browser caching is acceptable.
Detect malicious destinations and phishing after creation with asynchronous scanning and revocation.
Failure modes and recovery
Cache outage increases database load; use read replicas, circuit breakers, and short-lived local cache.
Key generator pool runs low; pre-generate in batches and alert well before exhaustion.
Analytics pipeline lags; redirects continue while analytics freshness degrades.
A viral link becomes a hot partition; replicate the hot mapping and avoid shard-by-prefix designs.
Signals to monitor
Track redirect p99, the time within which 99% of redirects finish. Compare it with cache hit rate and database fallback requests per second: a falling hit rate can turn a fast lookup path into a database bottleneck.
Measure analytics lag separately so delayed dashboards are distinguishable from failing redirects. For malware revocation, measure both the number of blocks and how long it takes a block to reach every serving cache.
| Choice | Best when | Cost |
|---|---|---|
| Random pre-generated keys | You want non-guessable codes and low collision risk. | Needs key-pool service and monitoring. |
| Base62 sequence keys | You want simple deterministic generation. | Codes are predictable unless salted or permuted. |
| Cache-aside | Reads dominate and stale reads are acceptable for a short time. | Miss storms need protection. |
Try this scenario
Explain how you would keep redirects under 50 ms during a celebrity link spike while the analytics pipeline is delayed by 10 minutes.
Wrap-up
The interview angle, deeper walkthrough, and upgrade checklist above show how to talk the design. Use this card to lock invariants, rehearse a failure story, drill the math, and leave with one clean takeaway.
Invariants
An active must resolve to its intended destination quickly; an expired, revoked, or blocked code must never redirect. Analytics lag must not block redirects: clicks may be delayed, but mappings stay correct.
Failure timeline: viral + stall
Same story as the hardening drill: protect the redirect path first when one code spikes and creates stall.
T+0 Celebrity posts one short link; redirect QPS jumps toward the peak redirect rate in Numbers to know T+30s Single Redis shard for that code saturates; coalescing not yet on T+2m Cache miss storm hits Cassandra; p99 redirect blows past 100ms SLO T+5m KGS pool depth hits zero on create path; new shortens fail globally T+15m Hot-key replicas + edge cache restore reads; creates wait on KGS refill Outcome Read path and write path fail differently: protect redirects first
Exercises
- Peak redirects are 39k/s with a 99% cache hit (see Numbers to know). What is DB fallback QPS? If hit rate drops to 90% for a cold viral code, how many DB QPS appear?
- Choose 301 vs for a paid campaign link that must count every click. Defend the choice against CDN caching and the redirect invariant above.
- The is empty for 10 minutes. Design create degradation that keeps uniqueness without blocking redirects for existing codes.
References
Redirect semantics and high-scale read-path design.
Key takeaway
Draw the HLD, walk the layers, then nail two paths: read is + redirect + async click event; write is validate + KGS + durable mapping. Cite the capacity figures in Numbers to know, name the stores (Redis, Cassandra, ClickHouse, Kafka), and explain what still works when analytics falls behind.
Practice and further reading
Use these exercises to connect the lesson to a design decision. State your assumptions, trace what happens to one request, and explain the rule that must remain true even during a failure. That rule is the design's correctness invariant.
Focus for this lesson: Design key generation, alias reservation, redirect latency, malicious-link controls, deletion propagation, analytics, and regional ownership.
Explain the decision
How do you reserve a custom alias without races?
Read the reasoning
Use a conditional insert or unique constraint in the authoritative mapping store and return conflict without read-then-write races.
Apply it to a production scenario
A shortened URL is deleted for abuse but remains at the CDN. What changes?
Read one approach
Purge/version edge entries, keep short negative/security TTLs, check a fast revocation set, and audit propagation time.
Work through the numbers
Six base62 characters provide how many possible keys?
Read the worked calculation
62^6 ≈ 56.8 billion, though occupancy, predictability, custom aliases, and collision strategy reduce practical capacity.
Reason through a failure
What must redirect storage guarantee?
Read the recovery reasoning
One active alias resolves to one authoritative destination/version; replay, cache, and regional replication cannot resurrect a deleted or remapped unsafe target.
Sources and further study
Continue with Design Social Microblog Feed.