System Design Case Study
Design a distributed rate limiter
The edge graph spikes: a ChatGPT-style API or Instagram-style like endpoint is getting hammered. One noisy tenant, one buggy retry loop, or one botnet can starve everyone else and burn expensive downstream cost.
A rate limiter is a fast yes-or-no gate. For every request it asks: who is calling, how much budget remains, and whether to allow, delay, or reject with .
What happens first?
User, IP, API key, tenant, route…
Burst vs smooth vs accurate.
Shared store decides allow/deny.
429 + headers, or queue later work.
Try it: distributed rate limiter decisions
Each request asks a centralized or sharded limiter. Exceeding the budget returns 429 without touching the expensive API.
- Client
- Gateway
- Limiter
- API
- Store
- Allowed
- 0
- Rejected
- 0
- Mode
- token-bucket
- Path hop
- 0
Budget
per key0 allowed, 0 rejected.
Placement
gatewayDecide before the expensive path whenever possible.
Consistency
approxSharded limiters may slightly over-admit under race; choose accuracy vs latency.
Send requests until the budget trips, and compare token-bucket vs sliding-window framing.
What actually happens
A request hits the API gateway (or app middleware). The gateway extracts a , matches a rule for that route and plan, then asks a shared counter store for an .
If allowed, the request continues and response headers show remaining budget. If denied, return 429 with . Background jobs might queue instead of drop. When the counter store is down, each route chooses or , not one blunt global switch.
At interview scale you often hear 1M+ QPS across thousands of API servers. That forces a fast shared store, careful memory math for counters, and honesty about local pre-limit overshoot.
How to choose the limiter design
1. What are you protecting? Abuse and fairness need strict keys. Expensive payment or model calls need hard limits. Cheap product reads may tolerate soft bursts.
2. Burst or smooth? Use when short spikes are fine. Prefer leaky-bucket style when you must smooth output. Use when you want a good accuracy/memory balance.
3. What if Redis is down? Read-heavy browse may . Login, rewards, and paid third-party routes should or use a strict local fallback.
Simple rule of thumb
Start with key → algorithm → atomic shared counters → 429 headers. Change only when the product needs something specific: bursts, multi-region approximate quotas, or per-route failure policy.
In simple terms: do not pick an algorithm because it sounds advanced. Pick it because it matches burst needs and failure risk.
Case-study upgrade
Distributed Rate Limiter Math
A rate limiter decides if a request should pass before expensive work happens. The hard part is shared counters, hot identities, and failure policy.
Assumptions and formulas
- Throughput target: 1M QPS. Counter operation request plus response: about 500 bytes.
- Rules create active counters by user, client, route, method, region, or tenant.
- Exact sliding logs are costly; approximate windows are usually enough for high-volume limits.
Concrete math
- Central counter traffic: 1M QPS x 500 bytes = 500 MB/sec before replication.
- Counter memory: key 40 to 80 bytes + value 16 bytes + TTL metadata/overhead 50 to 100 bytes = 100 to 200 bytes/counter.
- 50M active counters/window = 5 to 10 GB raw; with replicas plan 15 to 30 GB.
- Exact sliding log: 1M QPS x 60 sec = 60M timestamps. At 16 bytes raw, that is 960 MB before sorted-set overhead.
- Local pre-limit overshoot = gateways x local allowance. 500 gateways x 100 local tokens can overshoot by 50K.
Design decisions forced by numbers
- Use local pre-limits for latency, but keep small buckets for strict tenants.
- Shard active counters and split hot keys by route or region when needed.
- Choose fail-open or fail-closed per API (Application Programming Interface) risk.
Scaling to millions
- At 1M users: shared counter store and fixed/sliding windows are enough.
- At 10M users: add rule compilation, active-counter TTL cleanup, and tenant sharding.
- At 100M+ users: add regional approximate limits, global reconciliation, hot-key splitting, and store latency SLO.
Readable flow
Interview follow-ups to expect
- What happens if the limiter store is down?
- How much overshoot is acceptable?
- Which limits must be globally strict and which can be regional?
Operational readiness checklist
After the math, explain how this capacity plan is measured, protected, launched, and evolved.
Observability
- Attach each capacity estimate to a dashboard metric: p99 latency, queue age, cache hit rate, shard skew, error rate, and freshness lag.
- Alert on rate-of-change and saturation, not only hard failures; backlogs and hot keys usually warn before an outage.
Security and abuse
- Budget for auth checks, rate limits, quota isolation, audit logging, privacy filters, and abuse-review side paths.
- Separate trusted internal events from user-controlled input so retries and backfills cannot bypass policy checks.
Degraded modes
- Define the fallback when a dependency is slow: stale cache, partial results, read-only mode, delayed async work, or admission control.
- Protect the invariant first; reduce freshness, personalization, or analytics quality before risking duplicate or corrupt state.
Testing and rollout
- Test average, peak, and hot-object traffic; include retry storms, cold caches, lagging workers, and regional dependency loss.
- Use canaries, shadow reads, dual-write validation, feature flags, and explicit rollback criteria for risky path changes.
Rollout, backfill, and runbooks
- Make migrations and backfills idempotent, resumable, rate-limited, and visible through progress and error metrics.
- Write the runbook before launch: owner, dashboards, paging thresholds, mitigation switches, replay steps, and repair commands.
10x evolution
- For distributed rate limiter math, state which number forces the next move: cache memory, shard count, write amplification, queue lag, egress, or failover reserve.
- Add the next layer only when the metric forces it: sharding, regional reads, async replay, hot-key isolation, or stricter backpressure.
Scope you should nail down
Functional
- Limit by user_id, IP, API key, device, tenant plan, and endpoint
- Multiple windows: per second, minute, day, week
- Configurable rules: limit, window, key type
- Return 429 with Retry-After; expose remaining and reset
Non-functional
- 1M+ QPS across cluster; ~10K API servers
- ~1M unique users, ~100 rules
- Add <1ms latency on the decision path
- Consistent across servers; limiter outage must not take down all APIs blindly
Why use it?
- Protect from DoS, bots, retries, and client loops
- Control cost for payments, credit checks, model calls
- Preserve fairness across tenants
What are we limiting?
- User actions: posts/sec, likes/sec, reward claims/week
- Abuse surfaces: account creates/day by IP or device
- Plan quotas: free, pro, enterprise, internal bypasses
Where should it live?
- Client-side limits are advisory only
- API gateway for common auth, route, IP, and API-key limits
- App code for domain rules like checkout or reward abuse
Capacity picture
Store traffic: 1M decisions/sec × 500 bytes round trip ≈ 500MB/sec to the counter store before replication and retries.
Counter cardinality: 1M clients × 100 rules = 100M possible counters; active counters per window are the real sizing input.
Active-window memory: 50M active counters × 150 bytes ≈ 7.5GB raw; 3 replicas make it about 22.5GB before allocator overhead.
Sliding log warning: exact timestamps at 1M QPS for a 60 sec window = 60M timestamps × 16 bytes = 960MB raw, but sorted-set overhead can turn this into many GB.
Local pre-limit overshoot: 10K gateways × 100 local tokens = 1M possible extra requests before central sync; high-risk routes need smaller local buckets.
APIs worth sketching
isAllowed(client_id, endpoint, timestamp) → boolean
// Returns true if allowed, false if rate limited. Increments counter if allowed.
getRateLimitInfo(client_id, endpoint) → { limit, remaining, reset }
// Current rate limit state without consuming. Used for X-RateLimit-* headers.Core algorithms, explained simply
Every algorithm answers the same question with different memory and burst behavior. Expand a card when you want the update formula.
A token bucket has a maximum balance and refill rate. For capacity 20 and refill five tokens/s, an idle client can make a 20-request unit-cost burst and then sustain about five requests/s. Atomically refill from elapsed time, cap at 20, and subtract the request cost if enough remains. Tokens do not refill faster because the client sent more traffic.
tokens = min(capacity, tokens + (now - last_refill) * rate)
if tokens >= 1:
tokens -= 1
return ALLOW
return DENYLeaky bucket
Requests drain at a fixed leak rate. Overflow is rejected. Smooth output, little or no burst.
leaked = (now - last_leak) * leak_rate
queue = max(0, queue - leaked)
if queue < capacity:
queue += 1
return ALLOW
return DENYFixed window counter
Count requests in a fixed window (for example 1 minute). Simple, but 100 at 0:59 plus 100 at 1:00 can mean 200 in one second at the boundary.
window = floor(now / 60)
key = "client:" + client_id + ":" + window
count = cacheStore.INCR(key)
if count == 1: cacheStore.EXPIRE(key, 60)
return count <= limitSliding window log
A sliding log stores timestamps of admitted requests within the recent interval. Before deciding, atomically remove expired timestamps, count the remaining entries, and add the new request only if the limit permits it. This enforces the stated rolling-window rule closely, but storage grows with requests in the window. A hot key also makes pruning and counting part of the critical path.
cacheStore.ZADD(key, now, now)
cacheStore.ZREMRANGEBYSCORE(key, 0, now - window_sec)
count = cacheStore.ZCARD(key)
return count <= limitA sliding-window counter keeps the current bucket and an overlap-weighted portion of the previous bucket. It uses bounded state per key, but estimates how earlier requests were distributed within the previous interval. If those requests arrived in one burst, the estimate can admit too many or reject too early. Choose it when that approximation is acceptable and explain the expected error behavior.
// prev_count = requests in previous window
// overlap = (window_end - now) / window_size
estimated = prev_count * overlap + curr_count
return estimated <= limitArchitecture in one glance
Rules load from config DB into gateway memory. The counter store holds runtime counts only.
Making it distributed
Centralized shared store
A shared counter service gives gateways one decision point per quota key, provided each check-and-update is atomic. Sharding spreads different keys, but does not split a single strict hot quota without additional allocation rules. Replicas may improve durability or recovery; reading stale replicas to decide admission can overspend the budget. The latency depends on placement and load, so benchmark the round trip rather than assuming one millisecond.
Local + sync
Each server keeps local counters and syncs periodically. Lower latency, eventual consistency. Risk: the same user can exceed the global limit by hopping servers.
-- Lua script for atomicity
local count = cacheStore.call('INCR', KEYS[1])
if count == 1 then
cacheStore.call('EXPIRE', KEYS[1], ARGV[1])
end
if count > tonumber(ARGV[2]) then
return 0 -- DENY
end
return 1 -- ALLOWCounter creation, increment, expiry, and the admission decision should form one atomic operation. If a client increments and then crashes before setting expiry, a fixed-window key may remain active indefinitely. A script avoids that local race, but its replication and failover behavior still determines whether recently consumed quota can be lost. Keep operations bounded so one large script does not stall unrelated keys.
Rules engine
Define whether rules compose or override one another. Many products require a request to satisfy a platform cap, tenant quota, and route limit together; choosing only the most specific rule would accidentally bypass the broader cap. Compile a versioned rule set for fast matching at the gateway, and attach its version to decisions. Test policy changes in observation mode before enforcing a new quota.
// JSON rule format
{ "endpoint": "/api/*", "limit": 100, "window": 60, "key": "user_id" }
{ "endpoint": "/api/search", "limit": 20, "window": 60, "key": "ip" }
// Key extraction: user_id from JWT, ip from X-Forwarded-ForWhen the counter store fails
Allow all requests if the store is down. Availability over strict limiting. Risk: abuse during the outage.
Reject when the store is down. Use when the limit is load-bearing (payments, login abuse).
Local fallback
A local fallback can preserve limited service during a counter outage, but its safety depends on how allowances were allocated. Giving each of 100 replicas the full global quota can admit 100 times too much. Preallocate bounded shares or leased token batches and stop when they are spent. Decide how scale-out and disconnected replicas affect the budget before describing the fallback as strict.
Hard part: per-route failure policy and overshoot
A single global failure policy is too blunt. Read-only product traffic may fail open; login, payments, reward claims, and expensive exports should fail closed or use a strict local fallback.
if counter_store_available:
return central_atomic_decision(rule, key)
if rule.failure_policy == "fail_closed":
return DENY_WITH_RETRY_AFTER
if rule.failure_policy == "local_strict":
return local_bucket_decision(rule.local_limit, key)
return ALLOW_WITH_DEGRADED_HEADEROvershoot formula
If 500 gateways may each spend 20 additional unaccounted tokens, up to 10,000 extra admissions can occur under that model. Pre-deducted leases are different: if the central allocator already reserved those tokens, spending them does not itself exceed the global allocation, although crash recovery and duplicate lease issuance can. State exactly which balance is authoritative and when local allowances are charged or reclaimed.
Blast radius
Large tenants and expensive endpoints get dedicated keys, smaller local buckets, and lower retry budgets.
Recovery
When the store returns, ramp central enforcement gradually and log policy version plus fallback decisions.
HTTP response headers
Clients use these for backoff and user feedback. enables smart retry.
What to do with a denied request
Drop with 429
Best for user-facing APIs, abuse control, login, posting, reward claims, and account creation.
Queue for later
Useful for background exports, batch notifications, or recoverable order work during overload.
Hard limit
Never exceed the threshold. Use for fraud, paid third-party calls, and contractual quotas.
Soft limit
Permit a short burst and warn or degrade before rejecting. Friendly for brief UX spikes.
Client best practices
- Cache responses; prefer push, webhooks, or conditional requests over polling.
- Honor Retry-After; use exponential backoff with jitter.
- Batch when the API allows, and show friendly wait messages.
- Treat 429 as “slow down,” not as a 500-class outage.
Partitioning, regions, and monitoring
Shard the counter store by client_id hash so one client's keys land together. Use hash tags like rate_limit:{user123}:api_search for multi-key ops on one node. Rule matching stays at the gateway; the store only holds counters.
Race conditions
Avoid read-check-write across separate commands. Use Lua, transactions, or CAS so check and update are atomic.
Synchronization
Sticky sessions help but break during scale-out. A shared store keeps gateways stateless and consistent.
Multi-region
Enforce low-latency local limits near the user. For global quotas, choose approximate regional buckets, periodic aggregation, or slower global coordination based on business risk.
Monitoring
- Allowed, denied, queued rates by rule and tenant
- Store latency, errors, hot keys, memory, timeouts
- False positives and downstream overload after fail-open
Layered limiting: L3/L4 at firewalls and WAFs stops volumetric abuse early. L7 in gateways and services understands users, routes, tenants, and business rules.
Algorithm comparison
| Algorithm | Memory | Accuracy | Burst | Use case | Main weakness |
|---|---|---|---|---|---|
| Token bucket | O(1) | High | Yes | API, streaming | Tuning capacity vs refill |
| Leaky bucket | O(1) | High | No | Smooth output | Queued work can delay newer requests |
| Fixed window | O(1) | Low | Yes* | Simple, high QPS | Boundary bursts |
| Sliding window log | O(n) | High | No | Accurate, low volume | Stores timestamps, heavy memory |
| Sliding window ctr | O(1) | Medium | Yes | Best balance (common) | Assumes even prior distribution |
* Boundary burst at window edge
Production systems often use sliding window counter for balance, and token bucket when burst tolerance is required (for example batch uploads).
Bottlenecks
Counter store latency
Every request hits the store. Mitigate with clustering, short-TTL local allow caches, and connection pooling.
Hot keys
A popular user or IP overloads one shard. Hash client + endpoint; dedicate shards for huge tenants.
Rule matching overhead
Many patterns per request. Compile rules to a trie or regex cache; cache rule → counter key mapping.
Store as single point
Store down means limiter degraded. Use Sentinel/Cluster, explicit fail open/closed, and local fallback.
Clock skew
Skewed clocks warp window edges. Prefer store TIME or tightly NTP-synced clocks.
Production readiness
SLOs and metrics
- Decision p99 adds <1ms; policy reload propagates in <60 sec.
- Measure allowed, denied, fallback, overshoot estimate, and false positives by rule.
- Store SLO: latency, timeouts, hot-key skew, memory headroom, replica lag.
Alerting
- Page on store outage for fail-closed routes, policy load failure, or downstream overload after fail-open.
- Warn on deny spikes, hot keys, memory pressure, and unexpected local fallback use.
- Alert by tenant and endpoint class for noisy-neighbor problems.
Runbook and degraded mode
- Store slow: safe routes → local strict; expensive routes → fail closed.
- Hot key: split tenant keys or move to a dedicated shard.
- Bad policy: roll back version; keep old compiled rules until healthy.
Security and threat model
- Assets: policy config, counters, plans, bypass lists, audit logs.
- Actors: botnets, token sharing, IP rotation, compromised internal keys, malicious policy edits.
- Controls: signed policy changes, scoped bypasses, identity-aware keys, audit logs.
Test plan
- Boundary-window, clock-skew, duplicate, and race tests per algorithm.
- Load test 1M decisions/sec with hot-key, timeout, and policy reload scenarios.
- Chaos test fail-open/fail-closed routes and verify downstream protection.
Rollout, DR, and 10x scale
- Ship observe-only policies, compare would-deny, then enforce by tenant or endpoint.
- DR uses regional local limits when global coordination is unavailable; reconcile later.
- At 10x: trie-compiled rules, shard hot tenants, edge simple limits, approximate regional quotas.
Review the design
Algorithms: token bucket (burst), fixed window (simple, boundary burst), sliding window counter (best balance).
Storage: shared counter store for distributed counts. Lua for INCR + EXPIRE atomicity.
Rules: JSON config by endpoint, limit, window, and key. Match by user_id or IP.
Failure: fail open vs fail closed per route. Local fallback counters.
Headers: X-RateLimit-Limit, Remaining, Reset; Retry-After on 429.
Scale: cluster, shard by client_id, connection pooling.
Compare a starting answer with a more complete design
A starting answer
Pick a rate-limit key, choose an algorithm, store counters in a shared fast store, and return 429 with Retry-After when rejected.
Developing the reasoning
Discuss atomic Lua updates, local versus global limits, fail-open versus fail-closed behavior, tenant plans, hot keys, multi-region approximation, and limiter observability.
Deeper walkthrough
Distributed rate limiter deeper walkthrough
A rate limiter is a fast policy decision service. The important parts are key design, atomic state updates, multi-region behavior, and what happens when the limiter itself fails.
Requirement refinements
Clarify limit dimensions: user, API key, IP, endpoint, tenant plan, method, region, and internal service exemptions.
State the failure policy per endpoint: fail-open for availability or fail-closed to protect expensive downstream systems.
API design details
Return limit headers such as remaining tokens, reset time, and Retry-After so clients can back off correctly.
Include policy version and matched rule in logs for debugging unexpected throttling.
Data model and storage
Store limiter state by compact keys such as plan:user:endpoint:window or token bucket state.
Keep policy definitions separate from runtime counters so plan changes do not require clearing all counters.
Scaling strategy
Use Lua or equivalent atomic operations for exact single-region decisions.
For global limits, choose between approximate local buckets, periodic aggregation, or high-latency global coordination.
Reliability, recovery, and operations
Handle hot keys by sharding large tenants or combining local pre-limiters with central enforcement.
Monitor limiter latency, deny rate, store errors, hot-key skew, and downstream errors after fail-open events.
Deepening reference
Distributed Rate Limiter: realistic production notes
Beginner mental model: A rate limiter is a tiny fast decision engine with strict atomic counters and clear failure policy.
Production scope: Production rate limiters support many dimensions: user, IP, API key, endpoint, tenant plan, region, and internal bypass. The answer must say what happens when the limiter is unavailable.
Choose what the first version must guarantee
Start by deciding which distributed rate limiter capabilities are essential. Describe the main read and write operations before choosing infrastructure. For each operation, identify the stored records it uses and the condition that makes its result correct. A fast response is useful only if it satisfies that condition.
Next, separate work the user must wait for from work that can finish later. Estimate requests per second, storage growth, and the busiest object or customer. Use those estimates to choose one bottleneck to investigate. The calculations below are example workloads with stated assumptions; they are not measured limits of a particular product.
Concrete capacity calculations
Traffic: 1M client requests/sec through the gateway means the limiter must answer 1M decisions/sec with very low latency.
State: 10M active limit keys with 5 counters each, about 32 bytes per counter plus overhead, is roughly 1.6 GB raw counter state before replication and metadata.
Sliding log cost: storing every request timestamp for 1M req/sec over a 60-second window is 60M entries. At 16 bytes each that is about 1 GB raw, which is why approximate sliding windows or token buckets are common.
Shard sizing: if one counter shard can safely handle 100K operations/sec, 1M/sec needs at least 10 shards; use 2x to 3x headroom and isolate hot tenants.
Global limits: splitting 10K req/min across 3 regions can allocate 3.3K/min each, but unused regional budget is stranded unless regions periodically rebalance.
Scaling to millions of users
Place the limiter close to the gateway so rejected requests do not consume downstream capacity.
Partition by limit key hash. For giant tenants, split internal dimensions by route or token bucket segment to avoid one hot key.
Use local warm buckets for ultra-low latency and periodically reconcile with regional state when exact global limits are not required.
Define fail-open versus fail-closed per endpoint. Public browsing may fail open; expensive writes or login attempts may fail closed.
Return clear Retry-After and remaining-limit headers so well-behaved clients reduce retry pressure.
Sequence and data-flow diagram
Walk this path after the high-level design so it is clear what is synchronous, what is asynchronous, and where the system can degrade safely.
Read the diagram definition
sequenceDiagram
participant C as Client
participant G as Gateway
participant L as Limit service
participant S as Counter store
participant D as Downstream
C->>G: API request
G->>L: Check key and route
L->>S: Atomic counter update
S-->>L: Allow or deny
alt Allowed
G->>D: Forward request
D-->>C: Response
else Denied
G-->>C: 429 with retry time
endCore flow to walk through
Request enters gateway with identity and route.
Limiter computes key and algorithm state, atomically allows or rejects.
Gateway returns request downstream or 429 with Retry-After and limit headers.
Production readiness
Distributed Rate Limiter: ship, operate, and evolve
Now examine the same request flow during failures and deployments. Tie each operational control to the user behavior it protects.
Observability
Track distributed rate limiter p95/p99 latency, error rate, saturation, and user-visible freshness separately for read, write, and async paths.
Alert on leading indicators such as queue age, hot-key skew, cache miss storms, reconciliation mismatches, and error-budget burn.
Security and abuse
Enforce authentication, authorization, rate limits, input validation, audit trails, and privacy checks at the service boundary.
Treat abuse controls as a first-class path: throttling, quota isolation, moderation hooks, revocation, and safe handling of sensitive data.
Degraded modes
Name what still works if cache, search, ranking, stream processing, object storage, or a regional dependency is slow.
Prefer stale-but-safe reads, partial results, async retry, admission control, and feature disablement over corrupting core state.
Testing and rollout
Load test average, peak, and hot-object traffic; add failure-injection for retries, timeouts, backlog growth, and dependency outages.
Ship with canaries, shadow reads or dual writes when useful, rollback switches, schema compatibility checks, and replay validation.
Runbooks
Document dashboards, paging thresholds, owner actions, rollback steps, data repair commands, and customer-impact language.
Keep backfills and replays idempotent, resumable, rate-limited, and observable so repair work does not create a second incident.
10x evolution
Say which metric forces the next architecture change: shard count, cache footprint, write amplification, queue lag, p99 latency, or regional failover reserve.
Evolve one bottleneck at a time: partition by access pattern, isolate hot tenants, regionalize reads, add replayable async pipelines, then harden multi-region behavior.
Hard parts worth deepening
Atomicity matters; use server-side scripts or single-key operations for distributed counters.
Multi-region limits are hard because global strong consistency is expensive.
Fail-open protects availability; fail-closed protects downstream systems.
Different traffic classes need different policies.
Failure modes and recovery
Counter-store hot key for a giant tenant; shard by key or use local token buckets with periodic sync.
Limiter store is down; apply configured fail-open or fail-closed by endpoint criticality.
Clock skew affects fixed windows; prefer server-side time and sliding/token algorithms.
Clients retry aggressively after 429; return Retry-After and use exponential backoff guidance.
Signals to monitor
Observe allowed and denied requests together with limiter latency. A high denial rate may reflect a genuine burst or an incorrect quota rule; break it down by tenant and route before changing limits.
Hot-key count and store errors show pressure on shared counters. If a store failure permits requests through, measure the resulting downstream overload so the availability policy does not hide a cascading failure.
| Choice | Best when | Cost |
|---|---|---|
| Token bucket | Bursts should be allowed up to a budget. | More state than fixed window. |
| Sliding window | Accuracy around boundaries matters. | Higher memory or approximation. |
| Local limiter | Ultra-low latency is required. | Global limits are approximate. |
Try this scenario
Design a global API key limit of 10,000 requests/minute across three regions and state your consistency trade-off.
Primary sources
- IETF draft: RateLimit header fields for HTTP: standard rate-limit response fields.
- Redis: Rate limiting glossary: token/leaky bucket implementations.
- Google SRE: Handling Overload: admission control framing.
Practice and further reading
Use these exercises to connect the lesson to a design decision. State your assumptions, trace what happens to one request, and explain the rule that must remain true even during a failure. That rule is the design's correctness invariant.
Focus for this lesson: Build a distributed limiter with versioned policies, atomic counters, hybrid local/global budgets, bounded regional overshoot, fairness, and load tests.
Explain the decision
How do you make a Redis token-bucket update race-free?
Read the reasoning
Perform elapsed-time refill, cap, decision, decrement, and expiry in one server-side script/transaction using one clock source.
Apply it to a production scenario
A policy rollout accidentally reduces a partner from 10,000 to 100 QPS. What limits blast radius?
Read one approach
Version policies, validate bounds, shadow decisions, canary tenants/routes, retain instant rollback, and alert on rejection delta.
Work through the numbers
Four regions each get a 2,500-token local lease from a 10,000/s global policy. What outage risk exists?
Read the worked calculation
A region may consume its whole lease while others are idle; lease size controls utilization, coordination QPS, and bounded overshoot.
Reason through a failure
What must cross-region limiting document?
Read the recovery reasoning
The maximum allowed global overrun during partition and the per-route choice between availability and strict quota enforcement.
Sources and further study
Continue with Design Web Crawler.