System Design Networking Deep Dive
TCP Congestion Control Mechanisms
Handshake trivia gets you into the room; explains why a “fast” API feels stuck across regions, why bulk uploads sawtooth, and why opening 500 connections can make everyone slower.
This note complements Networking Fundamentals: , , , loss- vs delay-based control, intuition, and the design choices those mechanics force on APIs.
Control loop
cwnd caps unacked data on the path.
Probe capacity exponentially at first.
Grow carefully; cut hard on congestion.
Loss, delay, or model-based (BBR).
What actually happens
Flow control (receiver window) says how much the peer can accept. says how much the network can absorb. The sender transmits at most the minimum of those constraints.
Feedback arrives one later. On long fat networks the is large, so you need many packets in flight to fill the pipe—and recovery after a cut is slow because every decision waits on RTT.
For system design, congestion control is why multiplexing, connection pools, payload size, and retry storms are performance features—not only application concerns.
Slow start, ssthresh, and AIMD
Read the diagram definition
flowchart TD
SS[Slow start: cwnd grows exponentially] --> Hit{Loss or ssthresh?}
Hit -->|ssthresh| CA[Congestion avoidance: additive increase]
Hit -->|loss signal| MD[Multiplicative decrease: cut cwnd]
CA --> Loss2{Loss?}
Loss2 -->|yes| MD
Loss2 -->|no| CA
MD --> Set[Set ssthresh ≈ flight/2]
Set --> Rec[Recovery then CA or limited slow start]
Rec --> CAroughly doubles each RTT until loss or . Then congestion avoidance adds about one segment per RTT (classic Reno intuition; CUBIC uses a cubic function of time since the last congestion event but preserves the “gentle up, sharp down” spirit).
’s multiplicative decrease is why a single loss event can halve throughput for a flow—and why synchronized losses across many flows create sawtooths on shared links.
Interview phrasing
“Slow start is exponential probing; congestion avoidance is conservative sharing; AIMD cuts on congestion signals so the network recovers.” Then connect it to your API’s RTT and object sizes.
Loss-based vs delay-based vs BBR
| Family | Primary signal | Typical behavior |
|---|---|---|
| Loss / marked CE | Fills buffers; high util; latency risk | |
| RTT inflation | Shorter queues; can lose to aggressive peers | |
| BW + min RTT model | Pace near BDP; probe BW/RTT phases |
is the loss-based failure mode on oversized queues: congestion is “invisible” as loss until delay is already awful. AQM and delay-aware control attack that. When interviewing, you do not need kernel patch-level BBR detail—you need the model: estimate bottleneck bandwidth and min RTT, send near that product, periodically probe.
From problem to mechanism
Problem. A mobile app uploads 4 MB photos to a region 80 ms away. Users report “stuck at 20% then finishes,” and P99 upload time is terrible compared to LAN tests.
Naive system. “Add more upload threads” / open 8 TCP connections per photo. Congestion increases; cellular radios and shared Wi‑Fi fare worse; drops.
Break it. Each new connection re-enters . Losses cause multiplicative cuts. Parallel flows synchronize on the bottleneck and amplify queueing.
Mechanism. Respect BDP: reuse connections or HTTP/2/QUIC streams on one congestion controller, size chunks to keep the pipe full without retry storms, and measure goodput under realistic RTT—not datacenter localhost.
Correctness invariant for the path
In-flight data stays near what the bottleneck can clear within about one min-RTT. Far below wastes capacity; far above buys delay and loss. Application concurrency must not fight that control loop blindly.
BDP and API latency / throughput
Read the diagram definition
sequenceDiagram
autonumber
participant App as API client
participant TCP as TCP sender
participant Net as Bottleneck link
participant Srv as Server
Note over App,Srv: Cold connection, large BDP path
App->>TCP: 256 KB request body
TCP->>Net: slow start flights (cwnd small)
Net-->>TCP: ACKs arrive 1 RTT later
TCP->>TCP: cwnd doubles each RTT
Note over TCP: Short RPC may finish before cwnd fills BDP
TCP->>Srv: remaining bytes
Srv-->>App: response
Note over App,Srv: Reused connection keeps larger cwnd → better P99BDP sketch
bandwidth = 100 Mbps ≈ 12.5 MB/s
RTT = 40 ms = 0.04 s
BDP ≈ 12.5 MB/s × 0.04 s ≈ 500 KB in flight
If cwnd starts at ~10–20 segments (~15–30 KB):
need several RTTs of slow start to approach 500 KB
a 40 KB JSON POST may complete mid-ramp
Design implications
- Keep-alive / connection pools for chatty microservices
- Prefer multiplexing (H2/QUIC) over many cold TCPs
- Large uploads: resumeable chunks; one CC context when possible
- Cross-region: expect multi-RTT ramp; budget timeouts accordingly
- Do not confuse "server p50=5ms" with "client p99" on a 80ms pathWorked example: retry storm vs congestion
Scenario
Bottleneck saturated; RTT climbs 30ms → 120ms (bufferbloat)
Loss-based flows cut cwnd; goodput falls
Client timeout = 50ms (set for LAN)
t0 Timeouts fire; clients retry immediately
t1 Offered load ≈ 2–3×; queues deepen further
t2 More loss → more multiplicative decreases
t3 Cascade until shedding / backoff
Better client design
- Timeout ≥ few RTTs of the real path (or adaptive)
- Exponential backoff + jitter
- Cap in-flight requests per destination (app-level CC)
- Idempotent RPCs; hedged requests only with careful budgets
- Prefer server load-shed signals (429/503) over blind retry
System design takeaway
Congestion control is per-connection; your app must not
invent a second, hostile control loop on top.Design choices congestion control forces
Payload and RPC shape
Huge single requests on high-RTT paths spend many RTTs in slow start. Chunked uploads with resume trade metadata complexity for restartability after a cut. Tiny RPCs amortize poorly unless connections stay warm.
Connection pooling and multiplexing
One congestion-controlled connection with many streams (HTTP/2, QUIC) usually beats dozens of independent AIMD flows that overshare the bottleneck. Head-of-line blocking differs by protocol—but the congestion lesson remains: fewer control loops can be kinder to the path.
Fairness and multi-tenant edges
AIMD fairness is per flow. A greedy client that opens N connections can claim ~N shares. Rate limits and per-user connection caps are fairness mechanisms sitting above TCP.
Idle restart
After idle, many stacks restart slow start or reduce cwnd. Long-poll or sparse traffic patterns pay ramp costs again—another vote for thoughtful pooling and for not assuming “TCP is warmed forever.”
Where it shows up in larger systems
CDN and edge downloads
Service mesh / east-west
Bulk replication / backups
QUIC
Trade-offs
| Choice | Gain | Cost |
|---|---|---|
| Loss-based CC (CUBIC) | High utilization on deep buffers | Latency under bufferbloat; loss ≠ congestion always |
| Delay-based CC | Shorter queues, snappier interactive feel | Can lose share to aggressive loss-based peers |
| BBR-style modeling | Aims for high rate with shallow queues | Version quirks; coexistence complexity |
| Many parallel TCP flows | Sometimes faster ramp / more share | Unfairness; more loss; harder ops |
| Multiplexed single CC | One control loop; connection reuse | HOL issues (classic H2); shared fate |
| Aggressive client retries | Masks rare blips on healthy paths | Amplifies congestion; meltdown risk |
Choosing and explaining TCP congestion control
A complete explanation connects the mechanism to a requirement. Describe what improves when you introduce it, which new failure cases become possible, and how you would know that the trade-off is worthwhile.
Explain the core idea
Senders limit in-flight data with cwnd; slow start probes exponentially; AIMD cuts hard on congestion and grows carefully afterward.
Loss-based control fills buffers; delay-based and BBR-style approaches try to keep queues short by reading delay or modeling BDP.
API design must respect RTT and BDP: connection reuse, bounded concurrency, and retries that do not fight the control loop.
When it belongs
Cross-region latency, upload/download throughput, or CDN edge behavior.
Explaining why more connections or tighter timeouts made an outage worse.
Comparing HTTP/1.1 connection sprawl vs multiplexed protocols.
How it fails
Cold connections + large BDP → multi-RTT ramps dominate short RPCs.
Bufferbloat → huge RTT before loss; timeouts and retries explode.
Parallel flows grab unfair share and synchronize collapses.
Non-congestive wireless loss throttles loss-based CC falsely.
Trade-offs to weigh
Utilization versus latency (deep queues vs shallow).
Many flows versus one multiplexed congestion context.
Fast client timeouts versus congestion-safe backoff.
Common misunderstandings
Explaining only the three-way handshake and calling networking “done.”
Assuming datacenter p50 latency predicts client p99 on the public internet.
Treating BBR as magic “no congestion” instead of a different control model.
Wrap-up
Exercises
- Compute the pipe. 400 Mbps, 60 ms RTT. What BDP must cwnd approach to saturate? How many RTTs of classic slow start doubling from ~14 KB roughly get you there?
- Fix the client. A feed API times out at 30 ms for users with 50 ms RTT. Sketch timeout, pool, and retry policies that respect congestion control.
- Argue the CC family. Interactive video call vs overnight backup on a shared last-mile link—when do you prefer loss-based vs delay-aware behavior, and what else must the network provide (AQM, QoS)?
Topic-specific references
- RFC 5681: TCP Congestion Control — slow start, congestion avoidance, fast retransmit/recovery vocabulary.
- ACM Queue: BBR Congestion Control — model-based intuition beyond loss-only signals.
- Bufferbloat Project — why giant queues turn congestion into latency disasters.
Key takeaway
Congestion control is a feedback loop over RTT that keeps shared networks alive. Design APIs for BDP, warm paths, and retries that cooperate with cwnd—not against it.