System Design Networking
Realtime Updates
Imagine you are watching a live auction on your phone. A rival bidder places a higher bid. The seller's server knows first, but your screen only updates when your app learns the news. Classic HTTP waits for you to ask again. Realtime design is about choosing how that news reaches you without burning the fleet or leaving the UI stale.
This page is the chooser. It gives you a ladder of five channels, from cheap and slightly stale to expensive and instant, and a set of questions that tell you which rung to stop on. The mechanics of each rung live on the sibling notes linked below; this note only decides among them, weighing freshness, direction of traffic, and infrastructure cost.
Climb only when forced
Ask again every few seconds; fine when seconds of lag are acceptable.
Hold each request open until news arrives; works through plain HTTP proxies.
One open HTTP response streams server events; the client rarely sends anything back.
One persistent two-way connection; worth its cost only when both sides talk often.
Peer-to-peer audio, video, and media; not a channel for ordinary text chat.
Try it: polling versus push channels
Publish one notification and step the client under short polling, SSE, or WebSocket. Compare wasted polls and how many long-lived connections you hold.
- Events published
- 0
- Client caught up through
- waiting
- Wasted polls
- 0
- Open connections
- 0
Publish server-side events and step the client. Polling discovers updates on the next interval; SSE and WebSocket push as soon as data arrives.
Networking basics · start here
A decision spine, not a protocol deep-dive
The auction story above shows the gap this page fills. Classic HTTP is request → response: the client asks, the server answers. That model fails when the server has news and the client should learn it without asking on a timer. The fix is a ladder of channels, each more capable and more expensive than the last, and the discipline to climb it only when a product constraint forces you.
Ladder: → → → → .
Helpful if you already know: Basic HTTP request and response. Useful but not required: sticky sessions (pinning one client to one server) and L4 vs L7 load balancing (routing by IP and port vs by HTTP details).
How to read this page
First-time readers
Read the definition above first, then follow sections top to bottom. Under each title, the In simpler terms box restates the idea before diagrams and tables.
Mental model
Ask: who opens the connection, who speaks first, and what happens if packets arrive late or twice?
When explaining
Lead with a simple definition, then add trade-offs, failure modes, and one small calculation.
What actually happens when the server learns first
Separate the command from the update. In an auction, placing a bid is a command that must be validated and durably ordered by the auction service. Broadcasting the new highest bid is an update sent after that decision. A fast socket does not decide which simultaneous bid wins; storage and concurrency control do. Clients should use the authoritative result to correct any optimistic display.
Before choosing a rung on the ladder, it helps to see exactly where request and response breaks down. In a normal page load, the browser asks and the server answers, and that is the end of the exchange. Live products break that model: a new bid, a chat message, a score tick, or a typing indicator is born on the server (or on another user's device) before your device knows anything about it. Nothing in plain HTTP lets the server speak first.
Without a push channel you have two bad options. You can poll, meaning the client asks again on a timer, which wastes requests when nothing changed and still adds delay up to the timer interval. Or you can leave the user staring at a stale screen. Realtime design escapes that choice by first naming three things: who generates events, how fresh they must be, and whether the client must send as often as it receives, which is what a (two-way) channel is for.
Once you have answered those three questions, the channel choice becomes a cost conversation rather than a buzzword contest, and the rest of this page is that conversation. The protocol mechanics themselves are covered on the sibling notes, so open these when you want the wire-level detail:
How to choose a realtime channel
With the three questions named (who generates events, how fresh, which direction), four checks walk you up the ladder one rung at a time. Stop at the first rung that satisfies the product.
1. Are seconds of lag OK? Admin dashboards and slow inventory sync can use . Operations stay simple because every request is an ordinary short HTTP call, and you never have to run a farm of servers that hold thousands of open connections.
2. Is traffic mostly server → client? Prefer (Server-Sent Events, a one-way stream over HTTP) for auctions, notifications, and scoreboards. Use when a proxy between client and server blocks or buffers the stream.
3. Do both sides send often? Chat, presence, and collaborative cursors justify , along with the stateful server farm that comes with them (each server remembers which clients are attached to it).
4. Is it live audio or video between peers? That is , a media protocol, not a generic text chat channel.
Simple rule of thumb
Start at the bottom of the ladder and climb only when freshness, direction, or media constraints force you. In interviews, say the bill out loud: held connections, (pinning each client to one server), and (one event copied to many watchers) cost money even when idle. Do not default to WebSockets for every "live" UI; if push dominates, SSE often wins.
A decision path you can walk aloud
The four checks above become a flowchart you can sketch on a whiteboard. Each amber diamond is one of the checks; follow the branch that matches your product and stop at the first coloured protocol node.
Server must push updates to clients
How fresh? Seconds OK?
Yes
Short polling
Sub-second or continuous
Check direction next
Need frequent client → server too?
No: push dominates
SSE (server → client stream)
Fallback: long polling if proxies block streams
Yes: chat, collab, games
Check media next
Audio / video or peer media?
No
WebSockets (duplex frames)
If WS blocked → SSE
Yes
WebRTC (P2P media + data)
Said aloud, the walk is short. Seconds of lag are fine, so short polling is honest and easy to operate. Push-only over HTTP, so SSE, with long polling as the fallback when streams get blocked. Frequent two-way traffic, so WebSockets and the stateful farm they bring. Live audio or video between peers, so WebRTC, and never for generic chat text.
Compare channels by constraints
Freshness is an end-to-end budget, not only transport latency. Time may be spent committing an event, waiting in a broker, routing it to connection servers, queuing behind a slow client, and rendering it. If an update must appear within one second, allocate that budget across the path and measure event-creation-to-display delay. Replacing polling with sockets cannot fix a broker backlog that already consumes several seconds.
The flowchart picks a rung; this table shows what each rung costs once it is running. A few abbreviations appear in the cells: LB is the load balancer, L4 and L7 are the network layers it routes on (IP and port vs HTTP details), EventSource is the browser API for SSE, QPS is queries per second, and ICE, NAT, and TURN are the WebRTC machinery for finding a path between two peers behind home routers, with TURN being the relay server used when no direct path exists.
| Constraint | Short poll | Long poll | SSE | WebSocket | WebRTC |
|---|---|---|---|---|---|
| Direction | Client pulls | Client pulls (held) | Uni (server → client) | Bi (full duplex) | Bi (often P2P media) |
| Browser support | Universal | Universal | Strong (EventSource) | Strong | Strong; NAT/TURN hard |
| LB layer | L7 fine | L7 fine | L7 OK; watch buffering | Often L4; sticky / affinity | Signaling via HTTP/WS; media UDP |
| Stateful cost | Low | Held request / worker | Open response per client | High (socket + maps) | High (ICE/TURN/media) |
| Reconnect / replay | Client cursor / since= | Client cursor | Last-Event-ID | App resume tokens | Re-signal + re-ICE |
| Scale cost | QPS × empty polls | Open HTTP × users | Open streams + fan-out | RAM × sockets + pub/sub | TURN bandwidth spikes |
When each channel wins
The table compares rungs in the abstract; here is the same ladder mapped onto products you might be asked to design. wins for admin dashboards and slow inventory sync, where "refresh every 10–30s is fine" is a real requirement and operational simplicity matters more than latency. wins as a compatibility layer rather than a first choice: legacy chat or inbox products used it when the HTTP Upgrade handshake (the step that turns a request into a WebSocket) was blocked, the same "held GET" pattern early mobile messengers relied on.
wins for auction bid ticks, in-app notification toasts, and live sports scoreboards, because almost all traffic flows server → many watchers while client actions are rare POSTs (place a bid, mark as read). You stay on ordinary HTTP and recover from a dropped connection with replay, where the browser tells the server the last event it saw so missed events can be resent.
win for group chat, presence, collaborative cursors and documents, and multiplayer games, where both sides send small messages often and at low latency. You must then justify the memory per socket, the sticky routing, and the (publish/subscribe) message bus that fans one message out across servers; the WebSockets note covers that bill in detail.
wins for one-to-one or small-group audio and video calls. Signaling (the setup messages that let two peers find each other) can ride WebSockets or HTTP, while the media itself wants UDP peer-to-peer with a TURN relay as the fallback when a direct path is impossible. Do not reach for it for text chat or notification feeds.
What long-lived channels do to your infra
The cost of broadcasting depends on recipients. One room producing ten 500-byte updates per second for 20,000 connected viewers produces about 100 MB per second of payload delivery before framing and encryption. A low producer event rate can therefore create a high outbound workload. Route events only to gateways that have subscribers, and decide whether repeated state updates can be combined when a viewer falls behind.
Every rung above short polling keeps a connection open for minutes or hours, and that single fact is what makes the stateful rungs expensive. Three pieces of infrastructure feel it first, and the numbers at the end show how quickly the cost adds up.
If connection state lives on one node (which clients are subscribed, their in-memory queues), the load balancer must send each client back to the same node every time. The alternative is to move presence and fan-out onto a shared bus so that any node can own any socket and stickiness stops mattering.
for WebSockets
Prefer L4 (routing on IP and port) for large persistent-socket farms, because it sticks a TCP flow to a backend at connect time and never has to parse HTTP. Use an balancer (routing on HTTP paths and headers) when you need path rules for ordinary HTTP and long-poll traffic.
Proxy buffering for SSE
Proxies assume a response ends soon, so they buffer it. On the stream path, disable response buffering, send heartbeats more often than the proxy's idle timeout, and forbid caching. Otherwise events arrive in delayed blobs or the stream dies silently with nobody noticing.
Numbers to know
- One 600B event to 10K subscribers is about 6MB outbound before protocol overhead.
- Chat: 5% of 1M online users sending 2 msg/min, each fanned to 20 peers, is about 33K outbound msgs/sec.
- RAM: 1M sockets at about 15KB server state each is 15GB before queues and indexes.
A trap worth naming early
Now that the infra bill is visible, the most common interview mistake is easy to explain. Candidates often say "we'll use WebSockets" for any live UI. Interviewers want you to ask first: Is traffic mostly server → client? If yes, SSE (or even polling) usually wins, with a smaller operational surface, middleboxes (the proxies and firewalls between client and server) that already understand HTTP, and a simpler reconnect story.
Only pick a duplex channel when both directions need frequent low-latency frames, and when you do, say the bill out loud: per-connection memory, sticky or L4 routing, draining connections off a server before you restart it, reconnect storms when a server disappears, and a pub/sub backplane for fan-out (see "What long-lived channels do to your infra" above).
Degraded modes when the happy path fails
A reconnect needs a recovery contract. The client presents its last processed event position; the server replays later events if they remain in retention, then continues live delivery without a gap. If the position is too old, return a fresh snapshot and a new position rather than silently skipping history. Duplicate replay is normal around failures, so apply event identifiers or monotonic versions on the client.
Different event types deserve different overflow policies. A chat message should remain in durable history and be replayed. A typing indicator can expire because replaying it later is misleading. A cursor position can replace an older queued position from the same user. Define these semantics before setting buffer sizes; otherwise an overloaded connection turns harmless transient updates into unbounded memory growth.
Having chosen a rung and priced it, the last step is to decide where you land when that rung is blocked. Each arrow below is one step down the ladder, triggered by a specific failure.
Whatever the rung, pair every reconnect with a cursor that says where the client left off ( for SSE, a resume token for WebSockets, or a since= parameter for polling) and with jittered backoff (random delays between retries) so a brief blip does not turn thousands of simultaneous reconnects into a self-inflicted denial of service.
Choosing and explaining realtime channel choice
A complete explanation connects the mechanism to a requirement. Describe what improves when you introduce it, which new failure cases become possible, and how you would know that the trade-off is worthwhile.
Explain the core idea
Start from push need: request/response alone leaves the client stale until it polls.
Climb the ladder: poll → long poll → SSE → WebSocket → WebRTC; stop when constraints are met.
Name direction, freshness, LB/sticky cost, and reconnect/replay before naming a protocol.
When it belongs
The product requires live UI: chat, auctions, notifications, scores, presence, or calls.
The interviewer asks how the server pushes updates to browsers or mobile clients.
You need to justify why not WebSockets for a mostly one-way feed.
How it fails
Proxy buffering or idle timeouts kill SSE streams.
WebSocket farms without sticky or shared pub/sub lose messages after reconnect.
Reconnect storms without jitter overwhelm connection servers.
WebRTC without TURN fails for a large minority of NATs.
Trade-offs to weigh
Polling is simple but burns QPS and adds delay up to the interval.
SSE stays HTTP-shaped but is push-only; commands need separate requests.
WebSockets buy duplex at the cost of stateful infra and harder LB.
WebRTC optimizes media latency but adds signaling, ICE, and TURN cost.
Common misunderstandings
Defaulting to WebSockets for every “realtime” requirement.
Ignoring sticky sessions, L4 vs L7, and proxy buffering.
Forgetting resume tokens / Last-Event-ID so reconnects drop events.
Using WebRTC for text chat or notification delivery.
Wrap-up
What to remember
Realtime is a product constraint, not a protocol fashion. Name who generates events, how fresh they must be, and whether both sides speak often. Then pick the cheapest channel that works, with one explicit fallback for when middleboxes disagree.
Primary sources
- RFC 6455: The WebSocket Protocol: the standard for the full-duplex channel at the top of the text ladder.
- HTML: Server-Sent Events: the specification for server-to-client event streams and EventSource.
- RFC 9110: HTTP Semantics: the request/response model that plain polling is built on.
Practice and further reading
Use these exercises to connect the lesson to a design decision. State your assumptions, trace what happens to one request, and explain the rule that must remain true even during a failure. That rule is the design's correctness invariant.
Focus for this lesson: Select polling, SSE, WebSockets, or WebRTC from directionality, ordering, replay, proxy, mobile-power, and connection-count constraints.
Explain the decision
When is SSE a better default than WebSockets?
Read the reasoning
For server-to-browser text events where HTTP semantics, automatic reconnect, and one-way flow matter more than bidirectional binary messages.
Apply it to a production scenario
A dashboard needs updates every 30 seconds for 2,000 users behind corporate proxies. What do you choose?
Read one approach
Start with SSE or conditional polling, validate proxy buffering and connection limits, and provide resume IDs plus a polling fallback.
Work through the numbers
Polling 100,000 clients every 10 seconds produces how many requests/s before payloads?
Read the worked calculation
10,000 requests/s; compare this with long-lived connection memory, heartbeat bandwidth, and update fan-out.
Reason through a failure
What must reconnect logic preserve?
Read the recovery reasoning
Resume from an acknowledged event position, deduplicate replayed events, add jitter, and never turn a regional recovery into a synchronized storm.
Sources and further study
Continue with WebSockets.