System Design Networking
WebSockets
Imagine a Slack or Discord channel during a launch. You type, someone else is already typing, presence dots (the small "online" indicators) flicker, and a new message lands in every open tab within a blink. That product needs both sides to speak at any moment, not a browser that asks "anything new?" every few seconds.
A starts life as an ordinary HTTP request and then upgrades that one connection into a long-lived channel. You gain messaging (both sides can send at once), and you inherit an operations bill: (routing a client back to the same server), , reconnect storms, and the cost of fanning one message out to many sockets. This page walks through the handshake, the scaling model, and the failure modes in that order.
What happens first?
The client sends one normal HTTP request asking to switch protocols.
The server agrees, and plain request/response HTTP stops on this connection.
From now on either side pushes small messages at any time.
Ops bill
You now own sticky routing, heartbeats, reconnect logic, and pub/sub fan-out.
Try it: WebSocket heartbeat versus NAT idle timeout
Tune ping interval, advance simulated time, and watch when keepalives preserve the connection versus when silent drops force a reconnect.
- Connection
- open
- Elapsed
- 0s
- Pings sent
- 0
- Pongs received
- 0
Advance time to see ping/pong keepalives. If the NAT idle timer expires before traffic, middleboxes may drop the socket without a TCP reset.
Networking basics · start here
Keep one connection open; both sides can push
Helpful if you already know: HTTP upgrade handshake concept (starts as HTTP, becomes WS).
How to read this page
First-time readers
Read the definition above first, then follow sections top to bottom. Under each title, the In simpler terms box restates the idea before diagrams and tables.
Mental model
Ask: who opens the connection, who speaks first, and what happens if packets arrive late or twice?
When explaining
Lead with a simple definition, then add trade-offs, failure modes, and one small calculation.
What actually happens on the wire
The established channel is a transport, not a durable inbox. A successful send to a local socket buffer does not prove the remote application processed the message. For a chat product, persist the message with a stable identifier, acknowledge the durable state to the sender, and distinguish that acknowledgment from recipient delivery and read receipts. Those are three separate milestones with different failure cases.
Think of HTTP as sending letters: one request, one response, and the exchange is usually over. A WebSocket is closer to picking up the phone. Once the call connects, both sides talk until someone hangs up, and nobody has to keep writing in to ask whether anything is new.
The call is set up with a normal HTTP request that carries an header. The server replies , and from that moment the same TCP socket (the network connection HTTP was already using) carries WebSocket instead: text, binary, ping, pong, or close. The handshake section below shows the exact headers.
The protocol hands you nothing but those frames, so your application has to decide what a message means. Most teams settle on JSON envelopes with a type, payload, and id, add versioning so old and new clients can coexist, authenticate during the upgrade, serve everything over (WebSocket over TLS encryption), and rate-limit messages per connection. That work starts before you scale, because a schema is hard to change once millions of clients speak it.
The bigger shift is architectural. A stateless REST tier assumes each request disappears quickly, so any server can handle any call. A WebSocket server holds per-connection state and memory for as long as the socket is open. Large chat products cite millions of concurrent sockets, which is why interviews concentrate on connection maps (which user is on which server), backplanes (a shared message bus between servers), and load balancing that respects long-lived connections.
How to choose WebSockets
Knowing how the channel works, the first decision is whether you need one at all. Three questions settle it.
1. Do both sides need frequent low-latency messages? Chat, multiplayer games, and collaborative cursors: yes. Notification feeds and LLM token streams (a language model's output, which flows one way) usually do not; prefer (Server-Sent Events, a one-way push over plain HTTP).
2. Are you ready for stateful connections? Memory per socket, , draining connections on every deploy, and reconnect logic become part of the product cost, not an afterthought.
3. Will real networks allow the Upgrade? Corporate proxies and firewalls still block or reset upgraded connections, so plan an SSE or fallback from day one (the fallback ladder later on this page shows the order).
Simple rule of thumb
Default to simpler push first, and reach for WebSockets when two-way traffic is a real requirement rather than because "realtime" sounds impressive. If SSE plus an occasional POST covers 90% of the product, stay there. WebSockets should earn their place.
When a simpler tool wins
- Server-to-client only (notifications, feeds): use SSE, which reconnects automatically and reuses existing HTTP infrastructure.
- Infrequent updates (every 30s or more): short polling wins, because an idle socket wastes server memory.
- Strict request-response: use REST or gRPC, since one reply per message needs no persistent connection.
- AI token streaming: use SSE or fetch streaming, because model output flows one direction only.
- Dashboards refreshing every few seconds: SSE or polling, since they rarely justify WebSocket infrastructure.
Where WebSockets earn their place
Use them when both sides need to send at any time. The classic products, and what each one leans on:
- Chat and messaging apps: two-way send plus typing indicators and presence dots.
- Multiplayer browser games: positions update 20–60 times per second, so sub-50ms latency and binary frames matter.
- Live trading: order books stream at very high message rates; clients send orders, servers push fills.
- Collaborative editing (shared docs, whiteboards): OT/CRDT operations (conflict-resolution algorithms) and live cursors need duplex.
- IoT and telemetry (smart home, sensors): devices push continuously, servers send commands back, per-message overhead matters.
- Live dashboards (monitoring, market data): metrics stream down and users steer the feed by subscribing.
Compare: WebSockets vs SSE vs long polling vs HTTP/2 Push
The table puts the four options side by side. Read the Direction row first (who is allowed to speak first), then Overhead and Latency; HTTP/2 Push is a browser feature for preloading resources rather than a messaging channel, which is why it sits last.
| Feature | WebSockets | SSE | Long Polling | HTTP/2 Push |
|---|---|---|---|---|
| Direction | Bidirectional | Server → Client | Client → Server | Server → Client |
| Connection | Persistent (ws://) | Persistent (HTTP) | Repeated HTTP | Persistent (HTTP/2) |
| Overhead/msg | 2-14 bytes | ~50 bytes | ~800 bytes | ~20 bytes |
| Latency | <1ms (same DC) | <1ms (same DC) | ~100ms+ (poll) | <1ms |
| Best for | Chat, gaming | Notifications, feeds | Simple fallback | Preloading resources |
How to choose WebSocket, SSE, or polling
WebSocket
Use for frequent two-way messages. Budget memory per connection, heartbeats, reconnect storms, sticky routing, and backpressure.
SSE
Use for server-to-client streams over HTTP. It has automatic reconnect and simple text events, but client commands still need separate requests.
Polling
Use when freshness can be seconds and simplicity matters. Lower intervals improve freshness but multiply empty requests.
Failure handling
An event stream that must preserve every update needs a replay cursor and retained history. A polling endpoint that returns the latest complete state may simply read a fresh snapshot instead. In both cases, spread retries over time and limit work for slow clients. Recovery requirements depend on whether the product promises event history or only the current state.
Simple rule of thumb
Frame the answer around freshness, directionality, connection count, and replay. Then size fan-out and memory before promising realtime.
Chat at 1M online
If 5% send 2 messages/min and each fans out to 20 recipients: 50K senders x 2 / 60 x 20 = ~33K outbound messages/sec.
Feed updates
For one million users receiving five updates per minute, outbound rate is 1,000,000 × 5 ÷ 60, or about 83,333 events per second. At 600 bytes per event, that is 50 MB/s of payload before protocol overhead. These are delivered copies: if one source event reaches many users, count every recipient when estimating outbound bandwidth.
Connection memory
1M open sockets x 15KB/server state is ~15GB RAM, before queues, presence maps, and per-room indexes.
The handshake, step by step
The upgrade shown below is the HTTP/1.1 handshake. WebSockets can also be established using extended CONNECT in supported HTTP/2 or HTTP/3 deployments, so verify the full client-to-proxy-to-origin path. After authentication, plan for credentials expiring while the connection remains open. Reauthorization or a controlled reconnect prevents a long-lived socket from keeping permissions that have since been revoked.
Now that you know when to use one, here is exactly what the client and server exchange. The request and the reply are ordinary HTTP; everything after them is not. Open each step to see the real headers.
Step 1: HTTP Upgrade Request
GET /chat HTTP/1.1
Host: example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13Step 2: Server Accepts Upgrade
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=Step 3: Full-Duplex Communication
Both sides now send whenever they like (). Text or binary frames carry 2–14 bytes of overhead each, ping and pong frames keep the line alive (the heartbeat section explains how), and the connection stays open until one side sends a close frame.
Frame types you will see
Every message after the handshake is one of a handful of frame kinds, identified by a small numeric code in the frame header:
0x1: Text frame (UTF-8)
0x2: Binary frame (raw)
0x8: Close frame
0x9 / 0xA: Ping / Pong
It is still one TCP connection
Because it is one socket, the usual pattern is to open one connection per client and multiplex rooms or channels over it with typed messages, rather than opening a socket per room. The two transcripts below make the letters-versus-phone-call difference concrete.
HTTP (letters)
Client: "Give me /api/messages"
Server: "Here are 50 messages"
Connection closed
Client: "Any new messages?"
Server: "Nope"
…repeat forever
WebSocket (phone call)
Client: "Upgrade to WebSocket"
Server: "101 Switching Protocols"
Connection stays open
Server: "New message from Alice!"
Client: "Send: Hello Alice!"
…both sides talk freely
What the code looks like
The broadcast example demonstrates frame handling, but a product server also validates the message type and size, checks room membership on every relevant action, and bounds each client’s outgoing queue. A slow receiver must not keep accumulating copies indefinitely. Disconnect it with a resumable position for durable messages, or discard superseded presence updates according to the application’s stated policy.
Libraries handle the handshake for you; what you actually write is the message handling on both ends. The server below broadcasts every chat message to all connected clients, and the browser client switches on the type field of each envelope, which is the schema idea from earlier in practice.
Server (event-loop runtime)
const WebSocket = require('ws');
const wss = new WebSocket.Server({ port: 8080 });
wss.on('connection', (ws) => {
console.log('Client connected');
// Send welcome message
ws.send(JSON.stringify({
type: 'welcome',
data: { userId: 'user_123', rooms: ['general'] }
}));
// Handle incoming messages
ws.on('message', (raw) => {
const msg = JSON.parse(raw);
if (msg.type === 'chat') {
// Broadcast to all connected clients
wss.clients.forEach((client) => {
if (client.readyState === WebSocket.OPEN) {
client.send(JSON.stringify({
type: 'chat',
from: msg.from,
text: msg.text,
ts: Date.now()
}));
}
});
}
});
ws.on('close', () => console.log('Client left'));
});Client (Browser)
const ws = new WebSocket('wss://chat.example.com');
ws.onopen = () => {
console.log('Connected!');
// Send a chat message
ws.send(JSON.stringify({
type: 'chat',
from: 'Alice',
text: 'Hello everyone!'
}));
};
ws.onmessage = (event) => {
const msg = JSON.parse(event.data);
switch (msg.type) {
case 'welcome':
console.log('Joined as', msg.data.userId);
break;
case 'chat':
displayMessage(msg.from, msg.text);
break;
case 'typing':
showTypingIndicator(msg.from);
break;
}
};
ws.onclose = (event) => {
console.log('Disconnected:', event.code);
// Implement reconnection logic
};
ws.onerror = (err) => console.error(err);What flows on the wire after the handshake
Every line below is one text frame. Notice how little each one carries beyond the JSON itself compared with a full HTTP request.
← Server: {"type":"welcome","data":{"userId":"user_123"}}
→ Client: {"type":"chat","from":"Alice","text":"Hello!"}
← Server: {"type":"chat","from":"Alice","text":"Hello!","ts":1672531200}
← Server: {"type":"typing","from":"Bob"}
← Server: {"type":"chat","from":"Bob","text":"Hey Alice!","ts":1672531205}
// Each message is a WebSocket TEXT frame
// Frame overhead: just 2-6 bytes (vs ~800 bytes per HTTP request)Horizontal scale: sticky sessions and pub/sub
An established TCP connection already remains attached to its selected backend. Sticky routing matters when a new or reconnecting connection needs state that exists only on the old backend. Shared subscription information and a durable resume position allow reconnects to land elsewhere. A connection registry maps an authenticated user or session to current gateways; it needs expiry or leases because a crashed gateway cannot reliably unregister itself.
The code above works on one server. The moment you add a second, the fact that each socket lives on a specific machine becomes the central design problem.
The problem
A sends a message to B, but Server 1 does not know where B is connected, and it cannot write to a socket it does not own.
Solution: backend
Server 1 publishes the message to a shared bus, every connection server subscribes, and Server 3 delivers it to B's local socket. This is how most chat systems work. Redis Pub/Sub handles high fan-out (one message copied to many subscribers); for cross-region durability, teams often add Kafka or NATS.
Connection registry (who is where)
The registry is routing metadata, so it must not become the only record of an undelivered message. A send may race with a user moving from one gateway to another, leaving the registry temporarily stale. Store durable messages independently, treat a missed live push as recoverable, and have the client catch up from its last confirmed position. This avoids trying to make a transient socket mapping perfectly consistent with every message write.
For a direct message the sender's server looks up the target server first, so the fleet also keeps a registry of which user sits on which node:
On connect: SET user:123 ws2
On disconnect: DEL user:123
On message: look up the target server, then route.
Keep that mapping in Redis rather than in server memory (in interview language, "externalize the state"), so a server's RAM only holds the socket objects themselves. Pair the registry with or consistent hashing (hash the user id to pick a gateway) when you want the same user to land on the same node repeatedly.
At scale: gateway servers and guild fan-out
The same idea at very large scale looks like this. Gateway servers built on an actor-based runtime (the BEAM virtual machine, which runs millions of lightweight processes) hold one process per connection, and a message to a guild (a large group with many members) fans out through pub/sub to every member's socket.
Gateway servers
Message routing
The statefulness tension
Everything in this box exists because of that tension. The registry, the backplane, and the gateway layout are all ways of moving state out of individual servers so the fleet behaves a little more like the stateless REST tier you are used to.
The infrastructure bill vs REST
Stateless REST (the dream)
- Any load balancer works, because round-robin can send each request to any server.
- Servers are stateless, so you scale by adding boxes and the balancer routes automatically.
- A crash just means the user retries and lands on another healthy server.
- Deploys replace servers one by one; state lives in the database or cache, never in memory.
- Standard HTTP monitoring tools work unchanged, because there is nothing to track between requests.
Stateful WebSocket (the reality)
- You need sticky sessions or consistent hashing so a client returns to its own server.
- Servers are stateful, so adding more blindly does nothing for connections already pinned elsewhere.
- A crash drops every connection on that node, and all of them reconnect at once.
- Deploys cannot simply restart a node; every socket on it has to be moved first.
- Cross-server messages need the pub/sub backplane and the connection registry described above.
- You need heartbeats, reconnection logic, and WebSocket-aware monitoring that counts open sockets.
How to handle this in an interview: name the moves in order. Externalize state (the registry above), route messages through a pub/sub backplane rather than server to server, and degrade gracefully to SSE or long polling when the upgrade fails. Safe deploys are the fourth move, covered under load balancing below.
WebSocket architecture and backpressure
Client A
Connection server A
Pub/sub broker
Fan-out workers
Connection server B
Client B
Presence registry (from Conn A)
Durable message store (from fan-out)
Client B can reconnect with a resume token to connection server B.
Slow consumer
A slow client can receive bytes more slowly than the application produces events. Bound the queued bytes for each connection so that one client cannot consume the server memory. Coalesce replaceable state such as cursor positions, or disconnect and resume later from durable history. Do not silently drop an important chat message just because dropping a typing indicator is safe.
Reconnect storm
After a server restart, many clients notice the broken connection together. Randomized delay, called jitter, spreads their reconnect attempts over time. A resume token identifies their last acknowledged position so reconnecting does not require a full history download. Limit connection admission while recovering; otherwise reconnect work can consume the capacity needed to serve existing clients.
Auth lifecycle
Authenticate during upgrade, refresh tokens while connected, and re-authenticate after reconnect before replaying private events.
Getting the upgrade through the network, and balancing the load
Pub/sub solves routing between your own servers. This section is about everything between the browser and those servers, because a WebSocket only exists if every hop lets the upgrade through and then leaves the connection alone.
Infra must allow the upgrade end-to-end
- Edge and load balancer: enable WebSocket (HTTP Upgrade) support, raise idle timeouts, align them with .
- Proxies must forward
Connection: Upgradeand never buffer the duplex stream like an ordinary HTTP body. - Corporate firewalls and some middleboxes (inspection devices) still reset upgraded connections; see the fallback ladder below.
- Prefer balancing for large persistent-connection farms, and where HTTP needs path or host routing.
Load balancing strategies for socket farms
An HTTP load balancer assumes short-lived requests that any server can take. A WebSocket balancer must respect long-lived connections whose state lives on one specific server, which leads to four techniques.
1. L4 (TCP) balancing: route at connect time, then stick. Simple, but the spread gets uneven over time, and old servers keep their connections across deploys until drained.
2. Consistent hashing: user_id → hash → server. The same user lands on the same gateway, and when a server is added or removed only a small share of connections has to move.
3. : mark the old server as draining → new connects go elsewhere → send GOAWAY (a "please reconnect" signal) to existing clients → they reconnect gradually, using the backoff from the reconnect section → shut down after a timeout.
4. Geographic routing: US users → us-east gateways, EU users → eu-west. Cuts the cross-ocean round trip on every message.
Numbers to know: memory, connections, and bandwidth
With routing settled, the next question every design gets is capacity: how many sockets fit on one box, and what does each message cost? Every number on this page lives here; later sections refer back rather than repeating them.
Thread-per-connection (breaks at scale)
The naive server gives every socket its own thread. 10K connections = 10K threads × 1MB stack = 10GB RAM before the application does any work, so the model collapses long before a million users.
Event-driven I/O (scales to millions)
Production servers instead run an event loop: one thread asks the kernel which of thousands of sockets have data ready and handles only those. epoll (Linux), kqueue (macOS), and IOCP (Windows) are the operating-system calls that make this possible.
epoll (Linux) / kqueue (macOS) / IOCP (Windows)
Single thread monitors ALL connections:
events = epoll_wait(fd, ...)
for event in events:
handle(event.fd)Memory per WebSocket connection
With threads out of the way, memory per socket becomes the limit. The table adds up where the bytes go:
| Component | Memory |
|---|---|
| TCP socket buffers (recv + send) | ~8 KB |
| TLS state (if wss://) | ~5 KB |
| Application state (user, room) | ~1-2 KB |
| Connection object overhead | ~0.5 KB |
| Total per connection | ~10-15 KB |
Multiply that per-socket cost by concurrent users and the size of the fleet falls out:
Large chat services often cite ~900K connections per actor-based runtime server; carefully tuned event-loop runtimes reach hundreds of thousands to about a million per box.
Worked example: memory and a reconnect storm
~12 KB per idle socket × 500K connections ≈ 6 GB RAM on one gateway before message buffers. If that node dies and all 500K clients reconnect in the same second, even a healthy replacement is hit by a thundering herd (a burst of accept, auth, and subscribe work arriving together). Spread the reconnects over ~30–60s with plus jitter (the reconnect section shows the loop) and the cliff becomes a manageable ramp.
What very large deployments report
Binary frames and compression
Memory is one axis; bandwidth is the other. At 1 billion messages a day, the encoding you choose changes the bill by roughly a factor of three:
is the protocol's optional per-message compression, negotiated with Sec-WebSocket-Extensions: permessage-deflate. JSON often shrinks 60–80% and binary 10–30%, a huge bandwidth saving, but every message costs CPU and every connection holds a ~300KB compression context. Many teams therefore disable compression for tiny messages (under 100 bytes) and enable it only for large payloads.
Keep connections alive: heartbeats, reconnects, and fallbacks
Capacity planning assumes connections stay up, but real networks quietly kill idle sockets and whole nodes disappear. Three habits keep the channel usable: detect dead peers, reconnect politely, and fall back when the upgrade is refused.
Heartbeats: protocol ping/pong and application pings
Protocol ping/pong (RFC 6455)
Server sends a PING frame (often every 30 seconds).
Client replies with a PONG frame.
No PONG within ~10s → treat the connection as dead and clean up.
Application-level heartbeat
Why both? The protocol detects TCP-level death, while an application ping proves the code above the socket is still responsive and measures RTT (round-trip time). The interval is not arbitrary either: home routers running NAT (network address translation) drop idle TCP connections after 60–300s, so pings must arrive well inside that window or connections die silently.
Reconnect without crushing the fleet
When a heartbeat fails or a node dies, every affected client wants to reconnect immediately. Use so retries slow down and, thanks to the random jitter, spread out:
attempt = 0
while not connected:
delay = min(BASE * 2^attempt, MAX_DELAY)
jitter = random(0, delay * 0.5)
wait(delay + jitter)
try connect()
attempt++Why jitter matters: a server crashes, 1M clients reconnect at the same instant, the replacement collapses under the stampede, and the cycle repeats. With jitter the reconnects spread and recovery stays smooth. While offline, queue outgoing messages, remember the last message ID so the client can resume without gaps, and show the connection state in the UI so users understand why messages are pending.
Fallback ladder when Upgrade is blocked
Reconnecting only helps if a WebSocket can be opened at all. When a proxy or firewall refuses the upgrade, step down the ladder one rung at a time: first, then , then plain short polling.
// WebSocket helper library does this automatically:
const socket = io('https://api.example.com', {
transports: ['websocket', 'polling'], // try WS first
upgrade: true, // upgrade polling→WS if possible
reconnectionDelay: 1000, // start retry at 1s
reconnectionDelayMax: 30000, // cap at 30s
reconnectionAttempts: Infinity, // never give up
});
// Manual fallback strategy:
function connect() {
try {
return new WebSocket('wss://api.example.com/ws');
} catch {
// Fallback to SSE
return new EventSource('/events');
}
}Helper library vs raw WebSockets
The fallback code above is exactly what helper libraries automate: Upgrade, long-poll fallback, backoff, and rooms, at the cost of a custom protocol layer on top of the standard. Choosing between them is a trade of control against development speed.
Raw WebSockets
- The standard protocol (RFC 6455), with nothing proprietary between your code and the wire.
- No automatic reconnection, so the backoff loop above is yours to write.
- No room or channel concept; you multiplex topics with typed messages yourself.
- No fallback to polling, so blocked upgrades need your own ladder.
- Minimal per-frame overhead (see the comparison table), which matters at very high message rates.
- Best for high-performance systems and custom protocols where control matters most.
WebSocket helper library
- A library layered on top of WebSockets, adding its own message framing.
- Automatic reconnection with backoff, so clients recover without extra code.
- Rooms and namespaces built in, so broadcasting to a group is one call.
- Falls back to long polling when the WebSocket upgrade fails.
- Higher overhead, because the custom protocol layer wraps every message.
- Best for rapid development and compatibility with networks that block upgrades.
At large scale: Slack, voice/chat apps, and trading platforms often use raw WebSockets with custom reconnection for maximum control over performance and protocol behaviour.
Failure timeline, what to watch, and what must never become false
Everything above assumed the happy path. Here is how a routine deploy turns into an outage when sticky routing, missing jitter, and interact, followed by the signals that expose it early and the invariant the system must protect throughout.
Failure timeline: sticky node drain gone wrong
T0 Deploy kills gateway G1; LB still sticky-hashes users → G1 T1 Clients get TCP RST; reconnect immediately with no jitter T2 G2 accepts stampede; auth + Redis subscribe storms; p99 spikes T3 Heartbeats miss → more closes → more reconnects (positive feedback) T4 Drain: mark G1, GOAWAY, jittered reconnect, wait for conn≈0, then stop
Read T3 carefully: missed heartbeats close more sockets, which triggers more reconnects, which delays more heartbeats. Persistent connections make deploys a distributed-systems problem, not a rolling HTTP restart.
What to watch when something feels broken
Connection health
- Concurrent connections per node, so an uneven or overloaded gateway stands out.
- Upgrade success rate and close codes, which reveal proxies or auth rejecting connections.
- Ping/pong RTT and missed-heartbeat count, the earliest sign of dying peers.
- Reconnect rate after deploys or regional failures, to catch a forming stampede.
Message flow
- Inbound and outbound messages per second, the baseline for spotting fan-out spikes.
- Queue depth per room, guild, tenant, or topic, which exposes overloaded hot shards early.
- Slow consumers and dropped messages, because one stuck client can back up others.
- lag between connection servers, which users experience as delayed delivery.
Client reality
- Mobile backgrounding pauses timers and network activity, so heartbeats stop silently.
- Corporate proxies may kill idle upgraded connections even after a successful handshake.
- Browsers need wss://, origin validation, and token refresh while the socket stays open.
- Clients must resume from the last message ID after reconnect to avoid gaps.
Correctness invariant
What must never become false
After a reconnect, a client resumes from a durable cursor (its last message ID or sequence number) so delivery is at-least-once: a message may arrive twice, but it never silently vanishes when the product promised to show it. Connection maps and fan-out stay consistent with the pub/sub backplane, so a message accepted for a room reaches every subscribed socket that still owns that membership, or the client is told it must resync.
Wrap-up
You have seen the handshake, the scaling model, the operational habits, and the ways they fail. Use the exercises to check the ideas stuck, then follow the references for the primary sources.
Exercises
- Size a gateway. 800K connections at 15 KB each plus a 2 KB average outbound buffer. How many boxes for 5M concurrent users with 30% headroom? (Numbers to know above has the per-socket figures.)
- Pick a heartbeat. NAT idle timeout is 90s. Choose a ping interval and a missed-pong policy, then decide what user-visible state you show after a silent drop.
- Design the fallback. A corporate proxy blocks Upgrade. Design the fallback ladder and the message resume contract so chat history does not fork.
Topic-specific references
- RFC 6455: The WebSocket Protocol : handshake, frames, ping/pong, close codes.
- RFC 7692: Compression Extensions for WebSocket : permessage-deflate costs and negotiation.
- WHATWG WebSockets Living Standard : browser API behavior and security considerations.
What to remember
Justify duplex before you pick WebSockets. Externalize connection state, route fan-out through , reconnect with jitter, , and keep an or long-poll fallback for hostile networks.
Security defaults: always , authenticate on upgrade, rate-limit per connection, and validate the Origin header (which page opened the socket).
Practice and further reading
Use these exercises to connect the lesson to a design decision. State your assumptions, trace what happens to one request, and explain the rule that must remain true even during a failure. That rule is the design's correctness invariant.
Focus for this lesson: Operate long-lived bidirectional connections with authentication renewal, ordering, backpressure, resume tokens, fan-out, and reconnect control.
Explain the decision
How do you resume a WebSocket session after reconnect?
Read the reasoning
Client sends its last acknowledged sequence or resume token; server replays retained messages, marks gaps explicitly, and deduplicates client commands.
Apply it to a production scenario
A million clients reconnect after a gateway deploy. How do you avoid collapse?
Read one approach
Drain gradually, use exponential backoff with full jitter, admission caps, resume tokens, autoscaling pre-warm, and regional connection budgets.
Work through the numbers
Estimate heartbeat egress for 1 million connections sending 80-byte frames every 30 seconds.
Read the worked calculation
About 2.67 MB/s or 21.3 Mb/s before framing, TLS, acknowledgements, and regional replication.
Reason through a failure
What ordering invariant should a chat protocol state?
Read the recovery reasoning
Messages in one conversation have a monotonic server sequence; duplicates are allowed only if clients can identify and suppress them.