Design a real-time chat system

The mobile, desktop, or browser client first resolves a service address and reaches a healthy edge. Static assets such as avatars, stickers, and application bundles can come from a CDN, while private chat traffic continues to an authenticated gateway. Geographic routing reduces avoidable travel time, but the conversation’s write owner and data location determine where a durable send must ultimately run.

The edge terminates the encrypted connection where appropriate, rejects oversized or excessive requests, and routes accepted traffic to the right service. DDoS filtering and a web application firewall reduce unwanted load; application authorization still decides whether this user may send to this conversation. Keep connection establishment separate from membership checks because permissions can change while a socket remains open.

A real-time gateway holds each device’s connection, validates session state, and exchanges heartbeats to detect broken paths. A connection registry maps a user and device to the gateway currently holding its socket. Presence is an approximate view derived from those heartbeats; it can briefly remain online after a disconnection, so delivery cannot rely on presence as proof that a message arrived.

Delivery workers locate the current gateway, send a frame, and track the device acknowledgment separately from durable message acceptance. A stale registry entry or failed gateway is repaired by reconnect and history replay. Typing indicators expire and can be dropped under pressure; durable messages remain in history. Bound each device’s outgoing buffer so a slow receiver cannot exhaust gateway memory.

Ingestion authenticates the sender, checks current conversation permissions and payload limits, and associates a stable client_message_id with the send. The conversation owner assigns an ordered sequence and records the message durably before returning a durable-acceptance acknowledgment. Deduplication must recover the same identifier and sequence on retry; simply generating another sequence number does not make a repeated send safe.

A replicated log such as Kafka or Pulsar can be the accepted-message source while persistence workers build queryable history. Route a conversation’s events to the same partition and preserve its processing order; many conversations can share that partition. Unread counts, delivery fanout, and permitted search projections consume the accepted event independently, so a slow secondary feature does not block durable ingestion.

Store message history by conversation and sequence, adding a time bucket when an indefinitely growing conversation would create an oversized partition. Cassandra or Scylla are possible query-oriented stores, but the chosen database must meet the required durability and pagination behavior. Keep identity, membership, roles, and settings in an authoritative catalog with a clear version for permission decisions.

Images and attachments belong in object storage, with message rows referencing their authorized object versions. A CDN can deliver permitted media without making objects public by accident. Search is a derived projection and may lag; with end-to-end encryption, the server cannot build a plaintext content index from ciphertext. Deletion and retention workflows must reconcile message metadata, attachment references, and any searchable copies.

When a device is offline, a push service uses its platform token to send a bounded notification, often collapsing several updates into one conversation alert. That alert is a prompt to sync, not the durable message itself. On reconnect, the client requests messages after its last applied sequence; the server replays gaps and makes duplicates harmless. An expired history cursor requires a stated resynchronization policy.

Group management records role changes and membership revocation independently of presence. Apply the current access policy to sending, history reads, and delivery to removed members; an eventually refreshed member list must not silently extend revoked access. Delivered and displayed receipts use separate progress records, often per device. A displayed receipt reports a client action, not proof that a human read or understood the message.

With end-to-end encryption, sending devices encrypt for the intended recipient devices, and the server stores and routes ciphertext with the metadata needed for delivery. Device key verification, key rotation, recovery, and adding or removing devices become product requirements. Use an established protocol and supported libraries rather than inventing encryption from a diagram.

Private key material stays under client control, while a service may distribute public key bundles and device-directory information. A new group membership or device change can require new encryption state to protect future messages. Server-side plaintext search and automatic content inspection are limited by this boundary; scope search, reporting, backups, and moderation according to what participants deliberately reveal.

Measure the stages users experience: send to durable acknowledgment, accepted message to device delivery, reconnect to completed catch-up, and the age of undelivered work. Trace a stable message identifier through gateway, ingestion, log, history store, and delivery without logging private message bodies. Monitor connection counts and outgoing-buffer pressure because a quiet CPU graph can hide failing real-time delivery.

Operations need controlled account restrictions, feature rollout, backups, and a tested regional recovery plan. A replicated message log and the membership catalog must recover to compatible states so failover does not grant stale permissions or forget accepted messages. Additional regions require explicit write ownership and spare capacity; active-active deployment is a design choice with conflict and operating costs, not a default checkbox.