Design a Real-Time Chat Application (WhatsApp / Slack)

WebSocket Connection Gateway Layer

The gateway fleet manages persistent full-duplex TCP connections, handling socket keep-alives, connection heartbeats, and client reconnection storms.

• Transport Protocol: WebSocket provides full-duplex framing with tiny headers (2 to 14 bytes) compared to HTTP request-response overhead.
• Connection Density: Each instance runs an event-driven networking runtime (Netty or Go epoll/kqueue) managing approximately 100,000 concurrent sockets with 64 KB buffers.
• Load Balancing: L4 load balancers distribute long-lived TCP connections across the gateway fleet, while TLS termination is handled by a TLS-aware edge proxy or gateway ingress, with the Session Service in Redis acting as the authoritative registry for user device locations.
• Session Heartbeats: Clients dispatch lightweight ping frames every 30 seconds. If no heartbeat arrives within 90 seconds, the gateway terminates the connection and removes the device session.

Session Registry Service (Redis)

The Session Service maintains an in-memory directory mapping user IDs to their active device sessions, server identifiers, and socket descriptors.

• Multi-Device Data Model: Stored as sessions:{user_id} containing a hash of active device records, where each field maps a device_id to its host server_id, connection timestamp, and last heartbeat.
• Automatic Expiration: Each device session carries a 5-minute time-to-live (TTL) refreshed periodically by client heartbeats, ensuring disconnected devices clean up automatically without stale routing entries.

Message Router (Cross-Server Communication)

The Message Router bridges isolated gateway instances so that devices connected to different physical servers can communicate with minimal latency.

• Fast Path (Redis Pub/Sub): For online-to-online direct messages, the gateway publishes the payload to a channel named after the recipient's server_id, achieving sub-5 ms cross-pod routing. Because the message is already safely persisted in Cassandra before routing, any dropped Pub/Sub notification or network glitch is safely recovered by the client via Cassandra sync upon reconnect.
• Durable Asynchronous Streams (Kafka): For offline wake-up alerts, large group fan-out, and search indexing pipelines, messages route through partitioned Kafka topics to provide backpressure buffering and replay capabilities. Kafka is strictly used for asynchronous background jobs and not as an ordinary mailbox for direct messages.

Message Persistence & Sequencing Service

The Message Service coordinates message idempotency, assigns deterministic sequence identifiers, and commits records to Cassandra before acknowledging delivery to the sender.

• Identity, Idempotency, and Ordering: The client attaches a clientMessageId for retry idempotency, enabling safe resends without duplicate processing if an ACK is lost. The Message Service assigns a globally unique 64-bit Snowflake message_id for internal server-side identity and correlation. Within the specific conversation, it assigns a monotonically increasing conversation_seq to establish an authoritative, deterministic total order.
• Sequence Allocation Mechanism: To prevent race conditions and duplicate sequence numbers across concurrent senders, the Message Service coordinates sequencing per conversation. Writes for a given conversation_id route through a conversation partition sequencer (such as an assigned consistent-hash worker node or atomic sequence counter lease) that increments and assigns the next sequential integer atomically before writing to Cassandra.
• Write-Before-ACK Guarantee: Messages are written to Cassandra with local quorum consistency before the server returns an acknowledgment frame ({status: "sent", server_message_id, conversation_seq}) to the sender, ensuring durability against server node crashes before confirming receipt.

Cassandra Message Store Architecture & Bucket-Aware Sync

Cassandra serves as the primary time-series message repository, optimized for high-throughput write workloads and chronological range scans within bounded partitions.

• Bounded Partition Key: ((conversation_id, bucket), conversation_seq) groups messages by conversation and sequence range bucket (such as bucket = conversation_seq / 100000). This bounds partition size to under 100 MB even for active groups with millions of messages, preventing hot partition degradation.
• Clustering Key: conversation_seq DESC stores rows physically sorted by descending conversation sequence number, enabling single-seek retrieval of recent chat history.
• Bucket-Aware Sync Progression: When a client issues a sync request with its sync cursor (last_synced_conversation_seq), the Message Service calculates the starting bucket (last_synced_conversation_seq / 100000) and queries Cassandra for rows where conversation_seq > last_synced_conversation_seq within that bucket. If the returned batch is smaller than the requested limit and newer sequence buckets exist, the service queries subsequent buckets until the page size is fulfilled or the latest sequence is reached.
• LSM-Tree Storage Engine: Append-only sequential writes handle over 58,000 peak writes per second without random disk seek bottlenecks.

Real-Time User Presence and Heartbeats

The presence subsystem tracks online status and last-seen timestamps without generating broadcast storms across large contact lists.

• Heartbeat Mechanism: Active device sessions maintain presence:{user_id} = "online" in Redis with a 60-second TTL, refreshed every 30 seconds as long as at least one device session is active.
• Throttled Presence Fan-Out: Rather than broadcasting presence changes to all 10,000 contacts of a user, the system broadcasts updates only to mutual contacts in active open chat windows, batching updates every 5 seconds.

Group Message Fan-Out Service

The Group Service manages membership rosters, admin permissions, and distribution strategies for multi-party conversations.

• Private Small Groups (≤ 256 members): Uses fan-out on write (push model). The gateway synchronously queries active device session records in Redis and delivers frames directly to connected participant gateway instances via Redis Pub/Sub fast path.
• Large Public Channels (1,000+ members): For large enterprise channels (such as Slack or Discord), fan-out on write creates quadratic write amplification. The system persists the message once in the channel timeline, and clients fetch updates via fan-out on read (pull model) or subscribe to channel event rooms.
• Group End-to-End Encryption (Sender Keys): Group messaging under E2EE uses Signal-style Sender Keys. Each sender maintains a ratchet-based sender-key state for the group and distributes the symmetric key material to other members through pairwise 1:1 encrypted channels (using Double Ratchet). When group membership changes (a member leaves or is removed), the sender rotates the sender key to preserve backward and forward secrecy.

Media Ingestion and CDN Distribution

Multimedia assets (images, videos, documents) bypass the stateful WebSocket connection to keep socket buffers lightweight and responsive.

• Client-Side Encryption: For E2EE chats, the client generates an ephemeral symmetric key, encrypts the media locally, and uploads the encrypted blob directly to Amazon S3 via a signed HTTP PUT URL.
• Encrypted Thumbnails: The client generates a low-resolution thumbnail preview, encrypts it locally, and embeds the ciphertext thumbnail along with the encrypted media key inside the WebSocket message payload. Recipients download the encrypted blob from the CDN and decrypt it client-side.

Push Notification Integration

When a recipient has no active device sessions, the gateway delegates message alerting to native mobile push notification networks.

• Trigger Condition: If the Session Service reports no active sessions in Redis, the gateway publishes a wake-up event to the push-notifications Kafka topic.
• Privacy and Collapsing: Push notifications convey generic alerts (such as "Alice sent 5 messages") without exposing private message ciphertext or sensitive plaintext to Apple APNs or Google FCM. The durable message remains in Cassandra, ready to be fetched when the recipient launches the app.

End-to-End Encryption (Signal Protocol Architecture)

For privacy-first messaging (such as WhatsApp), the server operates solely as an untrusted ciphertext router and public key directory.

• Key Agreement (X3DH): Users publish pre-key bundles (identity key, signed pre-key, one-time pre-keys) to the E2EE Key Service. Initiators fetch the bundle and derive shared session secrets without server knowledge.
• Ratchet Forward Secrecy & Healing: The Double Ratchet algorithm derives fresh ephemeral encryption keys for every message. This ensures forward secrecy (protecting past messages if current ephemeral keys are compromised) and post-compromise security (healing future session security once a new Diffie-Hellman ratchet exchange occurs).

Kafka Event Streaming Topology

Detailed topic layout, partition keys, and consumer group responsibilities:

Topic: group-fanout
  Partitions: 64
  Partition key: group_id
  Retention: 24 hours
  Producer: Group Service when coordinating asynchronous fan-out for groups with many participants
  Event Schema: { message_id, conversation_id, conversation_seq, sender_id, payload, timestamp }

Topic: push-notifications
  Partitions: 64
  Partition key: user_id
  Retention: 7 days
  Producer: Chat Gateway Server when recipient has no active sessions
  Event Schema: { user_id, conversation_id, sender_name, unread_count, collapse_key }

Consumer Groups:
  1. group-fanout-workers: Resolves group membership rosters and dispatches to member chat servers via Redis Pub/Sub
  2. push-workers: Dispatches batched push notifications to Apple APNs and Google FCM
  Dead Letter Queue: push-notifications-dlq after 3 retries

Synchronous Ingress Flow:
  Client WebSocket frame -> Chat Gateway validates auth & payload -> Message Service allocates monotonically increasing conversation_seq and persists to Cassandra with local quorum -> Server ACK returned to sender -> Chat Gateway queries Session Service in Redis for active recipient device sockets

Cross-Pod Delivery & Offline Catch-Up Flow:
  If recipient is ONLINE on another server -> Publish to recipient server Redis Pub/Sub channel (fast path)
  If delivery fails, socket disconnects, or session is stale -> No message is lost; Cassandra holds durable copy
  If recipient has no active sessions -> Publish wake-up event to push-notifications Kafka topic
  Authoritative recovery -> Offline and reconnecting clients sync missing messages directly from Cassandra by querying where conversation_seq > last_synced_conversation_seq across sequence buckets