Design a Live Comments System (like Facebook Live / YouTube Live)

1. The Late Joiner Catch-Up Problem

When a user opens a live video stream that has been broadcasting for an hour, presenting an empty comment panel feels jarring and disconnected. However, querying the persistent Cassandra database on every WebSocket connection would generate an unsustainable read surge during viral viewer influxes.

We solve this bootstrap challenge using a dedicated Redis circular buffer cache. For every active broadcast, Redis maintains a serving list keyed as recent_comments:{stream_id} capped at 100 items using atomic LPUSH and LTRIM operations.

When a new client establishes a WebSocket connection:

1. The gateway registers the client socket with the stream's Redis Pub/Sub channel first and temporarily buffers live frames.
2. The gateway fetches the last 100 comments from the Redis serving list and sends the available backfill.
3. The gateway drains the temporarily buffered live frames. The client application deduplicates by the unique 64-bit comment identifier so the handoff does not create duplicate visible comments.

2. Moderation Pipeline Latency and Shadow-Banning

Filtering toxic content, hate speech, and spam bots cannot introduce perceptible delays into legitimate user conversations. A purely synchronous machine-learning evaluation pipeline adds 50 to 100 milliseconds of latency per comment and causes entire chat feeds to stall if the moderation inference cluster experiences a spike in queue depth.

We employ a three-tier hybrid moderation architecture:

• Fast Synchronous Pre-Filter: An in memory dictionary and regex hash set evaluates incoming comments within 1 millisecond on the Comment Service. Obvious profanity and known abusive patterns are rejected instantly before the comment ever reaches the broadcast or persistence layers.
• Asynchronous Deep Scoring: Clean comments publish immediately to the live broadcast sampler, while simultaneously being written to the live-comments Kafka topic. A pool of background machine learning moderation consumers ingests from Kafka to score contextual toxicity, spam similarity, and user reputation. If a comment is flagged after broadcast, the moderation service records the durable deleted state and a Redis tombstone before emitting a delete_comment control frame over Redis Pub/Sub. Gateways remove the offending comment ID from viewer viewports within approximately 2 seconds after receiving the control frame.
• User Shadow-Banning: When abusive users or spam bots are detected, they are placed on a shadow-ban list stored in Redis (shadow_banned:{stream_id}). Comments submitted by a shadow-banned account return an immediate success acknowledgement to the author, and their client interface displays the comment normally. The comment is still persisted with moderation state set to shadow_banned, but the Comment Service does not forward it to Redis Pub/Sub or other viewers. The author still sees the local result, which makes the restriction less obvious to the abusive account.

3. Ephemerality vs Strict Comment Ordering

In a globally distributed streaming platform, network propagation delays between diverse geographical edge locations mean packets inevitably arrive out of physical order at backend ingress gateways. Attempting to enforce globally synchronized total order across millions of distributed clients would require expensive consensus protocols that ruin real time performance.

Because live chat is ephemeral and fast-scrolling, human viewers cannot distinguish minor timing discrepancies on the order of 50 to 100 milliseconds. We balance ordering and performance using Snowflake identifiers:

• Snowflake ID Generation: When the Comment Service receives a comment, it stamps the record with a 64-bit Snowflake ID generator containing a 41-bit millisecond timestamp, 10-bit machine identifier, and 12-bit sequence number. This provides approximate creation time ordering without requiring distributed synchronization.
• Client-Side Micro-Buffering: During critical scenarios such as live host Q&A sessions where sequential causality matters, client applications buffer incoming frames for 100 to 200 milliseconds, sorting items by Snowflake identifier in memory before appending them to the rendered chat container. This eliminates out of order jitter caused by variable network transit times without stalling the visual presentation.

4. WebSocket Gateway Failure and Thundering Herd Mitigation

A single WebSocket gateway node accommodates up to 100,000 persistent viewer connections. If that gateway crashes or experiences an abrupt hardware fault, 100,000 clients disconnect simultaneously. If all clients instantly attempt to reconnect to the surviving cluster, the resulting reconnect storm can cascade across remaining gateway instances.

We mitigate reconnect storms and ensure uninterrupted chat continuity through three synchronized mechanisms:

• Exponential Backoff with Jitter: Disconnected client applications apply an exponential backoff retry strategy with randomized jitter between 0 and 500 milliseconds. Spreading connection attempts across this window flattens the reconnection spike on edge load balancers.
• Resumption Cursor Handoff: When reconnecting, the client includes its last_seen_comment_id in the WebSocket connection query parameters.
• Server-Side Gap Replay: The new WebSocket gateway first subscribes the socket to the live Redis Pub/Sub channel and temporarily buffers those live frames. It then reads the Redis recent comments buffer (recent_comments:{stream_id}) for entries newer than last_seen_comment_id, sends the available gap, and drains the temporarily buffered live frames with client side deduplication. If the requested cursor is older than the retained Redis buffer, the gateway falls back to historical Cassandra pagination instead of assuming Redis can replay an unbounded gap. This ordering avoids the race between backfill and live subscription.