Design a Follower/Following System

Write Path: Follow Action

Walk through the complete follow and unfollow pipeline and how counter values remain consistent under concurrent mutations.

When Alice follows Bob, the system validates permissions (ensuring Alice is not blocked, is not following herself, and does not hold an existing edge), writes an atomic Cassandra BATCH across both tables, updates Redis sorted sets and counters, and publishes an event to Kafka. The entire write path executes in under 100 ms p99.

Follow write sequence:
1. Check block list: GET block:{alice}:{bob}, returning 403 if blocked
2. Idempotency: SET follow:idem:{key} NX EX 86400, returning cached response if key exists
3. Cassandra BATCH:
   INSERT INTO following (alice, now, bob)
   INSERT INTO followers (bob, bucket(alice), alice, now)
   INSERT INTO is_following (alice, bob)
4. Redis pipeline: ZADD following:{alice}, ZADD followers:{bob} (if < 100K)
   INCR follower_count:{bob}, INCR following_count:{alice}
5. Kafka publish: { event: "follow", actor, target, ts }
6. Return 201 Created

Event Bus (Kafka)

Follow writes return quickly after persisting to Cassandra and Redis. The Kafka follow-events topic drives asynchronous feed fan-out, push notifications, and suggestion index updates without adding latency to the client response.

Topic: follow-events
  Partitions: 64
  Partition key: followee_id (groups fan-out work per target user)
  Retention: 7 days
  Replication factor: 3, min.insync.replicas: 2

Producer: Follow Service after Cassandra BATCH + Redis cache update
  Event: { event_id, event: "follow" | "unfollow", actor_id, target_id, timestamp }

Consumer groups:
  1. feed-fanout: LPUSH new posts to follower feeds (skipped if target > 100K followers)
  2. notification: push alerts for "Alice followed you"
  3. suggestion-indexer: update mutual-friend graphs incrementally
  4. analytics: stream follow and unfollow trends to ClickHouse

Sync path: validate, execute Cassandra dual write, update Redis, publish event, and return 201 Created (< 100ms p99).
Async path: feed fan-out and notifications tolerate up to 5 seconds of lag.
DLQ: follow-events-dlq, alerting when feed-fanout lag exceeds 30 seconds.

Is-Following Lookup: 500K QPS Hot Path

Every profile visit, timeline load, and direct messaging authorization check evaluates follow status. The lookup follows a three-tier hierarchy:

1. Redis Cache: Key is_following:{a}:{b} returns in 1 ms with a 95% hit rate.
2. Cassandra Table: Performs an O(1) primary key lookup in approximately 5 ms on cache misses.
3. Negative Caching: Writes is_following:{a}:{b} = "0" with a 5-minute TTL to shield the database from repeated missing lookups.

Pipelined batch queries check 50 candidate connections in a single Redis MGET round trip when rendering mutual connections.

Celebrity Hot Partition: 100M Followers

Celebrity accounts apply compound partition keys: PRIMARY KEY ((user_id, bucket), follower_id) using bucket = hash(follower_id) % 100. This distributes 100M follower edges across 100 partitions of 1M rows each, ensuring stable compaction and read performance. Total counts update via atomic Redis INCR and DECR commands rather than executing expensive SELECT COUNT(*) queries.

Fan-out threshold: Accounts with over 100K followers switch dynamically to fan-out-on-read. Delivering a post to 100M inboxes via fan-out-on-write would require 100M Redis operations and ~6.4 GB of memory, which is operationally unsustainable. This hybrid architecture matches modern social network standards.

Unfollow, Block, and Consistency

Unfollow: Deletes rows from both tables, removes members from Redis sorted sets, and decrements counters atomically.

Block: Inserts into the blocks table, deletes existing follow relationships, and purges entries from both follower caches. The block check executes before every follow mutation.

Count Reconciliation: A scheduled Spark job aggregates edges per user, compares totals against Redis counters, and generates alerts if drift exceeds 1%. Remediation applies through administrative tooling rather than automatic overwrites to prevent masking counter bugs.

Follow Suggestions: Friends-of-Friends

Mutual connections calculate by gathering the following lists of users Alice follows, aggregating overlapping targets, ranking by shared frequency, and filtering out accounts already followed or blocked. An offline Spark job runs daily to populate the top 50 suggestions in Redis under suggestions:{uid}.

For celebrity profiles, the system intersects the viewing user's following list (~500 accounts) against the celebrity's followers using pipelined checks rather than materializing the full 100M-member set.