Design a Mentions & Tagging System

Mention Extraction and Validation

The write path must be fast and idempotent: parse mentions once at publish time, validate against permissions and block lists, persist rows in a separate mentions table, and hand off side effects to asynchronous consumers.

The client sends structured mention metadata alongside post text, where each entry includes the @username, character offset, and length so the server does not re-parse on every read. On publish, the server batch-resolves usernames to user IDs via a Redis hash, checks mention permissions (everyone, followers only, or nobody), and consults the block list before inserting rows. Invalid or blocked mentions are stored as plain text with no notification. Rate limits (20 mentions per post, 50 per hour per author) run in Redis sliding windows before any insert.

Edit-After-Publish Mention Diffing

Users frequently edit captions after publishing, requiring mention sets to diff cleanly without triggering duplicate alerts. On PATCH requests, compare parsed mention sets against stored records, notify only net-new recipients, and mark removed mentions inactive without mutating historical post body text.

User edits post text after publish:
  Original: "Lunch with @alice"
  Edited:   "Lunch with @alice and @bob"

Diff algorithm (on PATCH /posts/{id}):
  1. Parse old and new mention sets from stored offsets + fresh regex pass
  2. added = new_mentions - old_mentions   -> insert rows + Kafka notify
  3. removed = old_mentions - new_mentions -> mark status='removed', no re-notify
  4. unchanged = intersection              -> skip

Idempotency: UNIQUE(content_id, mentioned_user) prevents duplicate rows on re-save.
Notification policy: notify only on newly added mentions, never on removal.
Rate limit edits: max 5 edits/post/hour to prevent mention-spam via edit loop.

Mention Spam Rate Limits

Automated spam syndicates attempt to bypass feed filters by tagging high volumes of strangers. Defenses apply sliding window counters in Redis across author, target, and minute burst dimensions before insertion.

Per-author limits (Redis counters, sliding window):
  max 20 mentions per post
  max 50 mentions per hour per author
  max 200 mentions per day per author
  max 10 distinct users mentioned per minute (burst)

Per-target protection:
  Celebrity digest mode: >100 mentions/min -> batch notifications
  Block list: mention stored but notification suppressed
  New account (<7 days): cap 30 mentions/day until trust score rises

Abuse signals trigger shadow suppression:
  >80% mentions to non-followers
  Same @target spammed from >20 accounts in 1 hour
  Autocomplete enumeration: >100 prefix queries/min -> 429

Autocomplete Architecture

At 50K autocomplete queries per second, a database LIKE prefix scan will not survive, so the typeahead path requires sub-50ms latency. A two-tier strategy maintains responsive user interactions: the client caches the user following list from the Social Graph Store for instant suggestions among familiar connections, while the server handles global lookups via a Redis sorted set with lexicographic range queries:

ZRANGEBYLEX usernames "[al" "[al\xff" LIMIT 0 5

Results re-rank based on social proximity, mutual connections, popularity, and interaction recency. The full username index occupies approximately 20 GB of memory across 1B accounts.

Photo and Video Tagging

Photo tags differ structurally from text mentions: each tag records normalized coordinates in the 0 to 1 range so positions stay correct across thumbnail and full-resolution renders. Tags persist in a dedicated table indexed by tagged_user_idto power "photos of me" queries, dispatching alerts that read "Alice tagged you in a photo".

Event Bus Design (Kafka)

Once mention rows commit, the publish path emits a single event keyed by mentioned_user_id so all downstream consumers, including push notifications, the mentions inbox feed, and analytics, process in parallel without blocking the author. For messaging architecture patterns, review Message Queues Fundamentals.

Topic: mentions
  Partitions: 64
  Partition key: mentioned_user_id (routes notifications to correct shard)
  Retention: 7 days
  Replication factor: 3, min.insync.replicas: 2

Producer: Mention Service after validation + permission check
  Event: { event_id, post_id, author_id, mentioned_user_ids[], content_type, timestamp }

Consumer groups:
  1. notification: push and email dispatch to each mentioned user
  2. mention-feed: compile the "tagged in" feed index per user
  3. analytics: stream mention volume metrics to ClickHouse

Sync path: POST content -> store post -> validate mentions -> publish -> 201 Created.
Async path: notifications and feed indexes proceed without blocking post creation.
DLQ: mentions-dlq, alerting when notification consumer lag exceeds 30 seconds.