System Design Problem

Design Quora (Q&A Platform)

Commonly Asked By:QuoraPinterestGoogleMeta

Interview Setup

Interview Prompt

Design Quora, a large scale Q&A platform where users ask questions, write answers, upvote quality responses, and follow topics. Support 100M DAU with a read heavy workload of 30K feed views per second and 10K search queries per second, establishing quality answer ranking as the core differentiator.

Clarifying Questions (ask before designing)

QuestionWhy it matters
Should we deduplicate semantically similar questions at submission time?Ingesting 500K questions daily with semantic overlap fragments knowledge. Deduplication uses embedding similarity comparisons while balancing submission latency.
How should answers be ranked: using raw upvotes, a Wilson score confidence interval, or author domain expertise?Raw upvotes favor controversial answers with high downvote ratios. Wilson score intervals handle low sample sizes gracefully, and expertise weighting elevates domain authorities.
Is search engine optimization and web crawlability a first class requirement?Quora derives significant organic reach from indexed Q&A pages. CDN edge caching TTLs and server side rendering directly determine search engine indexing success.
How should timeline feeds be delivered: via pull on load or push fan-out for followed topics?Baseline browsing traffic is read dominant (1M non feed and non search reads per minute vs 10K answer writes per minute). Dedicated feed and search capacity is modeled separately, and hybrid fan out isolates prolific authors from high volume updates.

Scope

In scope

  • Question deduplication
  • Answer ranking
  • Topic graph
  • Expertise scoring
  • Knowledge base search
  • Capacity estimation with shown math

Out of scope (state explicitly)

  • Full ads auction and monetization stack
  • Building the full moderation infrastructure at scale beyond the basic moderation integration described here
  • Direct messaging and private chat

Functional Requirements

Establish alignment with the interviewer on the fundamental Q&A interactions, covering question submission, answer publishing, voting mechanisms, and personalized feed delivery. Clarify whether search discovery, automated moderation, and push notifications fall within initial scope, while emphasizing that statistical Wilson score ranking and search engine crawlability represent the defining architectural differentiators over a conventional forum.

Interview tip: Highlight Wilson score interval estimation for answer ranking early in the discussion because it distinguishes production quality ranking from naive upvote subtraction.

  • Ask questions: Users submit questions enriched with topic tags and contextual details.
  • Answer questions: Users author rich-text answers, supporting multiple community responses per question.
  • Upvote and downvote: Community members vote on answers and questions to surface high signal responses.
  • Follow topics, questions, and authors: Users subscribe to topics, questions, and contributors to receive relevant timeline updates.
  • Personalized feed: Deliver an aggregated home feed composed of relevant activity across followed topics, questions, and authors.
  • Search discovery: Full-text and semantic keyword search across published questions, answers, and topic tags.
  • Spaces and communities (optional extension): Curated topic centric communities governed by community moderators.
  • Answer requests: Askers invite specific subject matter experts and prolific authors to answer their questions.
  • Collaborative editing: Community wiki editing with transparent revision histories and rollback support.
  • Content moderation: Automated detection and quarantine of spam, hate speech, and low-quality submissions.

Non-Functional Requirements

The system operates under an asymmetric read-to-write workload where feed generation, search retrieval, and page rendering dominate operational resource usage. Interviewers routinely scrutinize search engine optimization through server side rendering, as well as statistical quality ranking where Wilson score intervals prevent low sample anomalies from outranking authoritative answers.

  • Low Latency: Serve personalized home feeds and search queries in under 500 ms at p99.
  • SEO and Crawlability: Public Q&A pages represent the primary organic acquisition funnel, requiring fast server side HTML rendering and aggressive edge CDN caching.
  • High Scalability: Support over 100 million daily active users, 300 million monthly active users, and more than 500 million cataloged questions.
  • Statistical Quality Ranking: Position the most authoritative and reliable answers at the top using confidence scoring rather than chronological or naive upvote sorting.
  • High Availability: Maintain 99.99% system availability for public reading paths through multi-layer caching and graceful read degradation.
  • Read Heavy Optimization: Accommodate an asymmetric traffic profile where baseline non feed and non search reads reach 1,000,000 reads per minute while answer writes reach 10,000 per minute. The dedicated feed and search capacity targets are tracked separately at roughly 30,000 feed views per second and 10,000 search queries per second.

Capacity Estimations

Capacity calculations model an asymmetric social knowledge platform where read traffic and search volume outpace author contributions by orders of magnitude.

MetricCalculationValue
DAUGiven100M
Questions asked / dayGiven500K
Answers posted / dayGiven2M
Votes / dayGiven50M
Feed views / secPeak planning target~30K
Search queries / secPeak planning target~10K
Avg answer sizeGiven2 KB
Total content storageGiven current corpus1 TB questions + 4 TB answers

Throughput and Storage Sizing Details

These calculations derive read amplification, write rates, and multi year storage requirements.

  • Write Throughput: 500,000 questions per day generate approximately 5.8 questions per second, while 2,000,000 answers per day contribute roughly 23.1 answers per second. Peak write traffic reaches roughly 100 writes per second.
  • Voting Ingestion Throughput: 50,000,000 votes per day correspond to an average write rate of roughly 579 votes per second, peaking at approximately 1,500 votes per second during peak global hours.
  • Read Throughput: 100 million daily active users viewing feeds result in roughly 30,000 feed queries per second at peak, while search traffic accounts for approximately 10,000 search queries per second.
  • Annual Storage Growth: 500,000 questions per day at an average size of 2 KB produce roughly 1 GB of question text daily, totaling approximately 365 GB annually. With 2,000,000 answers per day at 2 KB each, answer text adds 4 GB daily or roughly 1.46 TB annually. Over five years, raw text storage requires roughly 9.1 TB before indexing and replication.

Architecture Diagram

Interview tip: Emphasize that search engine optimization requires server rendered HTML for public question pages, so canonical answers must never hide behind client only Single Page Application rendering.

The system architecture partitions operations into three distinct processing paths: content ingestion for questions and answers, asynchronous quality ranking through Wilson score computations, and low latency delivery across personalized feeds and public pages.

Walk the interviewer through the lifecycle from initial question submission through answer authoring, atomic vote updates, and ranked feed aggregation. The read tier pairs an edge Content Delivery Network with Elasticsearch for full text search and Redis for hot timeline caching. Public question pages use server rendered HTML, while search queries use Elasticsearch for global retrieval. The 500 millisecond p99 target depends on cache hit rate, search load, ranking work, and backend capacity.

Loading...

Component Deep Dives

Answer Ranking with Wilson Score

Statistical Confidence Ranking

Naive upvote subtraction creates severe ranking distortions by favoring controversial high traffic answers over high confidence consensus answers. The Wilson score confidence interval solves this problem by modeling ratings as a Bernoulli trial and calculating the statistical lower bound of positive feedback.

The following comparison shows why Wilson score intervals outperform naive upvote subtraction.

Simple upvote - downvote subtraction:
  Answer A: 100 up, 2 down -> score = 98
  Answer B: 5 up, 0 down -> score = 5
  Result: A ranks higher, which appears reasonable.

Failure modes of naive subtraction:
  Answer C: 1 up, 0 down -> score = 1
  Answer D: 500 up, 400 down -> score = 100
  Result: D ranks higher, but it is heavily controversial (44% downvote rate).
  Answer C shows 100% approval but has only 1 vote, yielding low statistical confidence.

Wilson Score Interval (statistical lower bound):
  Considers both the ratio of positive votes and the total sample size.
  Low confidence (few votes): lower bound of the score remains low, preventing premature promotion.
  High confidence (many votes, high positive ratio): lower bound ranks near the observed mean.
  
  Formula (lower bound of 95% confidence interval):
  score = (p + z^2 / (2n) - z * sqrt((p * (1 - p) + z^2 / (4n)) / n)) / (1 + z^2 / n)
  where:
    p = upvotes / total_votes
    n = total_votes (upvotes + downvotes)
    z = 1.96 (for a 95% two-sided confidence interval)

Feed Generation: Hybrid Push and Pull

Hybrid Fan-Out Architecture

The design balances feed freshness and database write amplification across standard users and high follower creators.

Feed generation candidate sources:
  1. New answers posted to followed questions
  2. Newly asked questions tagged with followed topics
  3. Activity updates and answers authored by followed users
  4. Trending and system-recommended questions across related spaces

Hybrid feed model:
  1. Read recent candidate IDs from derived topic and author activity indexes rather than scattering across every question shard
  2. For creators and topics whose follower fan out fits the write budget, push content IDs into follower feed caches through bounded Kafka consumer batches. Partition very large follower sets into bounded buckets so one hot key does not overwhelm a single worker.
  3. For high follower creators or topics, suppress synchronous fan out and merge recent activity during feed reads
  4. Merge candidates, apply ranking and visibility filters, deduplicate content, and return the top 50 items
  5. Store precomputed content IDs and rank scores in Redis feed:{user_id} with a 5-minute TTL

Push mitigation for prolific authors:
  Define a high follower creator as an account whose follower count makes synchronous fan out exceed the configured write budget.
  When such a creator writes an answer, append the answer reference to the author activity index and relevant topic indexes. Followers merge that activity dynamically during read time feed aggregation.

Question Deduplication via Embeddings

Semantic Question Deduplication

The deduplication pipeline detects duplicate and semantically overlapping questions at submission time to concentrate discussion into authoritative knowledge hubs.

Sample input:
  User types: "What is the best programming language for beginners?"
  Existing candidate matches:
    "What programming language should beginners learn?"
    "Best first programming language to learn?"

Real time deduplication pipeline:
  1. On question submission: generate dense vector embedding using BERT or sentence-transformers
  2. Perform Approximate Nearest Neighbor (ANN) search against indexed question embeddings in Faiss or Milvus
  3. If cosine similarity exceeds the tuned 0.85 threshold: prompt the user with "Similar question already exists"
  4. User selection: merge into the existing question or confirm their submission is distinct
  
Consolidation benefit:
  Redirecting duplicate questions into an authoritative thread concentrates answers and discussions,
  avoiding knowledge fragmentation across 50 near identical threads.

Vector index sizing:
  Storage: question_embeddings collection in Milvus (768-dimensional vectors across 500M entries)
  Raw vector payload: 500M x 768 x 4 bytes ≈ 1.536 TB before ANN index structures, metadata, and replicas
  Latency: ~10ms retrieval for top-5 candidate matches as an illustrative benchmark target

API Design

Service Interface

TypeScript client interface defining question lifecycle models, answer structures, atomic voting payloads, follow operations, search results, and feed pagination contracts:

TYPESCRIPT
export type QuestionId = string;
export type AnswerId = string;
export type ContentId = QuestionId | AnswerId;
export type UserId = string;
export type TopicId = string;
export type Cursor = string;
export type PageSize = number;
export type RevisionId = string;
export type IdempotencyKey = string;
export type SearchQuery = string;
export type URL = string;

export type VoteTargetType = "question" | "answer";
export type VoteDirection = "up" | "down";
export type QuestionStatus = "open" | "closed" | "merged";
export type ModerationStatus = "visible" | "quarantined" | "removed";

export interface QuestionSummary {
  questionId: QuestionId;
  canonicalQuestionId?: QuestionId;
  currentRevisionId: RevisionId;
  authorId: UserId;
  title: string;
  details?: string;
  topics: TopicId[];
  url: URL;
  viewCount: number;
  answerCount: number;
  followCount: number;
  upvotes: number;
  downvotes: number;
  status: QuestionStatus;
  moderationStatus: ModerationStatus;
  createdAt: string;
}

export interface AnswerSummary {
  answerId: AnswerId;
  questionId: QuestionId;
  authorId: UserId;
  body: string;
  upvotes: number;
  downvotes: number;
  wilsonScore: number;
  currentRevisionId: RevisionId;
  isAccepted: boolean;
  moderationStatus: ModerationStatus;
  createdAt: string;
}

export interface VoteResult {
  targetType: VoteTargetType;
  targetId: ContentId;
  upvotes: number;
  downvotes: number;
  userVote: VoteDirection | null;
}

export type FeedItem =
  | { itemType: "question"; question: QuestionSummary }
  | { itemType: "answer"; answer: AnswerSummary };

export interface FeedResponse {
  items: FeedItem[];
  nextCursor: Cursor | null;
  hasMore: boolean;
}

export interface AnswerPage {
  items: AnswerSummary[];
  nextCursor: Cursor | null;
  hasMore: boolean;
}

export interface SearchHit {
  question: QuestionSummary;
  bestAnswerSnippet?: string;
}

export interface SearchResponse {
  items: SearchHit[];
  nextCursor: Cursor | null;
  hasMore: boolean;
}

export interface QuoraServiceClient {
  // Submit a new question. Retries with the same idempotency key must not create duplicates.
  askQuestion(
    title: string,
    details: string | undefined,
    topics: TopicId[] | undefined,
    idempotencyKey: IdempotencyKey
  ): Promise<QuestionSummary>;

  // Fetch one question, resolving merged questions through canonicalQuestionId when present.
  getQuestion(questionId: QuestionId): Promise<QuestionSummary>;

  // Fetch a cursor paginated answer list ordered by current answer ranking.
  getAnswers(questionId: QuestionId, pageSize: PageSize, cursor?: Cursor): Promise<AnswerPage>;

  // Post an answer to an existing question. Retries with the same idempotency key must not create duplicates.
  postAnswer(
    questionId: QuestionId,
    body: string,
    idempotencyKey: IdempotencyKey
  ): Promise<AnswerSummary>;

  // Request an answer from a specific author. Repeating the same request is idempotent.
  requestAnswer(questionId: QuestionId, authorId: UserId, idempotencyKey: IdempotencyKey): Promise<void>;

  // Edit an existing question using optimistic concurrency and preserve its revision history.
  editQuestion(
    questionId: QuestionId,
    title: string,
    details: string | undefined,
    baseRevisionId: RevisionId,
    idempotencyKey: IdempotencyKey
  ): Promise<QuestionSummary>;

  // Restore a previous question revision as a new current revision.
  restoreQuestionRevision(
    questionId: QuestionId,
    revisionId: RevisionId,
    idempotencyKey: IdempotencyKey
  ): Promise<QuestionSummary>;

  // Edit an existing answer using optimistic concurrency and preserve its revision history.
  editAnswer(
    answerId: AnswerId,
    body: string,
    baseRevisionId: RevisionId,
    idempotencyKey: IdempotencyKey
  ): Promise<AnswerSummary>;

  // Restore a previous answer revision as a new current revision.
  restoreAnswerRevision(
    answerId: AnswerId,
    revisionId: RevisionId,
    idempotencyKey: IdempotencyKey
  ): Promise<AnswerSummary>;

  // Create or change the user's vote atomically. Repeating the same request is idempotent.
  vote(
    targetType: VoteTargetType,
    targetId: ContentId,
    vote: VoteDirection,
    idempotencyKey: IdempotencyKey
  ): Promise<VoteResult>;

  // Follow a topic, question, or user. Repeating the same request is idempotent.
  followTopic(topicId: TopicId, idempotencyKey: IdempotencyKey): Promise<void>;
  followQuestion(questionId: QuestionId, idempotencyKey: IdempotencyKey): Promise<void>;
  followUser(userId: UserId, idempotencyKey: IdempotencyKey): Promise<void>;

  // Fetch a paginated home feed combining followed topics, questions, and authors.
  getHomeFeed(pageSize: PageSize, cursor?: Cursor): Promise<FeedResponse>;

  // Search public questions using lexical retrieval and ranking.
  searchQuestions(
    query: SearchQuery,
    pageSize: PageSize,
    cursor?: Cursor
  ): Promise<SearchResponse>;
}

Ask Question Endpoint

This HTTP POST endpoint submits new questions with topic tags and contextual details.

HTTP
POST /api/v1/questions HTTP/1.1
Host: api.quora.com
Authorization: Bearer <user_token>
Idempotency-Key: <idempotency_key>
Content-Type: application/json

{
  "title": "What is the best database for analytics?",
  "details": "I am comparing ClickHouse, BigQuery, and Redshift for high throughput OLAP...",
  "topics": ["databases", "analytics", "data-engineering"]
}

HTTP/1.1 201 Created
Content-Type: application/json

{
  "question_id": "q-984210",
  "url": "/q/what-is-the-best-database-for-analytics"
}

Post Answer Endpoint

This HTTP POST endpoint publishes an answer to an existing question.

HTTP
POST /api/v1/questions/q-984210/answers HTTP/1.1
Host: api.quora.com
Authorization: Bearer <user_token>
Idempotency-Key: <idempotency_key>
Content-Type: application/json

{
  "body": "ClickHouse excels for self hosted columnar analytics, while BigQuery offers serverless convenience..."
}

HTTP/1.1 201 Created
Content-Type: application/json

{
  "answer_id": "a-552190"
}

Vote on Content Endpoint

This HTTP POST endpoint records an atomic upvote or downvote against a question or answer.

HTTP
POST /api/v1/votes HTTP/1.1
Host: api.quora.com
Authorization: Bearer <user_token>
Idempotency-Key: <idempotency_key>
Content-Type: application/json

{
  "target_type": "answer",
  "target_id": "a-552190",
  "vote": "up"
}

HTTP/1.1 200 OK
Content-Type: application/json

{
  "target_type": "answer",
  "target_id": "a-552190",
  "upvotes": 153,
  "downvotes": 4,
  "user_vote": "up"
}

Search Questions Endpoint

This HTTP GET endpoint retrieves public questions through Elasticsearch lexical search and multi signal reranking with an opaque pagination cursor:

HTTP
GET /api/v1/search/questions?q=best%20database%20for%20analytics&page_size=20 HTTP/1.1
Host: api.quora.com
Authorization: Bearer <user_token>

HTTP/1.1 200 OK
Content-Type: application/json

{
  "items": [
    {
      "question": {
        "question_id": "q-984210",
        "current_revision_id": "17",
        "author_id": "u-1928",
        "title": "What is the best database for analytics?",
        "url": "/q/what-is-the-best-database-for-analytics",
        "view_count": 15342,
        "answer_count": 87,
        "follow_count": 921,
        "upvotes": 128,
        "downvotes": 6,
        "status": "open",
        "moderation_status": "visible",
        "created_at": "2026-09-07T04:20:00Z"
      },
      "best_answer_snippet": "ClickHouse excels for self hosted columnar analytics..."
    }
  ],
  "next_cursor": "eyJzY29yZSI6MC44OTJ9",
  "has_more": true
}

Common Error Responses

400 Bad Request: invalid input, missing required fields, or malformed JSON payload
401 Unauthorized: missing or invalid authentication token or API key
403 Forbidden: authenticated caller lacks required permissions for this resource
404 Not Found: requested resource ID does not exist
409 Conflict: duplicate write or version conflict, retry with a unique idempotency key
422 Unprocessable Entity: syntactically valid request failed semantic business validation
429 Too Many Requests: rate limit quota exceeded, client should honor Retry-After header
500 Internal Error: unexpected server failure, retry safely with an idempotency key
503 Service Unavailable: downstream dependency is unavailable or overloaded, retry with exponential backoff
504 Gateway Timeout: search index shard responded slowly, narrow query parameters or retry

Data Model

Relational Storage: MySQL via Vitess

The relational schema defines questions, answers, and a composite primary key voting table that enforces uniqueness and supports sharded lookups.

SQL
-- Questions, answers, votes, question_topics, question_follows, answer_requests,
-- and content_revisions are sharded by question_id to co-locate Q&A activity.
-- topic_follows and user_follows are sharded by user_id for subscription reads.
-- IDs are application generated or allocated through Vitess sequences.

CREATE TABLE questions (
    question_id              BIGINT NOT NULL PRIMARY KEY,
    author_id                BIGINT NOT NULL,
    title                    VARCHAR(500) NOT NULL,
    body                     TEXT,
    slug                     VARCHAR(500) NOT NULL,
    view_count               BIGINT DEFAULT 0,
    answer_count             BIGINT DEFAULT 0,
    follow_count             BIGINT DEFAULT 0,
    upvotes                  BIGINT DEFAULT 0,
    downvotes                BIGINT DEFAULT 0,
    status                   ENUM('open','closed','merged') DEFAULT 'open',
    moderation_status        ENUM('visible','quarantined','removed') DEFAULT 'visible',
    canonical_question_id    BIGINT NULL,
    created_at               TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at               TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    FULLTEXT INDEX idx_search (title, body),
    INDEX idx_author (author_id),
    INDEX idx_canonical (canonical_question_id)
);

-- Route slug lookups by a hash vindex on slug so each lookup targets one shard.
-- Preserve logical slug uniqueness through the Vitess vindex or an application-level uniqueness workflow.
CREATE TABLE question_slug_lookup (
    slug                     VARCHAR(500) NOT NULL PRIMARY KEY,
    question_id              BIGINT NOT NULL
);

CREATE TABLE answers (
    answer_id                BIGINT NOT NULL PRIMARY KEY,
    question_id              BIGINT NOT NULL,
    author_id                BIGINT NOT NULL,
    body                     TEXT NOT NULL,
    upvotes                  BIGINT DEFAULT 0,
    downvotes                BIGINT DEFAULT 0,
    wilson_score             DOUBLE DEFAULT 0,
    is_accepted              BOOLEAN DEFAULT FALSE,
    moderation_status        ENUM('visible','quarantined','removed') DEFAULT 'visible',
    created_at               TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at               TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    INDEX idx_question_score (question_id, wilson_score DESC, answer_id),
    INDEX idx_author (author_id)
);

CREATE TABLE votes (
    question_id              BIGINT NOT NULL,
    user_id                  BIGINT NOT NULL,
    target_type              ENUM('question','answer') NOT NULL,
    target_id                BIGINT NOT NULL,
    vote_type                ENUM('up','down') NOT NULL,
    created_at               TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at               TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    PRIMARY KEY (question_id, target_type, target_id, user_id),
    INDEX idx_user_votes (user_id, created_at)
);

-- For target_type='question', target_id equals question_id.
-- For target_type='answer', question_id identifies the parent question.
-- This keeps vote records related to a question within the same question based shard.

CREATE TABLE answer_requests (
    question_id              BIGINT NOT NULL,
    requester_id             BIGINT NOT NULL,
    requested_user_id        BIGINT NOT NULL,
    status                   ENUM('pending','accepted','declined','expired') DEFAULT 'pending',
    created_at               TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    PRIMARY KEY (question_id, requester_id, requested_user_id),
    INDEX idx_requested_user (requested_user_id, created_at)
);

CREATE TABLE content_revisions (
    question_id              BIGINT NOT NULL,
    content_type             ENUM('question','answer') NOT NULL,
    content_id               BIGINT NOT NULL,
    revision_id              BIGINT NOT NULL,
    editor_id                BIGINT NOT NULL,
    content_snapshot         JSON NOT NULL,
    created_at               TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    PRIMARY KEY (question_id, content_type, content_id, revision_id),
    INDEX idx_content_revision (question_id, content_type, content_id, created_at)
);

-- Topics are low-volume metadata and can live in an unsharded/global keyspace.
CREATE TABLE topics (
    topic_id                BIGINT NOT NULL PRIMARY KEY,
    name                    VARCHAR(255) NOT NULL,
    created_at              TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    UNIQUE KEY uq_topic_name (name)
);

CREATE TABLE question_topics (
    question_id              BIGINT NOT NULL,
    topic_id                 BIGINT NOT NULL,
    created_at               TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    PRIMARY KEY (question_id, topic_id),
    INDEX idx_topic_recent (topic_id, created_at DESC, question_id)
);

CREATE TABLE topic_follows (
    user_id                  BIGINT NOT NULL,
    topic_id                 BIGINT NOT NULL,
    created_at               TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    PRIMARY KEY (user_id, topic_id),
    INDEX idx_topic_user (topic_id, user_id)
);

CREATE TABLE question_follows (
    user_id                  BIGINT NOT NULL,
    question_id              BIGINT NOT NULL,
    created_at               TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    PRIMARY KEY (user_id, question_id),
    INDEX idx_question_user (question_id, user_id)
);

CREATE TABLE user_follows (
    follower_id              BIGINT NOT NULL,
    followed_user_id         BIGINT NOT NULL,
    created_at               TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    PRIMARY KEY (follower_id, followed_user_id),
    INDEX idx_followed_follower (followed_user_id, follower_id)
);

-- Durable idempotency records are sharded by user_id and committed with the business mutation.
-- Redis remains the fast path, while MySQL protects correctness if the cache is lost.
CREATE TABLE idempotency_records (
    user_id                  BIGINT NOT NULL,
    idempotency_key          VARCHAR(255) NOT NULL,
    request_hash             BINARY(32) NOT NULL,
    response_payload         JSON NOT NULL,
    created_at               TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    expires_at               TIMESTAMP NOT NULL,
    PRIMARY KEY (user_id, idempotency_key)
);

In-Memory Caching: Redis

Redis stores atomic vote counters, Wilson score sorted sets, user feed pointers, and derived follow indexes.

REDIS
# Derived vote counters for fast reads. MySQL vote rows and counters are authoritative.
HSET vote_count:{type}:{id} up 153 down 4

# Wilson score sorted set for rank ordering within a question
ZADD answer_order:{question_id} 0.892 a-552190
ZADD answer_order:{question_id} 0.741 a-552191

# Precomputed home feed pointers. The score stores the final feed rank.
ZADD feed:{user_id} 0.992 a-552190
ZADD feed:{user_id} 0.947 q-984210
EXPIRE feed:{user_id} 300

# Derived indexes keep feed candidate generation out of scatter reads across question shards.
ZADD topic_recent_questions:{topic_id} <created_at_score> <question_id>
ZREMRANGEBYSCORE topic_recent_questions:{topic_id} -inf <retention_cutoff_score>
ZADD author_activity:{author_id} <timestamp_score> <content_type>:<content_id>
ZREMRANGEBYSCORE author_activity:{author_id} -inf <retention_cutoff_score>
SADD followers:{author_id} <user_id>
SADD question_followers:{question_id} <user_id>
SADD topic_followers:{topic_id} <user_id>

# Redis idempotency cache for safe retries. Durable MySQL idempotency records remain authoritative.
SET idem:{user_id}:{idempotency_key} <request_hash_and_result> NX EX 86400

# High volume view counts are aggregated outside the primary write path and flushed asynchronously.
HINCRBY view_count:{type}:{id} views 1

# Materialized question counters such as answer_count and follow_count should use batched or sharded
# counter paths for very hot questions instead of an independent primary-row write for every event.

Fault Tolerance

Resilience and Failure Mitigations

The following table maps system failure modes to operational concerns and architectural mitigations.

ConcernSolution
Vote manipulationEnforce rate limits per IP and account while counting ranking weight only from accounts older than 7 days
Duplicate votesComposite PRIMARY KEY including question_id, target_type, target_id, and user_id prevents duplicate vote entries within the logical parent question
Search index lagElasticsearch updated through Debezium CDC from the MySQL or Vitess source with replication lag bounded under 30 seconds
SEO stalenessCDN edge cache configured with 5-minute TTL combined with asynchronous event driven invalidation triggered immediately after a committed content change
Answer qualityMulti-stage moderation combining automated ML spam classification, community report signals, and human moderation review queues

Vote Count and Wilson Score Concurrency

Handling race conditions in concurrent voting using atomic database increments paired with asynchronous Wilson score reconciliation:

Concurrent vote collision trace:
  T=0: upvotes=100, downvotes=4
  Thread A: reads (100, 4) -> votes up -> writes upvotes=101
  Thread B: reads (100, 4) -> votes up -> writes upvotes=101 (Lost update anomaly)

Mitigation strategy: Atomic transaction
  INSERT or update the user's vote row and adjust the target counters in one transaction.
  For a new upvote:
    UPDATE answers SET upvotes = upvotes + 1 WHERE answer_id = ?;
  If an existing downvote changes to an upvote:
    UPDATE answers SET downvotes = downvotes - 1, upvotes = upvotes + 1 WHERE answer_id = ?;

Score reconciliation pipeline:
  1. Relational database records the user's vote and updates the target counters atomically without a read modify write race.
  2. The same transaction handles vote direction changes by decrementing the old counter and incrementing the new counter.
  3. An asynchronous worker recomputes wilson_score from the updated (upvotes, downvotes) totals.
  4. Redis vote counters and answer_order sorted sets are refreshed from authoritative database state.
  5. Score recomputations execute in 30-second batches rather than firing per individual vote event.

Additional Considerations

Related Problems and Core Concepts

Knowledge discovery and community Q&A systems link directly to discussion ranking architectures explored in Reddit Discussion Forum and timeline fan-out pipelines in News Feed System. Handling spam detection, abusive behavior, and report triage connects to Content Moderation System, while direct user-to-user messaging is examined in Real-Time Chat. For approximate nearest-neighbor vector deduplication, review Vector Database and Semantic Search. To master foundational scaling concepts, explore Indexing and Query Optimization, Caching Patterns and Invalidation, Sharding and Partitioning, CDN and Edge Delivery, and System Design Interview Patterns.

Interview Walkthrough

  • 25-minute cut

    Skip the deep dives on author expertise and search scaling unless interviewing for a staff level role.

    • Anchor on core Q&A entities including questions, answers, topics, and votes (5 min)
    • Rank answers using Wilson score confidence intervals rather than raw upvote counts (6 min)
    • Mandate server side rendering with Next.js and CDN caching for organic SEO crawlability (5 min)
    • Shard MySQL horizontally with Vitess by question ID and cache hot pages at the edge (5 min)
    • Generate feeds with hybrid push and pull fan-out across followed topics and authors (4 min)
  • Anchor discussion on fundamental Q&A domain entities including questions, answers, topics, and votes, clarifying that browsing and search reads outnumber posting and voting writes by two orders of magnitude.
  • Establish answer ranking based on Wilson score confidence intervals rather than naive upvote totals, ensuring that a single-vote answer cannot rank above an authoritative response with hundreds of upvotes.
  • Mandate server side rendering for public question pages using Next.js, because web crawlers require complete semantic HTML and client only single page applications lose significant organic search traffic.
  • Shard MySQL horizontally using Vitess partitioned by question ID, and cache popular question pages at edge CDN locations with a 5-minute time to live.
  • Implement personalized feed generation with a hybrid fan-out model that pulls recent updates from followed topics on demand while fanning out posts for creators with manageable follower counts.
  • Deploy dedicated Elasticsearch clusters for full text lexical search across questions and answers, applying multi signal reranking based on quality scores and topic filters.
  • Handle concurrent voting through atomic database increments like UPDATE answers SET upvotes = upvotes + 1, batching asynchronous Wilson score recalculations every 30 seconds.
  • Warn against the common pitfall of ranking answers by simple upvote minus downvote subtraction, because controversial answers with high downvote rates will unfairly dominate high confidence domain responses.

Engineering Trade-offs

Server Side Rendering vs Client Side Rendering for SEO

Crawlability and indexing efficiency directly influence organic discovery for question and answer platforms.

Client Side Rendering (Single Page Application):
  Execution flow: Web crawler requests HTML, receives an empty shell div, and must execute JavaScript to render content.
  Disadvantages: Crawlers may defer or inconsistently execute JavaScript, which can delay indexing and make public content less reliably discoverable.

Server-Side Rendering with Edge Caching (Production standard):
  Execution flow: Node.js or Next.js servers render full semantic HTML with complete question and answer text before transmitting the payload.
  Advantages: Crawlers instantly index complete semantic text without rendering delays or JavaScript execution risks.
  Edge caching: Full HTML documents are cached across CloudFront edge locations with a 5-minute TTL.
  Dynamic personalization: Client side hydration overlays live vote counters, bookmark states, and personalized recommendations after initial paint.
  Impact: Assuming search engines generate a major share of platform traffic, server side rendering is the preferred production approach for public Q&A pages.

Relational MySQL via Vitess vs Document Store

This comparison evaluates ACID transaction guarantees, join relationships, and unbounded document scaling.

Relational Model (MySQL via Vitess):
  Advantages:
    - Structured relationships across questions, answers, and votes map cleanly to relational foreign keys.
    - Transactional row updates provide strong consistency for atomic vote tallies.
    - MySQL FULLTEXT can provide an early fallback search path. Production retrieval uses Elasticsearch with BM25 and additional ranking signals.
    - Vitess enables transparent horizontal sharding based on question_id keys.
  Trade-offs:
    - Schema evolution across billions of records requires managed online schema migration tooling.

Document Model (MongoDB or DocumentDB):
  Advantages:
    - Flexible document schema accommodates varying answer metadata and rich media attachments.
    - Embedding answers inside question documents allows single read document retrieval.
  Trade-offs:
    - Unbounded document growth risks violating the 16 MB BSON document size ceiling on popular questions.
    - Lack of relational joins pushes vote tally aggregation and cross entity consistency into application level code.

Multi Signal Search Ranking Pipeline

This pipeline executes full text lexical matching followed by engagement weighted reranking.

Search execution flow for query: "best database for analytics"

Pipeline stages:
  1. Lexical retrieval: Query the Elasticsearch cluster across denormalized question title, question body, answer text, and topic tag fields using BM25 relevance scoring. Return question documents and deduplicate answer or topic matches by question_id.
  2. Eligibility filtering: Exclude closed, merged, removed, or spam flagged content from the candidate set.
  3. Engagement reranking: Score the remaining bounded candidate set using a weighted feature combination:
     search_score = w1 * text_relevance (BM25)
                  + w2 * answer_quality (precomputed quality aggregate from top ranked eligible answers)
                  + w3 * view_count (historical question engagement)
                  + w4 * recency (temporal decay factor favoring fresh updates)
                  + w5 * follow_count (community subscriber volume)
  4. Presentation: Return the top 20 ranked questions enriched with the highest-scoring eligible answer snippet.

💬Review

Help Us Improve

How helpful was this walkthrough?

Click a star to rate. We actively use this feedback to refine and update our system design content.

Placeholder
Optional but highly appreciated!

Discussion

Share your thoughts, ask questions, or help others.

Loading comments...