Interview Setup
Interview Prompt
Design Quora, a large scale Q&A platform where users ask questions, write answers, upvote quality responses, and follow topics. Support 100M DAU with a read heavy workload of 30K feed views per second and 10K search queries per second, establishing quality answer ranking as the core differentiator.
Clarifying Questions (ask before designing)
| Question | Why it matters |
|---|---|
| Should we deduplicate semantically similar questions at submission time? | Ingesting 500K questions daily with semantic overlap fragments knowledge. Deduplication uses embedding similarity comparisons while balancing submission latency. |
| How should answers be ranked: using raw upvotes, a Wilson score confidence interval, or author domain expertise? | Raw upvotes favor controversial answers with high downvote ratios. Wilson score intervals handle low sample sizes gracefully, and expertise weighting elevates domain authorities. |
| Is search engine optimization and web crawlability a first class requirement? | Quora derives significant organic reach from indexed Q&A pages. CDN edge caching TTLs and server side rendering directly determine search engine indexing success. |
| How should timeline feeds be delivered: via pull on load or push fan-out for followed topics? | Baseline browsing traffic is read dominant (1M non feed and non search reads per minute vs 10K answer writes per minute). Dedicated feed and search capacity is modeled separately, and hybrid fan out isolates prolific authors from high volume updates. |
Scope
In scope
- Question deduplication
- Answer ranking
- Topic graph
- Expertise scoring
- Knowledge base search
- Capacity estimation with shown math
Out of scope (state explicitly)
- Full ads auction and monetization stack
- Building the full moderation infrastructure at scale beyond the basic moderation integration described here
- Direct messaging and private chat
Functional Requirements
Establish alignment with the interviewer on the fundamental Q&A interactions, covering question submission, answer publishing, voting mechanisms, and personalized feed delivery. Clarify whether search discovery, automated moderation, and push notifications fall within initial scope, while emphasizing that statistical Wilson score ranking and search engine crawlability represent the defining architectural differentiators over a conventional forum.
Interview tip: Highlight Wilson score interval estimation for answer ranking early in the discussion because it distinguishes production quality ranking from naive upvote subtraction.
- Ask questions: Users submit questions enriched with topic tags and contextual details.
- Answer questions: Users author rich-text answers, supporting multiple community responses per question.
- Upvote and downvote: Community members vote on answers and questions to surface high signal responses.
- Follow topics, questions, and authors: Users subscribe to topics, questions, and contributors to receive relevant timeline updates.
- Personalized feed: Deliver an aggregated home feed composed of relevant activity across followed topics, questions, and authors.
- Search discovery: Full-text and semantic keyword search across published questions, answers, and topic tags.
- Spaces and communities (optional extension): Curated topic centric communities governed by community moderators.
- Answer requests: Askers invite specific subject matter experts and prolific authors to answer their questions.
- Collaborative editing: Community wiki editing with transparent revision histories and rollback support.
- Content moderation: Automated detection and quarantine of spam, hate speech, and low-quality submissions.
Non-Functional Requirements
The system operates under an asymmetric read-to-write workload where feed generation, search retrieval, and page rendering dominate operational resource usage. Interviewers routinely scrutinize search engine optimization through server side rendering, as well as statistical quality ranking where Wilson score intervals prevent low sample anomalies from outranking authoritative answers.
- Low Latency: Serve personalized home feeds and search queries in under 500 ms at p99.
- SEO and Crawlability: Public Q&A pages represent the primary organic acquisition funnel, requiring fast server side HTML rendering and aggressive edge CDN caching.
- High Scalability: Support over 100 million daily active users, 300 million monthly active users, and more than 500 million cataloged questions.
- Statistical Quality Ranking: Position the most authoritative and reliable answers at the top using confidence scoring rather than chronological or naive upvote sorting.
- High Availability: Maintain 99.99% system availability for public reading paths through multi-layer caching and graceful read degradation.
- Read Heavy Optimization: Accommodate an asymmetric traffic profile where baseline non feed and non search reads reach 1,000,000 reads per minute while answer writes reach 10,000 per minute. The dedicated feed and search capacity targets are tracked separately at roughly 30,000 feed views per second and 10,000 search queries per second.
Capacity Estimations
Capacity calculations model an asymmetric social knowledge platform where read traffic and search volume outpace author contributions by orders of magnitude.
| Metric | Calculation | Value |
|---|---|---|
| DAU | Given | 100M |
| Questions asked / day | Given | 500K |
| Answers posted / day | Given | 2M |
| Votes / day | Given | 50M |
| Feed views / sec | Peak planning target | ~30K |
| Search queries / sec | Peak planning target | ~10K |
| Avg answer size | Given | 2 KB |
| Total content storage | Given current corpus | 1 TB questions + 4 TB answers |
Throughput and Storage Sizing Details
These calculations derive read amplification, write rates, and multi year storage requirements.
- Write Throughput: 500,000 questions per day generate approximately 5.8 questions per second, while 2,000,000 answers per day contribute roughly 23.1 answers per second. Peak write traffic reaches roughly 100 writes per second.
- Voting Ingestion Throughput: 50,000,000 votes per day correspond to an average write rate of roughly 579 votes per second, peaking at approximately 1,500 votes per second during peak global hours.
- Read Throughput: 100 million daily active users viewing feeds result in roughly 30,000 feed queries per second at peak, while search traffic accounts for approximately 10,000 search queries per second.
- Annual Storage Growth: 500,000 questions per day at an average size of 2 KB produce roughly 1 GB of question text daily, totaling approximately 365 GB annually. With 2,000,000 answers per day at 2 KB each, answer text adds 4 GB daily or roughly 1.46 TB annually. Over five years, raw text storage requires roughly 9.1 TB before indexing and replication.
Architecture Diagram
Interview tip: Emphasize that search engine optimization requires server rendered HTML for public question pages, so canonical answers must never hide behind client only Single Page Application rendering.
The system architecture partitions operations into three distinct processing paths: content ingestion for questions and answers, asynchronous quality ranking through Wilson score computations, and low latency delivery across personalized feeds and public pages.
Walk the interviewer through the lifecycle from initial question submission through answer authoring, atomic vote updates, and ranked feed aggregation. The read tier pairs an edge Content Delivery Network with Elasticsearch for full text search and Redis for hot timeline caching. Public question pages use server rendered HTML, while search queries use Elasticsearch for global retrieval. The 500 millisecond p99 target depends on cache hit rate, search load, ranking work, and backend capacity.
Component Deep Dives
Answer Ranking with Wilson Score
Statistical Confidence Ranking
Naive upvote subtraction creates severe ranking distortions by favoring controversial high traffic answers over high confidence consensus answers. The Wilson score confidence interval solves this problem by modeling ratings as a Bernoulli trial and calculating the statistical lower bound of positive feedback.
The following comparison shows why Wilson score intervals outperform naive upvote subtraction.
Simple upvote - downvote subtraction:
Answer A: 100 up, 2 down -> score = 98
Answer B: 5 up, 0 down -> score = 5
Result: A ranks higher, which appears reasonable.
Failure modes of naive subtraction:
Answer C: 1 up, 0 down -> score = 1
Answer D: 500 up, 400 down -> score = 100
Result: D ranks higher, but it is heavily controversial (44% downvote rate).
Answer C shows 100% approval but has only 1 vote, yielding low statistical confidence.
Wilson Score Interval (statistical lower bound):
Considers both the ratio of positive votes and the total sample size.
Low confidence (few votes): lower bound of the score remains low, preventing premature promotion.
High confidence (many votes, high positive ratio): lower bound ranks near the observed mean.
Formula (lower bound of 95% confidence interval):
score = (p + z^2 / (2n) - z * sqrt((p * (1 - p) + z^2 / (4n)) / n)) / (1 + z^2 / n)
where:
p = upvotes / total_votes
n = total_votes (upvotes + downvotes)
z = 1.96 (for a 95% two-sided confidence interval)Feed Generation: Hybrid Push and Pull
Hybrid Fan-Out Architecture
The design balances feed freshness and database write amplification across standard users and high follower creators.
Feed generation candidate sources:
1. New answers posted to followed questions
2. Newly asked questions tagged with followed topics
3. Activity updates and answers authored by followed users
4. Trending and system-recommended questions across related spaces
Hybrid feed model:
1. Read recent candidate IDs from derived topic and author activity indexes rather than scattering across every question shard
2. For creators and topics whose follower fan out fits the write budget, push content IDs into follower feed caches through bounded Kafka consumer batches. Partition very large follower sets into bounded buckets so one hot key does not overwhelm a single worker.
3. For high follower creators or topics, suppress synchronous fan out and merge recent activity during feed reads
4. Merge candidates, apply ranking and visibility filters, deduplicate content, and return the top 50 items
5. Store precomputed content IDs and rank scores in Redis feed:{user_id} with a 5-minute TTL
Push mitigation for prolific authors:
Define a high follower creator as an account whose follower count makes synchronous fan out exceed the configured write budget.
When such a creator writes an answer, append the answer reference to the author activity index and relevant topic indexes. Followers merge that activity dynamically during read time feed aggregation.Question Deduplication via Embeddings
Semantic Question Deduplication
The deduplication pipeline detects duplicate and semantically overlapping questions at submission time to concentrate discussion into authoritative knowledge hubs.
Sample input:
User types: "What is the best programming language for beginners?"
Existing candidate matches:
"What programming language should beginners learn?"
"Best first programming language to learn?"
Real time deduplication pipeline:
1. On question submission: generate dense vector embedding using BERT or sentence-transformers
2. Perform Approximate Nearest Neighbor (ANN) search against indexed question embeddings in Faiss or Milvus
3. If cosine similarity exceeds the tuned 0.85 threshold: prompt the user with "Similar question already exists"
4. User selection: merge into the existing question or confirm their submission is distinct
Consolidation benefit:
Redirecting duplicate questions into an authoritative thread concentrates answers and discussions,
avoiding knowledge fragmentation across 50 near identical threads.
Vector index sizing:
Storage: question_embeddings collection in Milvus (768-dimensional vectors across 500M entries)
Raw vector payload: 500M x 768 x 4 bytes ≈ 1.536 TB before ANN index structures, metadata, and replicas
Latency: ~10ms retrieval for top-5 candidate matches as an illustrative benchmark targetAPI Design
Service Interface
TypeScript client interface defining question lifecycle models, answer structures, atomic voting payloads, follow operations, search results, and feed pagination contracts:
export type QuestionId = string;
export type AnswerId = string;
export type ContentId = QuestionId | AnswerId;
export type UserId = string;
export type TopicId = string;
export type Cursor = string;
export type PageSize = number;
export type RevisionId = string;
export type IdempotencyKey = string;
export type SearchQuery = string;
export type URL = string;
export type VoteTargetType = "question" | "answer";
export type VoteDirection = "up" | "down";
export type QuestionStatus = "open" | "closed" | "merged";
export type ModerationStatus = "visible" | "quarantined" | "removed";
export interface QuestionSummary {
questionId: QuestionId;
canonicalQuestionId?: QuestionId;
currentRevisionId: RevisionId;
authorId: UserId;
title: string;
details?: string;
topics: TopicId[];
url: URL;
viewCount: number;
answerCount: number;
followCount: number;
upvotes: number;
downvotes: number;
status: QuestionStatus;
moderationStatus: ModerationStatus;
createdAt: string;
}
export interface AnswerSummary {
answerId: AnswerId;
questionId: QuestionId;
authorId: UserId;
body: string;
upvotes: number;
downvotes: number;
wilsonScore: number;
currentRevisionId: RevisionId;
isAccepted: boolean;
moderationStatus: ModerationStatus;
createdAt: string;
}
export interface VoteResult {
targetType: VoteTargetType;
targetId: ContentId;
upvotes: number;
downvotes: number;
userVote: VoteDirection | null;
}
export type FeedItem =
| { itemType: "question"; question: QuestionSummary }
| { itemType: "answer"; answer: AnswerSummary };
export interface FeedResponse {
items: FeedItem[];
nextCursor: Cursor | null;
hasMore: boolean;
}
export interface AnswerPage {
items: AnswerSummary[];
nextCursor: Cursor | null;
hasMore: boolean;
}
export interface SearchHit {
question: QuestionSummary;
bestAnswerSnippet?: string;
}
export interface SearchResponse {
items: SearchHit[];
nextCursor: Cursor | null;
hasMore: boolean;
}
export interface QuoraServiceClient {
// Submit a new question. Retries with the same idempotency key must not create duplicates.
askQuestion(
title: string,
details: string | undefined,
topics: TopicId[] | undefined,
idempotencyKey: IdempotencyKey
): Promise<QuestionSummary>;
// Fetch one question, resolving merged questions through canonicalQuestionId when present.
getQuestion(questionId: QuestionId): Promise<QuestionSummary>;
// Fetch a cursor paginated answer list ordered by current answer ranking.
getAnswers(questionId: QuestionId, pageSize: PageSize, cursor?: Cursor): Promise<AnswerPage>;
// Post an answer to an existing question. Retries with the same idempotency key must not create duplicates.
postAnswer(
questionId: QuestionId,
body: string,
idempotencyKey: IdempotencyKey
): Promise<AnswerSummary>;
// Request an answer from a specific author. Repeating the same request is idempotent.
requestAnswer(questionId: QuestionId, authorId: UserId, idempotencyKey: IdempotencyKey): Promise<void>;
// Edit an existing question using optimistic concurrency and preserve its revision history.
editQuestion(
questionId: QuestionId,
title: string,
details: string | undefined,
baseRevisionId: RevisionId,
idempotencyKey: IdempotencyKey
): Promise<QuestionSummary>;
// Restore a previous question revision as a new current revision.
restoreQuestionRevision(
questionId: QuestionId,
revisionId: RevisionId,
idempotencyKey: IdempotencyKey
): Promise<QuestionSummary>;
// Edit an existing answer using optimistic concurrency and preserve its revision history.
editAnswer(
answerId: AnswerId,
body: string,
baseRevisionId: RevisionId,
idempotencyKey: IdempotencyKey
): Promise<AnswerSummary>;
// Restore a previous answer revision as a new current revision.
restoreAnswerRevision(
answerId: AnswerId,
revisionId: RevisionId,
idempotencyKey: IdempotencyKey
): Promise<AnswerSummary>;
// Create or change the user's vote atomically. Repeating the same request is idempotent.
vote(
targetType: VoteTargetType,
targetId: ContentId,
vote: VoteDirection,
idempotencyKey: IdempotencyKey
): Promise<VoteResult>;
// Follow a topic, question, or user. Repeating the same request is idempotent.
followTopic(topicId: TopicId, idempotencyKey: IdempotencyKey): Promise<void>;
followQuestion(questionId: QuestionId, idempotencyKey: IdempotencyKey): Promise<void>;
followUser(userId: UserId, idempotencyKey: IdempotencyKey): Promise<void>;
// Fetch a paginated home feed combining followed topics, questions, and authors.
getHomeFeed(pageSize: PageSize, cursor?: Cursor): Promise<FeedResponse>;
// Search public questions using lexical retrieval and ranking.
searchQuestions(
query: SearchQuery,
pageSize: PageSize,
cursor?: Cursor
): Promise<SearchResponse>;
}Ask Question Endpoint
This HTTP POST endpoint submits new questions with topic tags and contextual details.
POST /api/v1/questions HTTP/1.1
Host: api.quora.com
Authorization: Bearer <user_token>
Idempotency-Key: <idempotency_key>
Content-Type: application/json
{
"title": "What is the best database for analytics?",
"details": "I am comparing ClickHouse, BigQuery, and Redshift for high throughput OLAP...",
"topics": ["databases", "analytics", "data-engineering"]
}
HTTP/1.1 201 Created
Content-Type: application/json
{
"question_id": "q-984210",
"url": "/q/what-is-the-best-database-for-analytics"
}Post Answer Endpoint
This HTTP POST endpoint publishes an answer to an existing question.
POST /api/v1/questions/q-984210/answers HTTP/1.1
Host: api.quora.com
Authorization: Bearer <user_token>
Idempotency-Key: <idempotency_key>
Content-Type: application/json
{
"body": "ClickHouse excels for self hosted columnar analytics, while BigQuery offers serverless convenience..."
}
HTTP/1.1 201 Created
Content-Type: application/json
{
"answer_id": "a-552190"
}Vote on Content Endpoint
This HTTP POST endpoint records an atomic upvote or downvote against a question or answer.
POST /api/v1/votes HTTP/1.1
Host: api.quora.com
Authorization: Bearer <user_token>
Idempotency-Key: <idempotency_key>
Content-Type: application/json
{
"target_type": "answer",
"target_id": "a-552190",
"vote": "up"
}
HTTP/1.1 200 OK
Content-Type: application/json
{
"target_type": "answer",
"target_id": "a-552190",
"upvotes": 153,
"downvotes": 4,
"user_vote": "up"
}Search Questions Endpoint
This HTTP GET endpoint retrieves public questions through Elasticsearch lexical search and multi signal reranking with an opaque pagination cursor:
GET /api/v1/search/questions?q=best%20database%20for%20analytics&page_size=20 HTTP/1.1
Host: api.quora.com
Authorization: Bearer <user_token>
HTTP/1.1 200 OK
Content-Type: application/json
{
"items": [
{
"question": {
"question_id": "q-984210",
"current_revision_id": "17",
"author_id": "u-1928",
"title": "What is the best database for analytics?",
"url": "/q/what-is-the-best-database-for-analytics",
"view_count": 15342,
"answer_count": 87,
"follow_count": 921,
"upvotes": 128,
"downvotes": 6,
"status": "open",
"moderation_status": "visible",
"created_at": "2026-09-07T04:20:00Z"
},
"best_answer_snippet": "ClickHouse excels for self hosted columnar analytics..."
}
],
"next_cursor": "eyJzY29yZSI6MC44OTJ9",
"has_more": true
}Common Error Responses
400 Bad Request: invalid input, missing required fields, or malformed JSON payload 401 Unauthorized: missing or invalid authentication token or API key 403 Forbidden: authenticated caller lacks required permissions for this resource 404 Not Found: requested resource ID does not exist 409 Conflict: duplicate write or version conflict, retry with a unique idempotency key 422 Unprocessable Entity: syntactically valid request failed semantic business validation 429 Too Many Requests: rate limit quota exceeded, client should honor Retry-After header 500 Internal Error: unexpected server failure, retry safely with an idempotency key 503 Service Unavailable: downstream dependency is unavailable or overloaded, retry with exponential backoff 504 Gateway Timeout: search index shard responded slowly, narrow query parameters or retry
Data Model
Relational Storage: MySQL via Vitess
The relational schema defines questions, answers, and a composite primary key voting table that enforces uniqueness and supports sharded lookups.
-- Questions, answers, votes, question_topics, question_follows, answer_requests,
-- and content_revisions are sharded by question_id to co-locate Q&A activity.
-- topic_follows and user_follows are sharded by user_id for subscription reads.
-- IDs are application generated or allocated through Vitess sequences.
CREATE TABLE questions (
question_id BIGINT NOT NULL PRIMARY KEY,
author_id BIGINT NOT NULL,
title VARCHAR(500) NOT NULL,
body TEXT,
slug VARCHAR(500) NOT NULL,
view_count BIGINT DEFAULT 0,
answer_count BIGINT DEFAULT 0,
follow_count BIGINT DEFAULT 0,
upvotes BIGINT DEFAULT 0,
downvotes BIGINT DEFAULT 0,
status ENUM('open','closed','merged') DEFAULT 'open',
moderation_status ENUM('visible','quarantined','removed') DEFAULT 'visible',
canonical_question_id BIGINT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
FULLTEXT INDEX idx_search (title, body),
INDEX idx_author (author_id),
INDEX idx_canonical (canonical_question_id)
);
-- Route slug lookups by a hash vindex on slug so each lookup targets one shard.
-- Preserve logical slug uniqueness through the Vitess vindex or an application-level uniqueness workflow.
CREATE TABLE question_slug_lookup (
slug VARCHAR(500) NOT NULL PRIMARY KEY,
question_id BIGINT NOT NULL
);
CREATE TABLE answers (
answer_id BIGINT NOT NULL PRIMARY KEY,
question_id BIGINT NOT NULL,
author_id BIGINT NOT NULL,
body TEXT NOT NULL,
upvotes BIGINT DEFAULT 0,
downvotes BIGINT DEFAULT 0,
wilson_score DOUBLE DEFAULT 0,
is_accepted BOOLEAN DEFAULT FALSE,
moderation_status ENUM('visible','quarantined','removed') DEFAULT 'visible',
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
INDEX idx_question_score (question_id, wilson_score DESC, answer_id),
INDEX idx_author (author_id)
);
CREATE TABLE votes (
question_id BIGINT NOT NULL,
user_id BIGINT NOT NULL,
target_type ENUM('question','answer') NOT NULL,
target_id BIGINT NOT NULL,
vote_type ENUM('up','down') NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (question_id, target_type, target_id, user_id),
INDEX idx_user_votes (user_id, created_at)
);
-- For target_type='question', target_id equals question_id.
-- For target_type='answer', question_id identifies the parent question.
-- This keeps vote records related to a question within the same question based shard.
CREATE TABLE answer_requests (
question_id BIGINT NOT NULL,
requester_id BIGINT NOT NULL,
requested_user_id BIGINT NOT NULL,
status ENUM('pending','accepted','declined','expired') DEFAULT 'pending',
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (question_id, requester_id, requested_user_id),
INDEX idx_requested_user (requested_user_id, created_at)
);
CREATE TABLE content_revisions (
question_id BIGINT NOT NULL,
content_type ENUM('question','answer') NOT NULL,
content_id BIGINT NOT NULL,
revision_id BIGINT NOT NULL,
editor_id BIGINT NOT NULL,
content_snapshot JSON NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (question_id, content_type, content_id, revision_id),
INDEX idx_content_revision (question_id, content_type, content_id, created_at)
);
-- Topics are low-volume metadata and can live in an unsharded/global keyspace.
CREATE TABLE topics (
topic_id BIGINT NOT NULL PRIMARY KEY,
name VARCHAR(255) NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
UNIQUE KEY uq_topic_name (name)
);
CREATE TABLE question_topics (
question_id BIGINT NOT NULL,
topic_id BIGINT NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (question_id, topic_id),
INDEX idx_topic_recent (topic_id, created_at DESC, question_id)
);
CREATE TABLE topic_follows (
user_id BIGINT NOT NULL,
topic_id BIGINT NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (user_id, topic_id),
INDEX idx_topic_user (topic_id, user_id)
);
CREATE TABLE question_follows (
user_id BIGINT NOT NULL,
question_id BIGINT NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (user_id, question_id),
INDEX idx_question_user (question_id, user_id)
);
CREATE TABLE user_follows (
follower_id BIGINT NOT NULL,
followed_user_id BIGINT NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (follower_id, followed_user_id),
INDEX idx_followed_follower (followed_user_id, follower_id)
);
-- Durable idempotency records are sharded by user_id and committed with the business mutation.
-- Redis remains the fast path, while MySQL protects correctness if the cache is lost.
CREATE TABLE idempotency_records (
user_id BIGINT NOT NULL,
idempotency_key VARCHAR(255) NOT NULL,
request_hash BINARY(32) NOT NULL,
response_payload JSON NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
expires_at TIMESTAMP NOT NULL,
PRIMARY KEY (user_id, idempotency_key)
);In-Memory Caching: Redis
Redis stores atomic vote counters, Wilson score sorted sets, user feed pointers, and derived follow indexes.
# Derived vote counters for fast reads. MySQL vote rows and counters are authoritative.
HSET vote_count:{type}:{id} up 153 down 4
# Wilson score sorted set for rank ordering within a question
ZADD answer_order:{question_id} 0.892 a-552190
ZADD answer_order:{question_id} 0.741 a-552191
# Precomputed home feed pointers. The score stores the final feed rank.
ZADD feed:{user_id} 0.992 a-552190
ZADD feed:{user_id} 0.947 q-984210
EXPIRE feed:{user_id} 300
# Derived indexes keep feed candidate generation out of scatter reads across question shards.
ZADD topic_recent_questions:{topic_id} <created_at_score> <question_id>
ZREMRANGEBYSCORE topic_recent_questions:{topic_id} -inf <retention_cutoff_score>
ZADD author_activity:{author_id} <timestamp_score> <content_type>:<content_id>
ZREMRANGEBYSCORE author_activity:{author_id} -inf <retention_cutoff_score>
SADD followers:{author_id} <user_id>
SADD question_followers:{question_id} <user_id>
SADD topic_followers:{topic_id} <user_id>
# Redis idempotency cache for safe retries. Durable MySQL idempotency records remain authoritative.
SET idem:{user_id}:{idempotency_key} <request_hash_and_result> NX EX 86400
# High volume view counts are aggregated outside the primary write path and flushed asynchronously.
HINCRBY view_count:{type}:{id} views 1
# Materialized question counters such as answer_count and follow_count should use batched or sharded
# counter paths for very hot questions instead of an independent primary-row write for every event.Fault Tolerance
Resilience and Failure Mitigations
The following table maps system failure modes to operational concerns and architectural mitigations.
| Concern | Solution |
|---|---|
| Vote manipulation | Enforce rate limits per IP and account while counting ranking weight only from accounts older than 7 days |
| Duplicate votes | Composite PRIMARY KEY including question_id, target_type, target_id, and user_id prevents duplicate vote entries within the logical parent question |
| Search index lag | Elasticsearch updated through Debezium CDC from the MySQL or Vitess source with replication lag bounded under 30 seconds |
| SEO staleness | CDN edge cache configured with 5-minute TTL combined with asynchronous event driven invalidation triggered immediately after a committed content change |
| Answer quality | Multi-stage moderation combining automated ML spam classification, community report signals, and human moderation review queues |
Vote Count and Wilson Score Concurrency
Handling race conditions in concurrent voting using atomic database increments paired with asynchronous Wilson score reconciliation:
Concurrent vote collision trace:
T=0: upvotes=100, downvotes=4
Thread A: reads (100, 4) -> votes up -> writes upvotes=101
Thread B: reads (100, 4) -> votes up -> writes upvotes=101 (Lost update anomaly)
Mitigation strategy: Atomic transaction
INSERT or update the user's vote row and adjust the target counters in one transaction.
For a new upvote:
UPDATE answers SET upvotes = upvotes + 1 WHERE answer_id = ?;
If an existing downvote changes to an upvote:
UPDATE answers SET downvotes = downvotes - 1, upvotes = upvotes + 1 WHERE answer_id = ?;
Score reconciliation pipeline:
1. Relational database records the user's vote and updates the target counters atomically without a read modify write race.
2. The same transaction handles vote direction changes by decrementing the old counter and incrementing the new counter.
3. An asynchronous worker recomputes wilson_score from the updated (upvotes, downvotes) totals.
4. Redis vote counters and answer_order sorted sets are refreshed from authoritative database state.
5. Score recomputations execute in 30-second batches rather than firing per individual vote event.Additional Considerations
Related Problems and Core Concepts
Knowledge discovery and community Q&A systems link directly to discussion ranking architectures explored in Reddit Discussion Forum and timeline fan-out pipelines in News Feed System. Handling spam detection, abusive behavior, and report triage connects to Content Moderation System, while direct user-to-user messaging is examined in Real-Time Chat. For approximate nearest-neighbor vector deduplication, review Vector Database and Semantic Search. To master foundational scaling concepts, explore Indexing and Query Optimization, Caching Patterns and Invalidation, Sharding and Partitioning, CDN and Edge Delivery, and System Design Interview Patterns.
Interview Walkthrough
- 25-minute cut
Skip the deep dives on author expertise and search scaling unless interviewing for a staff level role.
- Anchor on core Q&A entities including questions, answers, topics, and votes (5 min)
- Rank answers using Wilson score confidence intervals rather than raw upvote counts (6 min)
- Mandate server side rendering with Next.js and CDN caching for organic SEO crawlability (5 min)
- Shard MySQL horizontally with Vitess by question ID and cache hot pages at the edge (5 min)
- Generate feeds with hybrid push and pull fan-out across followed topics and authors (4 min)
- Anchor discussion on fundamental Q&A domain entities including questions, answers, topics, and votes, clarifying that browsing and search reads outnumber posting and voting writes by two orders of magnitude.
- Establish answer ranking based on Wilson score confidence intervals rather than naive upvote totals, ensuring that a single-vote answer cannot rank above an authoritative response with hundreds of upvotes.
- Mandate server side rendering for public question pages using Next.js, because web crawlers require complete semantic HTML and client only single page applications lose significant organic search traffic.
- Shard MySQL horizontally using Vitess partitioned by question ID, and cache popular question pages at edge CDN locations with a 5-minute time to live.
- Implement personalized feed generation with a hybrid fan-out model that pulls recent updates from followed topics on demand while fanning out posts for creators with manageable follower counts.
- Deploy dedicated Elasticsearch clusters for full text lexical search across questions and answers, applying multi signal reranking based on quality scores and topic filters.
- Handle concurrent voting through atomic database increments like
UPDATE answers SET upvotes = upvotes + 1, batching asynchronous Wilson score recalculations every 30 seconds. - Warn against the common pitfall of ranking answers by simple upvote minus downvote subtraction, because controversial answers with high downvote rates will unfairly dominate high confidence domain responses.
Engineering Trade-offs
Server Side Rendering vs Client Side Rendering for SEO
Crawlability and indexing efficiency directly influence organic discovery for question and answer platforms.
Client Side Rendering (Single Page Application): Execution flow: Web crawler requests HTML, receives an empty shell div, and must execute JavaScript to render content. Disadvantages: Crawlers may defer or inconsistently execute JavaScript, which can delay indexing and make public content less reliably discoverable. Server-Side Rendering with Edge Caching (Production standard): Execution flow: Node.js or Next.js servers render full semantic HTML with complete question and answer text before transmitting the payload. Advantages: Crawlers instantly index complete semantic text without rendering delays or JavaScript execution risks. Edge caching: Full HTML documents are cached across CloudFront edge locations with a 5-minute TTL. Dynamic personalization: Client side hydration overlays live vote counters, bookmark states, and personalized recommendations after initial paint. Impact: Assuming search engines generate a major share of platform traffic, server side rendering is the preferred production approach for public Q&A pages.
Relational MySQL via Vitess vs Document Store
This comparison evaluates ACID transaction guarantees, join relationships, and unbounded document scaling.
Relational Model (MySQL via Vitess):
Advantages:
- Structured relationships across questions, answers, and votes map cleanly to relational foreign keys.
- Transactional row updates provide strong consistency for atomic vote tallies.
- MySQL FULLTEXT can provide an early fallback search path. Production retrieval uses Elasticsearch with BM25 and additional ranking signals.
- Vitess enables transparent horizontal sharding based on question_id keys.
Trade-offs:
- Schema evolution across billions of records requires managed online schema migration tooling.
Document Model (MongoDB or DocumentDB):
Advantages:
- Flexible document schema accommodates varying answer metadata and rich media attachments.
- Embedding answers inside question documents allows single read document retrieval.
Trade-offs:
- Unbounded document growth risks violating the 16 MB BSON document size ceiling on popular questions.
- Lack of relational joins pushes vote tally aggregation and cross entity consistency into application level code.Multi Signal Search Ranking Pipeline
This pipeline executes full text lexical matching followed by engagement weighted reranking.
Search execution flow for query: "best database for analytics"
Pipeline stages:
1. Lexical retrieval: Query the Elasticsearch cluster across denormalized question title, question body, answer text, and topic tag fields using BM25 relevance scoring. Return question documents and deduplicate answer or topic matches by question_id.
2. Eligibility filtering: Exclude closed, merged, removed, or spam flagged content from the candidate set.
3. Engagement reranking: Score the remaining bounded candidate set using a weighted feature combination:
search_score = w1 * text_relevance (BM25)
+ w2 * answer_quality (precomputed quality aggregate from top ranked eligible answers)
+ w3 * view_count (historical question engagement)
+ w4 * recency (temporal decay factor favoring fresh updates)
+ w5 * follow_count (community subscriber volume)
4. Presentation: Return the top 20 ranked questions enriched with the highest-scoring eligible answer snippet.Review
How helpful was this walkthrough?
Click a star to rate. We actively use this feedback to refine and update our system design content.
Discussion
Share your thoughts, ask questions, or help others.