System Design Problem

Design Google Docs (Real-Time Collaborative Editing)

Commonly Asked By:GoogleMicrosoftAtlassianFigma

Interview Setup

Interview Prompt

Design Google Docs: real-time collaborative document editing where multiple users type simultaneously, see each other's cursors, and converge to the same document state. 50M DAU, 500K operations/sec globally.

Clarifying Questions (ask before designing)

QuestionWhy it matters
OT or CRDT, and does offline editing matter?OT needs central server ordering. CRDT can merge peer-to-peer on reconnect. Google Docs used OT, while Figma and Yjs use CRDT.
How many simultaneous editors per document is typical vs max?2 to 5 typical, 200 max. Above 200, switch to view-only mode or batch presence updates.
Must undo/redo work across collaborative edits?Per-user undo stack is isolated from others' ops. Requires operational transform on the undo stack too.
What's the document size limit and how does it affect op log storage?50 KB text + 500 KB embeds average. 500 TB version history with delta compression over 30 days.

Scope

In scope

  • OT (Operational Transformation) vs CRDT
  • Cursor position sync
  • Version vectors
  • WebSocket session management
  • Conflict-free merges
  • Undo/redo in collaborative context

Out of scope (state explicitly)

  • Client desktop/mobile app implementation
  • End-user file preview rendering for every format
  • Building raw block storage hardware

Functional Requirements

Start by asking your interviewer which conflict resolution approach is expected, specifically OT or CRDT. Confirm offline editing, presence cursors, and version history scope for this round. The real-time transport patterns are covered in Network Protocols: HTTP, gRPC, WebSocket & DNS.

  • Multiple users can simultaneously edit the same document in real time
  • Each user sees a live cursor and selections of other collaborators
  • Changes appear on all clients within 100 to 200ms of being made
  • Full version history with the ability to view and restore any retained past version
  • Commenting and suggestion mode (track changes)
  • Offline editing with automatic conflict resolution on reconnect
  • Rich text formatting: bold, italic, headings, lists, tables, images
  • Permissions: owner, editor, commenter, viewer
  • Document sharing via link with configurable access levels
  • Export to PDF, DOCX, plain text

Non-Functional Requirements

Your interviewer will care most about convergence guarantees and sub-200 ms remote edit latency. Call out OT vs CRDT early because it determines whether the design needs a single ordering authority.

  • Low Latency: Local keystrokes are reflected instantly. Remote changes become visible within 100 to 200ms
  • Consistency: All clients must eventually converge to the same document state. A CRDT can provide strong eventual consistency when its merge and delivery assumptions hold
  • Availability: 99.99%. Editing must continue even if some servers are down
  • Scalability: Support 500M documents and up to 200 concurrent editors per actively coedited document
  • Durability: Zero acknowledged edit loss under the stated failure model. An edit is accepted only after the operation log confirms durable quorum storage
  • Conflict Resolution: Concurrent edits must merge automatically without user intervention
  • Offline Support: Edits queued locally and synced on reconnect
  • Security: Encryption in transit (TLS) and at rest, RBAC, audit logs

Capacity Estimations

Run this math before assigning document servers. Concurrent editors per actively coedited document and operations per second determine WebSocket fan-out, while snapshot frequency drives storage growth. Treat the 10M active-document assumption as open or viewed documents, because not every active document is simultaneously coedited. The separate 20M WebSocket assumption captures concurrently connected users.

MetricCalculationValue
Total documentsGiven (assumption documented in value)500M
DAUGiven (product assumption)50M users
Concurrently active docsGiven. Peak load assumption for open or viewed documents10M
Editors per actively coedited docGiven. Typical collaboration assumption2-5 (up to 200 max)
Operations/sec (global)From Operations/day ÷ 86400 (+ peak factor in value)500K
Avg document sizeGiven. Typical workload assumption50 KB (text) + 500 KB (images/embeds)
Storage (docs)500M x 550 KB~275 TB
Version history (30 days)Given~500 TB (with delta compression)
WebSocket connectionsGiven. Concurrent editor connections are separate from the 10M active or viewed documents20M concurrent

Architecture Diagram

In the room, ask about OT vs CRDT before drawing. This is the architectural fork that determines the conflict resolution design.

Walk your interviewer through the per-document server model first. For the OT path shown here, one collaboration server per active document owns operation ordering. Clients send operations over WebSocket. The server transforms concurrent edits, persists durable operations and snapshots, and broadcasts the resulting document state to connected peers.

Loading...

Core Design Decision: OT vs CRDT

AspectOT (Operational Transformation)CRDT (Conflict-free Replicated Data Type)
How it worksTransform concurrent ops relative to shared server sequenceEach op carries unique IDs. Merge is commutative, associative, and idempotent
Single ordering server required?Yes: central server assigns canonical orderNo. Replicas can merge without one ordering authority. Servers can still provide relay and persistence
Offline supportHarder (ops must be rebased)Natural (merge on reconnect)
Used byGoogle Docs (original), SharePointFigma, Yjs, Automerge, Apple Notes
Design choiceUseful when a central ordering server is acceptableUseful when offline editing and decentralized merging are priorities

Collaboration Server: Why One Per Document

  • Each actively coedited document is assigned to exactly ONE collaboration server
  • Avoids distributed coordination for operation ordering on the OT path
  • The collaboration server holds actively coedited document state in memory for fast operation processing
  • If the server dies, another server loads the latest snapshot and replays the durable operation log
  • ZooKeeper owns the document assignment lease. Gateways use the cached assignment or consistent-hash lookup to route to that server

Document Session Lifecycle

1. User opens doc → authenticate and authorize → WebSocket Gateway routes to assigned Collaboration Server
2. If doc not in memory → Load latest snapshot from PostgreSQL
                        → Replay ops from Cassandra since snapshot
                        → Build in-memory state
3. User types → Client generates op → Sends via WebSocket
4. Server transforms or merges → assigns seq_num → durably persists to the operation log → acknowledges → broadcasts
5. Periodically (every 100 ops or 30 sec) → Write snapshot to PostgreSQL
6. Last user leaves → After 5 min idle, evict doc from memory

Component Deep Dives

OT vs CRDT: Merge Semantics

Start with this architectural fork because it determines how concurrent operations are ordered, merged, and synchronized offline. State the choice for rich text and explain the implications for offline support.

OT transforms concurrent operations against a canonical server order. It usually relies on a central collaboration server and requires rebasing for offline edits. CRDT operations carry identifiers scoped to a replica and causal metadata. Its merge rules support offline work without requiring a single ordering authority, although metadata and garbage collection add storage complexity.

Collaboration Server Memory Model

Per-document server memory and eviction policy matter at scale. Explain snapshot before eviction so a cold start can rebuild the document without replaying the full operation history.

In-memory: document tree + op log tail (last N ops for fast catch-up)
On cold start: PostgreSQL snapshot + replay Cassandra ops since snapshot_index
Hot doc (200 editors): batch ops every 50ms, and send presence updates every 100ms
Eviction: LRU after 5 min idle, taking a snapshot before evict

Cursor & Presence Sync

Cursor positions are ephemeral presence updates rather than document operations, so they are not persisted to the operation log. Throttle to 10 updates/sec per user. Use CRDT-style position identifiers rather than raw offsets so cursors stay valid after concurrent inserts.

Permissions on the Real-Time Channel

Authenticate the user before opening the WebSocket and authorize the requested document role. Cache permissions in Redis for the stated 5 minute TTL, but invalidate the cache when sharing permissions change. The collaboration server checks editor permission before accepting a mutation. Suggestion mode stores proposed changes separately until an authorized editor accepts or rejects them.

Offline to Reconnect Merge

The client buffers operations in IndexedDB while offline. On reconnect, it sends buffered operations with client_seq and client_id. For OT, the server rebases those operations against the current document version and returns transformed operations. For CRDT, the peers exchange missing operations using causal metadata and merge them deterministically. The selected merge algorithm resolves ordinary text conflicts automatically, so users do not need a manual merge dialog.

API Design

WebSocket Protocol (bidirectional)

Use stable domain identifiers in the application model. The wire example below keeps the operation payload compact and uses integer positions as an OT-style illustration. A CRDT implementation should carry stable position identifiers rather than byte offsets. The server returns the acknowledgement only after the accepted operation reaches the configured durable write quorum.

TYPESCRIPT
type DocId = string;
type ClientId = string;
type UserId = string;

type DocumentOperation =
  | { type: "insert"; pos: number; content: string; attrs?: Record<string, unknown> }
  | { type: "delete"; pos: number; len: number }
  | { type: "format"; pos: number; len: number; attrs: Record<string, unknown> };

interface OperationRequest {
  type: "operation";
  docId: DocId;
  clientId: ClientId;
  clientSeq: number;
  baseServerSeq?: number; // required for OT, omitted for CRDT
  ops: DocumentOperation[];
}

interface OperationAck {
  type: "ack";
  clientSeq: number;
  serverSeq: number;
}

interface CursorUpdate {
  type: "cursor";
  userId: UserId;
  positionId: string;
  selectionStartId?: string;
  selectionEndId?: string;
}
JSON
// Client → Server: operation based on the last observed server sequence
{
  "type": "operation",
  "doc_id": "d_abc123",
  "client_id": "c_789",
  "client_seq": 42,
  "base_server_seq": 100,
  "ops": [
    { "type": "insert", "pos": 15, "content": "Hello", "attrs": {"bold": true} },
    { "type": "delete", "pos": 10, "len": 3 },
    { "type": "format", "pos": 5, "len": 10, "attrs": {"italic": true} }
  ]
}

// Server → Client (acknowledgement)
{ "type": "ack", "client_seq": 42, "server_seq": 105 }

// Presence updates
{ "type": "cursor", "user_id": "u_xyz", "cursor": { "position_id": "p_42", "selection": { "start_id": "p_42", "end_id": "p_50" } } }

REST APIs

POST   /api/docs                         → Create new document
GET    /api/docs/{doc_id}                → Get document
DELETE /api/docs/{doc_id}                → Delete document (soft delete)
GET    /api/docs/{doc_id}/history        → Get version history
GET    /api/docs/{doc_id}/version/{v}    → Get doc at specific version
POST   /api/docs/{doc_id}/restore/{v}   → Restore to version v
POST   /api/docs/{doc_id}/share         → Update sharing permissions
POST   /api/docs/{doc_id}/export?fmt=pdf → Export document

Common Error Responses

400 Bad Request: invalid input, missing required fields, or malformed JSON payload
401 Unauthorized: missing or invalid authentication token or API key
403 Forbidden: authenticated caller lacks required permissions for this resource
404 Not Found: requested resource ID does not exist
409 Conflict: duplicate write or version conflict, retry with a unique idempotency key
422 Unprocessable Entity: syntactically valid request failed semantic business validation
429 Too Many Requests: rate limit quota exceeded, client should honor Retry-After header
500 Internal Error: unexpected server failure, retry safely with an idempotency key
503 Service Unavailable: downstream dependency is unavailable or overloaded, retry with exponential backoff

Data Model

Operation Log (Cassandra / ScyllaDB)

Table: operation_log
  Partition Key: (doc_id, seq_bucket)
  Clustering Key: seq_num (ASC)

  seq_bucket   BIGINT   -- floor(seq_num / 1,000,000)
  doc_id       TEXT
  seq_num      BIGINT
  user_id      TEXT
  op_type      TEXT  -- 'insert' | 'delete' | 'format'
  op_data      BLOB  -- Serialized operation (protobuf)
  timestamp    TIMESTAMP
  client_id    TEXT
  client_seq   INT

  # Idempotency key: (doc_id, client_id, client_seq)

Bucket the operation log by document and sequence range so a heavily edited document does not create an unbounded Cassandra partition. With 1M operations per bucket, a 1000 ops/sec hot document creates a new bucket roughly every 1000 seconds. Recovery starts at the snapshot sequence and reads only the required buckets.

Document Snapshots (PostgreSQL)

SQL
CREATE TABLE documents (
    doc_id UUID PRIMARY KEY,
    title TEXT,
    owner_id UUID,
    content_snapshot JSONB,
    snapshot_seq BIGINT,
    created_at TIMESTAMPTZ,
    updated_at TIMESTAMPTZ,
    is_deleted BOOLEAN DEFAULT FALSE
);

Redis: Active Sessions

# Active document sessions
HSET doc:session:{doc_id} client:{client_id} '{"user_id":"u_xyz","cursor":"p_42","connected_at":"..."}'

# Permission cache (TTL 5 min)
HSET doc:perms:{doc_id} user:{user_id} "editor"

# Cached copy of the ZooKeeper document assignment lease
# Gateway refreshes this TTL while the document remains active
SET doc:server:{doc_id} "collab-server-17" EX 300

Fault Tolerance

ConcernSolution
Collaboration server failureFailover uses ZooKeeper. The new server loads the latest snapshot, replays the operation log, and clients reconnect with exponential backoff
Conflict resolutionCorrectly implemented OT or CRDT provides convergence, while the durable operation log prevents loss of acknowledged operations
Offline editsEdits are buffered in IndexedDB and merged through the selected OT or CRDT algorithm on reconnect
Data durabilityCassandra with RF=3 stores acknowledged operations with quorum durability, and periodic snapshots are written to PostgreSQL for faster recovery
Hot document overloadBatch presence updates and operations, then switch to view-only mode above 200 editors
Duplicate operation deliveryThe idempotency key combines doc_id, client_id, and client_seq. Server-side deduplication prevents a retried operation from being applied twice

Collaboration Server Failure

1. WebSocket Gateway detects broken connection (heartbeat)
2. Gateway triggers failover: assigns doc to another collab server via ZooKeeper
3. New server loads latest snapshot from PostgreSQL
4. Replays all durable operations from Cassandra where seq_num > snapshot_seq
5. Clients reconnect (WebSocket auto-reconnect with exponential backoff)
6. Clients send their pending ops (buffered locally)
7. New server transforms and applies pending client ops

Recovery time: < 5 seconds

Conflict Resolution (OT Deep Dive)

Scenario: User A inserts "X" at position 5, User B deletes at position 3.

With OT:
  Server receives A's op first → applies insert("X", 5) → seq 101
  Server receives B's op (based on seq 100) → must transform:
    B's delete(3) is transformed against A's insert(5)
    Since 3 < 5: delete(3) stays at 3
    BUT A's insert(5) vs B's delete(3): since 5 > 3, adjust to insert(4)
  
  Both clients converge to same state ✓

CRDT Deep Dive

Using RGA (Replicated Growable Array):
  Each character = (uniqueID, value, isDeleted, parentID)
  UniqueID = (siteID, lamportClock), which is unique per replica and can be deterministically ordered

Two inserts same parent: order by uniqueID (siteID tiebreak)
Result: deterministic on ALL replicas with no single ordering server needed

Tombstone GC:
  Deleted chars kept as tombstones
  Periodic compaction: if all replicas have seen delete, remove tombstone

Additional Considerations

Collaborative Undo/Redo

Maintain a per-user undo stack that records that user's intent rather than a copy of the entire document. Undo creates a new inverse operation against the current document state, so it does not erase edits made by other users. Permission checks still apply to the generated operation.

1. User A inserts "X" → append operation to A's undo stack
2. User B edits the same document → document state advances
3. User A presses Undo → create an inverse operation for A's insert
4. OT: transform the inverse against intervening operations before applying
5. CRDT: emit a new operation that reverts A's prior mutation while preserving later edits from others
6. Persist the undo operation in the normal operation log so version history remains durable
7. Redo emits another forward operation. Never restore an old snapshot wholesale

Offline Editing & Sync

1. User goes offline → all edits stored in IndexedDB/SQLite
2. Local CRDT state diverges from the server
3. User comes back online:
   a. Client sends all buffered ops to the server
   b. Server sends missing operations using the client's last seen sequence or causal state
   c. CRDT merge: both sides apply all missing operations, ensuring convergence under the CRDT's merge assumptions
   d. No manual conflict resolution is needed for ordinary text edits

Version History & Snapshotting

  • Snapshot every 100 ops or every 30 seconds
  • Operation log kept for 90 days for the stated compliance requirement. The 500 TB capacity estimate covers 30 days of version history
  • To view version at time T: find latest snapshot before T, replay ops to T
  • Delta compression between adjacent snapshots (~90% reduction)

Real-Time Presence at Scale

Problem: 200 users in a doc, each sending cursor updates at 10/sec
Raw inbound updates: 200 × 10 = 2K cursor updates/sec
Naive fanout: 2K × 199 other editors ≈ 398K outbound deliveries/sec
Solution:
- Cursor throttling (client-side): limit each user to one update per 100ms
- Cursor batching (server-side): aggregate all 200 positions, broadcast every 100ms
- Batched fanout: 10 batches/sec × 199 recipients = 1,990 outbound batch messages/sec
- Message count reduction: roughly 200x compared with naive per-update fanout

Interview Walkthrough

  • 25-minute cut

    Skip arch50/arch75 depth unless staff.

    • One collaboration server per active document (6 min)
    • WebSocket for operations, with OT or CRDT for merging (7 min)
    • Local keystroke instant. Remote op < 200ms (6 min)
    • Version history via operation log compaction (6 min)
  • Lead with the OT vs CRDT decision as the defining architectural fork. Explain the trade-off using offline requirements, metadata cost, and coordination complexity.
  • Explain how concurrent edits converge: OT transforms ops against each other on a central server, CRDT merges ops commutatively without coordination.
  • Use WebSocket for real-time operation delivery. Batch cursor and presence updates to avoid roughly 398K outbound cursor deliveries per second when 200 users each send updates at 10/sec.
  • Persist an append-only op log plus periodic snapshots, replaying ops from the latest snapshot for version history and crash recovery, integrating patterns from Event Sourcing and CQRS.
  • Address offline editing by buffering operations locally and synchronizing on reconnect. Use CRDT causal merge when CRDT is selected, or OT rebasing when OT is selected.
  • Frame consistency using CAP Theorem and Consistency Models. Explain that the design remains available during partial failures while convergence and durable writes are preserved under the stated failure model.
  • Mention tombstone compaction for CRDT storage growth (~30% overhead after heavy editing).
  • Common pitfall: last-write-wins timestamps, where two users typing simultaneously will silently lose one user's edits.

Engineering Trade-offs

OT vs CRDT: The Core Collaborative Editing Decision

Collaborative editing balances convergence guarantees and offline support. OT vs CRDT is the defining trade-off.

Operational Transformation (OT):
  ✓ Compact representation (small operations)
  ✓ Works well for text (insert/delete are natural)
  ✗ A central server is required for canonical ordering
  ✗ Correctness is notoriously difficult to prove
  ✗ Complex to implement correctly for rich text

CRDT:
  ✓ Convergence follows from the CRDT data type and its merge assumptions
  ✓ Can merge without a single central ordering server. Production systems may still use servers for relay and persistence
  ✓ Easier to reason about correctness
  ✗ Higher storage overhead (each character has metadata)
  ✗ Tombstones accumulate: ~30% of storage after heavy editing

Design choice: explain both approaches, then choose based on offline requirements, metadata cost, and operational complexity.

Document Snapshot Strategy: Frequency vs Storage Cost

Alternative policy example:
Snapshot every 1000 ops:
  Recovery: at most 999 ops to replay → acceptable (< 1 sec)
  Storage: manageable

Version history checkpoints:
  - Every 1000 ops
  - On explicit "Version History" save (user-triggered)
  - On document sharing (ensure shareable state is snapshotted)

Delta compression between snapshots:
  60-90% storage reduction for documents with small incremental changes

💬Review

Help Us Improve

How helpful was this walkthrough?

Click a star to rate. We actively use this feedback to refine and update our system design content.

Placeholder
Optional but highly appreciated!

Discussion

Share your thoughts, ask questions, or help others.

Loading comments...