Interview Prompt
Design Design Pastebin.
Clarifying Questions (ask before designing)
| Question | Why it matters |
|---|---|
| Which of these is highest priority: Simpler variant of URL Shortener, Blob storage, Access control. Good warm-up.? | Forces scope negotiation — senior candidates trim before drawing boxes. |
| What scale should we design for — DAU, QPS, data volume? | Drives every capacity decision; shows structured thinking. |
| What are the read vs write patterns on the critical path? | Determines caching, DB choice, and replication topology. |
| What consistency and durability guarantees are required? | Separates strong-consistency paths from eventual ones — a senior differentiator. |
Scope
In scope
- Simpler variant of URL Shortener
- Blob storage
- Access control. Good warm-up.
- Capacity estimation with shown math
Out of scope (state explicitly)
- Client desktop/mobile app implementation
- End-user file preview rendering for every format
- Building raw block storage hardware
Assumptions
- Clarify scale (DAU, QPS, data volume) for pastebin in the first 5 minutes
- Standard reliability target 99.9%–99.99% unless problem implies higher (payments, booking)
- Managed cloud services (RDS, S3, Kafka, Redis) are acceptable building blocks
These foundational concepts underpin the patterns used in this problem. Review them before deep-diving into component-level trade-offs.
- Create paste: Submit text content, get a unique short URL
- Read paste: Anyone with the URL can view the paste
- Expiration: Configurable duration (10 min, 1 hour, 1 day, 1 week, never)
- Syntax highlighting: Support multiple programming languages
- Private pastes: Only accessible via exact URL
- User accounts: Registered users can manage their pastes
- Raw view: Plain text view without formatting
- Paste size limit: Max 10 MB per paste
- High Availability: 99.99%: reads are critical
- Low Latency: Paste loads in < 100 ms
- Read-Heavy: Read:Write ratio ~ 5:1
- Scalability: Support millions of pastes, 10K+ reads/sec
- Durability: Pastes must not be lost before their expiry
- Unique URLs: No collisions on paste identifiers
| Metric | Calculation | Value |
|---|---|---|
| New pastes / day | Given | 1M |
| Reads / day | Given | 5M |
| Writes / sec | Derived from daily volume ÷ 86400 (+ peak factor) | ~12 |
| Reads / sec | Derived from daily volume ÷ 86400 (+ peak factor) | ~58 (peak 300) |
| Avg paste size | Given | 10 KB |
| Storage / day | 1M × 10 KB | 10 GB |
| Storage / year | Given | 3.6 TB |
| Metadata per paste | Given | 500 bytes |
CDN (CloudFront / Fastly)
Paste reads are globally distributed and content is immutable once created. Cache rendered paste pages at edge with cache key = paste URL path. TTL = min(expiry_remaining, 24h) for expiring pastes, 7 days for never-expiring.
API Gateway
Rate limiting (token bucket per IP): anonymous 10 pastes/hour, authenticated 100/hour. CAPTCHA for anonymous writes. SSL termination at gateway.
Key Generation Service (KGS)
Pre-generate all possible 8-character Base62 keys (62^8 ~ 218 trillion). Store in key pool DB with unused_keys and used_keys tables. On paste creation: atomically pop a key. No collision checking, very fast, non-sequential.
Write Service
Receive paste content + expiration + language. Get unique key from KGS. Store content in S3: s3://pastes/{key_prefix}/{key}. Store metadata in MySQL. Return URL.
Read Service
Check Redis cache → if hit, return. If miss → check MySQL for metadata. If valid → fetch content from S3. Populate Redis cache (TTL = min(paste expiry, 1 hour)). Return with syntax highlighting.
Content Storage: S3 vs Database
Paste content can be up to 10 MB: too large for relational DB rows. S3 is optimized for object storage with 11-nines durability. DB stores only metadata (small, indexed, queryable). Compress with zstd before storing (3-10× savings).
Cleanup Worker
Runs every hour. Queries expired pastes, deletes content from S3, deletes metadata from MySQL, invalidates Redis cache.
Event Bus Design (Kafka)
Topic: pastebin-events Partitions: 64 (scale consumers horizontally) Partition key: entity_id (user_id / order_id — preserves per-entity ordering) Retention: 7 days (compliance) or 24h (high-volume telemetry) Replication factor: 3, min.insync.replicas: 2 Producer: idempotent producer enabled (enable.idempotence=true) Consumer: consumer group "pastebin-processors" - At-least-once delivery + idempotent handlers (dedup by event_id) - DLQ topic: pastebin-events-dlq (poison messages after 3 retries) - Lag alert: consumer lag > 60s → scale workers Design Pastebin: async side effects MUST NOT block the synchronous API response. Sync path: validate → persist source of truth → publish event → return 201 Async path: consumers update caches, indexes, notifications, aggregates
Create Paste
POST /api/v1/pastes
{
"content": "print('Hello, World!')",
"language": "python",
"expiry": "1d",
"is_private": false,
"title": "My first paste"
}
Response: 201 Created
{
"key": "abc12345",
"url": "https://pastebin.com/abc12345",
"raw_url": "https://pastebin.com/raw/abc12345",
"expires_at": "2026-03-14T10:00:00Z"
}Read Paste
GET /api/v1/pastes/{key}
Response: 200 OK
{
"key": "abc12345",
"content": "print('Hello, World!')",
"language": "python",
"title": "My first paste",
"created_at": "2026-03-13T10:00:00Z",
"expires_at": "2026-03-14T10:00:00Z",
"views": 42
}Delete Paste
DELETE /api/v1/pastes/{key}
Authorization: Bearer <token>
Response: 204 No ContentCommon Error Responses
400 Bad Request: invalid input, missing fields, or malformed JSON 401 Unauthorized: missing or invalid auth token or API key 403 Forbidden: authenticated but insufficient permissions 404 Not Found: resource ID does not exist 409 Conflict: duplicate write or version conflict; retry with idempotency key 422 Unprocessable Entity: valid syntax but invalid business logic 429 Too Many Requests: rate limit exceeded; honor Retry-After header 500 Internal Error: unexpected server fault; retry with idempotency key 503 Service Unavailable: dependency down or overloaded; use exponential backoff
MySQL: Paste Metadata
CREATE TABLE pastes (
paste_key CHAR(8) PRIMARY KEY,
user_id UUID,
title VARCHAR(256),
language VARCHAR(32) DEFAULT 'text',
content_size INT,
s3_path VARCHAR(256),
is_private BOOLEAN DEFAULT FALSE,
view_count INT DEFAULT 0,
expires_at TIMESTAMP,
created_at TIMESTAMP NOT NULL,
INDEX idx_user (user_id, created_at DESC),
INDEX idx_expiry (expires_at)
);S3: Paste Content
Bucket: pastebin-content
Key: pastes/{first_2_chars_of_key}/{paste_key}
Value: compressed text contentRedis Cache
Key: paste:{key}
Value: Hash { content, language, title, expires_at, view_count }
TTL: min(paste_expiry_remaining, 3600)KGS: Key Pool
CREATE TABLE unused_keys (key_value CHAR(8) PRIMARY KEY);
CREATE TABLE used_keys (key_value CHAR(8) PRIMARY KEY, used_at TIMESTAMP);| Concern | Solution |
|---|---|
| S3 unavailable | Multi-region S3 replication; serve from cache |
| MySQL failure | Read replicas + standby for failover |
| KGS failure | Each app server caches a batch of 1000 keys locally |
| Expired paste served | Redis TTL alignment + MySQL check on cache miss |
| Content corruption | Checksum stored in metadata; verified on read |
| Duplicate key | KGS guarantees uniqueness + DB primary key constraint |
Abuse Prevention
Rate limiting, CAPTCHA, content scanning (ML-based malware/phishing detection), size limits (10 MB per paste, 512 KB for anonymous).
Analytics
Track view count per paste (async via Kafka → batch update DB). Popular pastes dashboard, language distribution analytics.
Burn After Reading
Special paste type deleted after first read. After serving content, immediately delete from S3 + MySQL + Redis. Use Redis lock to ensure only one reader gets the content.
Interview Walkthrough
- Similar to URL shortener but emphasize content-addressable keys vs random IDs — clarify whether same content maps to same URL.
- Explain read-heavy ratio (views >> creates) and layer Caching Patterns on hot pastes at CDN + Redis.
- Cover expiration TTL and garbage collection of expired content from Blob Storage and metadata store.
- Discuss syntax highlighting as async post-processing, not blocking the create response.
- Mention content moderation scan (virus, spam) via async queue before making paste publicly indexable.
- Common pitfall: storing paste body in the database — large text belongs in Blob Storage with metadata row only.
Storage Architecture: MySQL + S3 vs Pure NoSQL
Option 1: MySQL (metadata) + S3 (content) ✓ - Paste content up to 10 MB — relational DB rows not designed for blobs - S3: 11-nines durability, cheap, CDN-friendly - MySQL: ACID transactions for metadata, indexed lookups, TTL via events Option 2: Cassandra for both ✓ No SPOF, auto-replication ✗ 2GB max cell size, no TTL for partial row data ✗ More complex than MySQL + S3 Option 3: DynamoDB + S3 ✓ Fully managed, auto-scaling ✗ AWS vendor lock-in, higher cost at scale Decision: MySQL + S3 is the right choice for Pastebin's scale.
Expiry Implementation
Option 1: MySQL Scheduled Events ✓ No external infra, transactional ✗ Doesn't clean S3, can't handle burst expirations Option 2: Cassandra TTL ✓ Zero application code ✗ Tombstones → read slowdown, doesn't clean S3 Option 3: Background Worker + S3 Lifecycle Policy ✓ Worker hourly cleans DB + S3 + Redis together S3 Lifecycle as safety net (6-month expiry) ✓ Transactional, handles batches gracefully ✗ Small delay: visible up to 1 hour past expiry
Read Path Optimization
Standard read path: Client → CDN (edge hit ~40%) → Redis (hit ~85%) → MySQL → S3 For anonymous pastes: CDN caches rendered page For burn-after-reading: never cached For private pastes: no public CDN caching Effective cache hit rates: CDN: ~40% of total requests Redis: ~85% of remaining MySQL + S3: ~9% of requests Result: < 10% reach MySQL/S3
Deduplication: Same Content, Same URL?
Decision: No dedup for Pastebin. Storage savings are negligible for text content. Dedup makes sense for binary content where cost matters.
Staff interviews expect you to articulate how the system evolves under real growth — not jump straight to the final architecture.
Phase 1: MVP (0 to 100K users)
Monolith or minimal services proving core pastebin flows. Optimize for shipping speed and correctness over scale.
Key components: Single region · Primary DB + Redis cache · Synchronous core path · Basic monitoring
Move to next phase when: p99 latency exceeds SLO or DB CPU sustained above 70%
Phase 2: Growth (100K to 10M users)
Split read/write paths, introduce async processing for non-critical work, add caching layers and horizontal scaling.
Key components: Read replicas or CQRS · Message queue for async work · CDN / edge caching · Service-level SLOs
Move to next phase when: Hot keys, fan-out bottlenecks, or ops toil from manual scaling
Phase 3: Scale (10M+ users)
Shard data plane, multi-region active-active or active-passive, formal DR runbooks, cost optimization.
Key components: Database sharding / partitioning · Multi-region replication · Auto-scaling + chaos testing · Dedicated platform/SRE ownership
Move to next phase when: Regional failure domain risk, compliance data residency, or linear cost growth unsustainable
SLOs & Error Budgets
| Metric | Target | Rationale |
|---|---|---|
| Core user-facing availability | 99.95% | Budget for planned maintenance + unplanned failures without user-visible outage. |
| p99 latency (critical path) | Problem-specific — state target early and tie to capacity math | Interview credibility comes from connecting SLO to architecture choices. |
| Error rate (5xx) | < 0.1% | Distinguishes transient blips from systemic failure requiring rollback. |
| Data durability | 99.999999999% (11 nines) for committed writes | Define which operations require fsync/quorum vs async replication. |
Incident Scenarios (2am reality)
| Scenario | How you detect | Mitigation |
|---|---|---|
| Primary database unavailable | Health check failures, connection pool exhaustion alerts, elevated 5xx | Failover to replica / promote standby; enable read-only degraded mode if writes impossible; queue writes if async path exists |
| Traffic spike (10× normal) | RPS anomaly alert, autoscaling lag, latency SLO burn rate | Rate limit non-critical endpoints; scale read path horizontally; pre-warm caches; shed load on expensive operations |
| Bad deploy causing elevated errors | Canary metric regression, error budget burn, deployment correlation | Automated rollback within 5 minutes; feature flag kill switch; maintain N-1 compatibility |
Cost Drivers (Staff lens)
- Egress bandwidth and CDN (often dominates media/data-heavy systems)
- Database storage + IOPS at scale (plan compaction, TTL, tiering)
- Compute for async pipelines (right-size workers, spot instances for batch)
- Managed service premiums vs operational headcount trade-off
Multi-Region & DR
Start single-region with cross-AZ redundancy. Add read replicas in secondary region for DR. Move to active-active only when latency SLO or data residency requires it — accept conflict resolution complexity explicitly.