Design a P2P File Transfer System (like BitTorrent)

Level Expected:

Interview Setup

Interview Prompt

Design a BitTorrent-style P2P file transfer system distributing a 4 GB file across a 10,000-peer swarm with 2 MB pieces, achieving download in ~5 minutes via rarest-first piece selection.

Clarifying Questions (ask before designing)

Question	Why it matters
Centralized tracker or fully decentralized DHT?	A centralized tracker is simpler but introduces a single point of failure, whereas DHT scales infinitely at the expense of slight discovery latency.
How to handle free-riders who download but never upload?	Tit-for-tat choking starves uncooperative leechers while optimistic unchoking discovers capable new contributors.
NAT traversal strategy for peers behind firewalls?	Approximately 60% of consumer peers operate behind NAT, necessitating STUN hole-punching with TURN relay fallback.
Piece size trade-off: 256 KB vs 2 MB vs 4 MB?	A 4 GB file divided into 2 MB chunks yields 2,000 pieces, balancing rarest-first scheduling granularity against metadata footprint.

Scope

In scope

Piece selection (rarest first)
Peer discovery (DHT/tracker)
Tit-for-tat incentive
NAT traversal (STUN/TURN)
Swarm management
Capacity estimation with shown math

Out of scope (state explicitly)

Detailed frontend/UI pixel implementation
Org structure, staffing, and hiring plan

Functional Requirements

Start by clarifying whether tracker-based BitTorrent, trackerless DHT, or both are in scope, because peer discovery represents the first architectural fork.

Distribute massive files (100 MB to 100 GB) across thousands of distributed peers without central bandwidth bottlenecks
Fixed-size piece decomposition: Partition files into 256 KB to 4 MB chunks that can be downloaded independently and out of order
Peer discovery: Locate active swarm members possessing needed pieces using centralized trackers or decentralized DHT
Rarest-first piece scheduling: Prioritize downloading pieces held by the fewest peers to maximize replication
Tit-for-tat fairness: Incentivize contribution by preferentially uploading to peers who reciprocate with download bandwidth
Torrent metadata and magnet links: Encapsulate file metadata, SHA-1 piece hashes, and tracker discovery endpoints
Kademlia DHT: Provide trackerless peer discovery using XOR distance routing
Cryptographic piece validation: Verify each completed chunk against its SHA-1 hash before advertising availability
Resumable transfers: Support resumption of interrupted downloads and enable immediate seeding of completed pieces

Non-Functional Requirements

The counterintuitive architectural quality of peer-to-peer networks is scalability through peers, where more downloaders generate more aggregate upload bandwidth. Ensure you address free-riding defenses and cryptographic integrity verification.

Inverse Scalability: Swarm capacity grows proportionally with downloader count, reversing traditional client-server bottlenecks
High Fault Tolerance: Any participating peer can disconnect without degrading file availability across the remaining swarm
True Decentralization: Eliminate single points of failure by supporting trackerless operation via Kademlia DHT
Strict Data Integrity: Detect and discard corrupted or tampered blocks before writing them to disk
Economic Fairness: Enforce game-theoretic tit-for-tat rules to prevent free-riding leechers from exhausting capacity

Capacity Estimations

Contrast single-client download durations against aggregate swarm throughput, because interviewers want to see that you understand why P2P outperforms client-server architectures only when the swarm is healthy.

Metric	Calculation	Value
File size	Given (assumption documented in value)	4 GB (typical movie)
Piece size	Given (assumption documented in value)	2 MB resulting in 2,000 pieces
Peers in swarm	Given (assumption documented in value)	10,000
Seeders (have complete file)	Given (assumption documented in value)	2,000
Leechers (downloading)	Given (assumption documented in value)	8,000
Per-peer upload capacity	Given (assumption documented in value)	1 Mbps avg
Total swarm upload capacity	10,000 x 1 Mbps	10 Gbps
Download time (single peer, 10 Mbps)	4 GB / 10 Mbps	~53 min
Download time (full swarm)	4 GB / 10 Gbps aggregate upload	~5 min at equilibrium

Reconciling download times: A single leecher downloading from a lone 10 Mbps source requires approximately 53 minutes to transfer a 4 GB file. In a healthy swarm of 10,000 peers employing rarest-first piece scheduling, aggregate swarm upload reaches approximately 10 Gbps, allowing effective downloads to complete in roughly 5 minutes at equilibrium. The 53-minute metric represents the worst-case solo leecher, whereas the 5-minute target demonstrates the exponential scaling of collaborative upload swarms.

Architecture Diagram

In the interview room, clarify tracker vs DHT discovery as the initial fork, making NAT hole punching critical when explaining how peers behind firewalls connect.

The architecture consists of three principal subsystems: discovery services (centralized trackers and Kademlia DHT nodes), the peer wire protocol governing bi-directional TCP data transfer, and the local choking state machine that prevents free-riding. NAT traversal mechanisms such as STUN and TURN ensure peers behind consumer firewalls can establish direct connections.

For foundational context on how peer coordination replaces centralized file servers, review Client-Server vs Peer-to-Peer Architectures.

Component Deep Dives

This section explores the core algorithms underpinning BitTorrent: rarest-first piece scheduling to prevent swarm starvation, tit-for-tat unchoking to incentivize uploading, and Kademlia DHT routing for decentralized discovery.

Piece Selection: Rarest-First Algorithm

Rarest-first ensures that poorly replicated chunks receive immediate download priority, protecting the swarm against seeder attrition.

Problem: If every peer downloads piece 1 first, piece 1 becomes abundant
while rare pieces risk permanent unavailability if the originating seeder departs.

Rarest-first strategy:
  1. Track which pieces each connected peer advertises via bitfield and have messages.
  2. Compute the aggregate availability count for each piece across all active peer connections.
  3. Prioritize downloading the piece with the lowest global availability first.
  4. Bootstrap exception: the first few pieces are selected uniformly at random to acquire uploadable data quickly.

Concrete priority example:
  Piece 1: Available across 50 peers -> Low priority
  Piece 42: Available from only 2 peers -> High priority (download immediately)
  Piece 99: Available from only 1 peer -> Critical priority
  
  If the single peer serving piece 99 disconnects, that piece is lost and the swarm cannot complete.

Tit-for-Tat (Choking Algorithm)

Game-theoretic unchoking rewards nodes that contribute upload bandwidth, reaching a cooperative Nash equilibrium across the swarm.

Problem: Free-riding leechers who consume swarm download bandwidth without uploading degrade performance.

Solution: Tit-for-tat reciprocating choking algorithm
  1. Every 10 seconds, unchoke the top 4 peers ranked by their upload rate to this node.
  2. Every 30 seconds, perform optimistic unchoking: select 1 random peer to evaluate its upload capacity.
  3. All other connected peers remain choked, meaning this node will not upload blocks to them.

Algorithmic outcome:
  - High-bandwidth uploaders receive reciprocal high-bandwidth downloads.
  - Non-contributing or throttled peers experience severely restricted download speeds.
  - Newly joined peers bootstrap successfully through the optimistic unchoke rotation.
  - Reaches a Nash equilibrium where mutual uploading maximizes aggregate swarm health.

Kademlia DHT Lookup

Kademlia employs an XOR metric to organize 160-bit node identifiers into binary routing trees, achieving O(log N) lookup complexity.

Kademlia XOR Routing Example:
  Local Node A ID: 01101001
  Target Info Hash: 01100011
  Initial XOR Distance: 01101001 XOR 01100011 = 00001010 (decimal 10)

Iteration 1:
  Select alpha=3 closest known routing contacts to target info hash.
  Dispatch parallel find_node(01100011) RPC queries over UDP.

Iteration 2:
  Nearest responding peer returns its own closest known contacts:
    [01100001, 01100010, 01100100]
  Distance evaluation: 01100010 XOR 01100011 = 00000001 (distance = 1, closer match).
  Dispatch follow-up find_node queries to these refined contacts.

Iteration 3:
  Node 01100010 responds directly with target contact [01100011].
  Issue get_peers(info_hash) to retrieve the active swarm peer IP list.

Theoretical hops: O(log2 N), yielding at most ~23 hops across 10M nodes.
Practical traversal: Resolves in 4 to 8 hops in real-world swarms.

API Design

BitTorrent Wire Protocol

BitTorrent does not rely on a REST API because the wire protocol itself acts as the interface. The protocol runs over persistent TCP sockets using compact binary framed messages.

Peer handshake over TCP:
  <pstrlen=19><pstr="BitTorrent protocol"><reserved=8 bytes>
  <info_hash=20 bytes><peer_id=20 bytes>

Core wire protocol messages:
  choke          -> Notifies peer that this node will not fulfill data requests.
  unchoke        -> Notifies peer that data transfers are now permitted.
  interested     -> Expresses desire for pieces advertised in peer's bitfield.
  not interested -> Declares that the peer possesses no needed pieces.
  have           -> Broadcasts completion of a specific piece index.
  bitfield       -> Transmits initial bitmap of all completed pieces.
  request        -> Solicits a 16 KB sub-piece block (piece index, byte offset, length).
  piece          -> Delivers raw binary payload corresponding to a requested block.

Common Error Responses

JSON

400 Bad Request: invalid input, missing required fields, or malformed JSON payload
401 Unauthorized: missing or invalid authentication token or API key
403 Forbidden: authenticated caller lacks required permissions for this resource
404 Not Found: requested resource ID does not exist
409 Conflict: duplicate write or version conflict, retry with a unique idempotency key
422 Unprocessable Entity: syntactically valid request failed semantic business validation
429 Too Many Requests: rate limit quota exceeded, client should honor Retry-After header
500 Internal Error: unexpected server failure, retry safely with an idempotency key
503 Service Unavailable: downstream dependency is unavailable or overloaded, retry with exponential backoff

Data Model

.torrent File (Bencoded)

The .torrent metadata file contains tracker announce URLs and a concatenated list of SHA-1 hashes covering every fixed-size piece.

JSON

{
  "announce": "http://tracker.example.com/announce",
  "info": {
    "name": "movie.mkv",
    "piece length": 2097152,
    "pieces": "<concatenated SHA-1 hashes of all pieces>",
    "length": 4294967296
  }
}

info_hash = SHA-1(bencoded info dict) yielding a 20-byte swarm identifier

Kademlia DHT Routing Table

Each node generates a 160-bit random identifier. The routing table maintains 160 k-buckets, where each bucket retains up to k=8 contact records for nodes residing within specific XOR distance intervals.

Fault Tolerance

Endgame Mode

When fewer than 5% of pieces remain incomplete, slow peer links can delay transfer completion. The client broadcasts requests for all outstanding blocks to all unchoked peers in parallel, accepting the first valid response and cancelling outstanding duplicates.

Peer Churn and Swarm Maintenance

Peers continuously join and depart. Clients re-announce to trackers every 30 minutes and republish to the DHT every 15 minutes, while maintaining a pool of 20 to 50 active TCP connections and replacing disconnected sockets dynamically.

Piece Corruption and Anti-Poisoning

Every downloaded piece is verified against its authoritative SHA-1 hash. If a checksum mismatch occurs, the block is discarded and re-requested from an alternate peer, while peers sending corrupt blocks repeatedly are blacklisted.

Additional Considerations

Why P2P Outperforms Client-Server for Large Files

In client-server architectures, server bandwidth is the bottleneck and infrastructure cost scales linearly with downloaders. In P2P architectures, additional downloaders contribute aggregate upload bandwidth, accelerating transfer speeds for the entire swarm. For example, 10,000 peers uploading at 1 Mbps yield 10 Gbps of collective transfer capacity.

Magnet Links and Trackerless Torrents

Magnet URIs encapsulate only the 20-byte cryptographic info hash. Clients bootstrap discovery through DHT nodes and fetch piece metadata directly from connected swarm peers via extension protocols, rendering centralized .torrent hosting unnecessary.

WebTorrent: Browser-Based P2P

WebTorrent facilitates peer-to-peer data exchange directly inside web browsers using WebRTC data channels. WebRTC signaling executes over WebSocket trackers, enabling streaming video distribution directly between browser tabs to minimize CDN egress bills.

Interview Walkthrough

25-minute cut
Focus on rarest-first scheduling, tit-for-tat incentives, and Kademlia DHT routing.
- 2 MB pieces with SHA-1 hash verification per chunk (8 min)
- Rarest-first piece scheduling across active peer connections (9 min)
- Tracker and Kademlia DHT for robust peer discovery (8 min)
Frame the design around content-addressed chunking: the 20-byte info hash identifies the swarm, embedding integrity verification and deduplication directly into the protocol.
Detail how trackers and DHT provide peer discovery, with clients maintaining 20 to 50 connections and replacing churned peers continuously.
Explain rarest-first piece selection: prioritizing the pieces held by the fewest peers accelerates overall swarm completion and prevents missing tail chunks.
Walk through tit-for-tat choking: upload bandwidth is allocated to peers who upload to you, while optimistic unchoking periodically probes new peers.
Describe endgame mode when less than 5% of data remains, requesting missing blocks from all peers in parallel to prevent tail latency stalls.
Explain cryptographic verification: every piece is validated via SHA-1 upon receipt, corrupt chunks are discarded, and offending peers are blacklisted.
Highlight the common pitfall of downloading pieces in sequential order, which ignores the rarest-first strategy and starves the swarm when popular pieces saturate.

Engineering Trade-offs

P2P systems trade decentralization against discoverability, contrasting tracker simplicity against DHT complexity.

Choking and Unchoking Algorithm: State Machine

The peer wire state machine governs upload throttling and reciprocal bandwidth allocation across four distinct connection states.

Connection state machine per peer connection tracks 4 flags:
  am_choking:      This node is currently choking the remote peer (not uploading)
  am_interested:   This node is interested in pieces held by the remote peer
  peer_choking:    The remote peer is choking this node (not uploading to us)
  peer_interested: The remote peer is interested in pieces held by this node

Evaluation cycles:
  1. Connection established -> Bitfield exchanged -> Evaluate every 10 seconds.
  2. Rank all interested peers by incoming upload speed to us -> Unchoke top 4 bidders.
  3. Every 30 seconds: Optimistically unchoke 1 random peer to discover prospective high-speed seeders.

Seeder operational policy:
  Seeders possess all pieces and do not receive uploaded data.
  Instead, seeders rank interested peers by their download speeds to maximize distribution rate.

Routing Table Maintenance in Kademlia

When a message arrives from node X, the local node calculates XOR distance and checks the corresponding k-bucket. If the bucket is full, the node pings the least-recently-seen entry. If that contact responds, it is retained because empirical studies show long-lived nodes remain online significantly longer, and node X is discarded. Buckets refresh periodically every 15 minutes.

Why Piece Size Matters

Selecting piece chunk sizes represents a delicate trade-off between metadata overhead, disk I/O efficiency, and rarest-first scheduling flexibility.

Small piece size (256 KB):
  - Finer granularity enables superior rarest-first distribution across early swarm stages.
  - Faster initial piece completion unlocks earlier reciprocal upload capability.
  - Minimal bandwidth wasted if a single piece fails SHA-1 checksum verification.
  - Cost: larger .torrent metadata files and increased disk seek overhead on spinning media.

Large piece size (4 MB):
  - Compact .torrent metadata file with minimal protocol header overhead.
  - Superior sequential disk I/O performance on modern NVMe and storage arrays.
  - Cost: slower initial piece completion delays entering the tit-for-tat upload pool.
  - Substantial bandwidth penalty if a 4 MB payload fails SHA-1 hash validation.

Industry standard: 256 KB to 2 MB, dynamically selected based on overall file size.

Help Us Improve

How helpful was this walkthrough?

Click a star to rate. We actively use this feedback to refine and update our system design content.

Placeholder

Discussion

Share your thoughts, ask questions, or help others.

Loading comments...

◉Interview Setup