Caching Patterns and Invalidation

Caching is the fastest lever in system design — and the easiest to get wrong. We add a cache when read QPS or latency demands it, then pick an invalidation strategy before drawing the Redis box.

Start with one rule: the cache is a copy, not the source of truth. If Redis disappears, you rebuild from the database. Everything else follows from how reads and writes interact with that copy.

Caching is the first lever on read-heavy designs — but only if we have an invalidation story. Three lenses:

Walk the cache stack top-down as interview steps. Step 1 — Browser/CDN: cache static and cacheable GETs at the edge. Step 2 — App check: read path hits Redis before the database. Step 3 — Miss path: load from DB, populate cache with TTL. Step 4 — Write path: update DB, then delete or update cache key — state cache-aside vs write-through. Step 5 — Stampede guard: mention singleflight or probabilistic early refresh for hot keys.

Core Caching Patterns Code Layout

Standard lazy-load Cache-Aside query and update paths executed in code:

Read:
  data = cache.get(key)
  if data is None:
      data = db.query(key)
      cache.set(key, data, ttl=300)
  return data

Write:
  db.update(key, data)
  cache.delete(key)  // Invalidate immediately to prevent stales

Pattern choice determines staleness vs write latency — we walk the table row by row:

• Lazy vs Aggressive Loading: Cache-Aside loads on demand; Write-Through preempts misses at the cost of write speeds.
• Read and write paths differ — cache-aside, write-through, and write-behind each trade staleness against write latency:

• Eviction policies — when memory fills, the cache must drop keys:

Cache-aside, write-through, and write-behind each optimize a different axis — we compare openly:

Cache-aside minimizes wasted memory but allows brief staleness; write-through maximizes consistency at the cost of write latency; write-behind maximizes write throughput but risks loss on crash.

Thundering herd, cache penetration, and avalanche are the classic interview probes — we lead with mitigations:

Invalidation strategy follows from how stale the product can tolerate — match strategy to use case:

Add a cache when read QPS dominates and hot keys follow a Zipfian distribution:

Caching connects to Bloom filters, CDN edge delivery, and Redis data structures:

• Bloom Filter: Space-efficient checks for non-existent cache keys.
• CDN & Edge Delivery: Caching static resources at the geographical network boundary.
• Redis Patterns: Specific in-memory data configurations and storage features.

The XFetch Probabilistic Cache Expiration Algorithm

To eliminate the Cache Stampede entirely without blocking thread pools, high-scale systems deploy **XFetch**. Instead of waiting for a key to expire to trigger recomputation, readers evaluate a probabilistic test during normal gets:

# Probabilistic early compute check
import random, math

def should_recompute(remaining_ttl, compute_time, beta=1.0):
    # beta > 1.0 makes early recomputation more aggressive
    return (remaining_ttl - (compute_time * beta * math.log(random.random()))) < 0

If the test evaluates to `True`, the reader returns the current cached value immediately, but asynchronously kicks off a background thread to refresh the cache key from the database.

Cache Consistency Race Conditions

Under high concurrency, the Cache-Aside pattern (write: update DB, delete cache) has a rare but catastrophic race condition:

1. Cache is empty. Client A queries key $K$; misses; queries DB and gets value $V1$.
2. Client B updates key $K$ in DB to $V2$; deletes cache key (currently empty anyway).
3. Client A (delayed by CPU scheduling) writes stale value $V1$ back into the cache.
4. The cache now stores stale value $V1$ indefinitely until the next TTL or write.

Mitigation: Enforce short TTL values as a safety net, or use transactional locking layers (e.g., MySQL locks or Redis locks) during cache recomputation.

Multi-Layer Cache Stack

Caching is not only "Redis in front of Postgres." Production systems stack layers — each with different TTL, invalidation cost, and hit ratio:

1. Client / browser cache: HTTP Cache-Control headers for static assets and idempotent GET responses — zero server load on repeat visits.
2. CDN edge: Geo-distributed cache for images, video segments, and public API GETs with short TTL — see CDN & Edge Delivery.
3. In-process (local) cache: Caffeine/Guava map inside each app pod — microsecond reads for config and hot keys; invalidated via pub/sub on change.
4. Distributed cache (Redis/Memcached): Shared across pods; survives pod restarts; handles hot keys with replication or key splitting.
5. Database buffer pool: OS/page cache and InnoDB buffer — last line before disk; not a substitute for application-level design.

Interview signal: walk the read path top-down — "CDN miss → Redis miss → read replica → primary" — and state where each layer adds value vs complexity.

Do not cache everything — profile read paths first and cache only keys with high read frequency and tolerable staleness. Caching data that changes every request adds latency without benefit.

Hot Key Mitigation

A viral post or celebrity profile concentrates reads on one cache key. A single Redis node serving that key becomes a bottleneck. Mitigations: replicate the hot key across Redis nodes with random read selection, local in-process caching on app servers for ultra-hot keys, or pre-warm CDN/edge caches before traffic spikes.