Caching: cache-aside, write-through, write-back, eviction, stampedes

A cache speeds up reads but must stay in sync with truth (your database). The strategy you pick—how the cache is written and invalidated—dictates correctness and performance.

Cache-Aside (Lazy Load)

How it works:

Read request:
  1. Check cache: cache hit? Return value. Done.
  2. Cache miss? Fetch from database, store in cache, return value.

Write request:
  1. Write to database.
  2. Invalidate cache (delete the key).

Pros:

• Simple to implement: cache is a passive lookup layer.
• No write overhead: writes don't touch cache, only reads populate it.
• Flexible: only frequently-read data gets cached (self-tuning).

Cons:

• Cache-miss latency: first read is slow (database latency).
• Stale reads possible: between write and invalidation, old cached value might be served.
• Complex invalidation: you must manually invalidate on writes.

Code sketch:

String getUserName(int userId) {
  String cacheKey = "user:" + userId;
  String cached = cache.get(cacheKey);
  if (cached != null) return cached;

  // Cache miss
  String fromDB = database.getUserName(userId);
  cache.set(cacheKey, fromDB, 3600); // 1 hour TTL
  return fromDB;
}

void updateUserName(int userId, String newName) {
  database.updateUserName(userId, newName);
  cache.delete("user:" + userId);
}

When to use: General-purpose caching, especially reads that outnumber writes.

---

Write-Through

How it works:

Write request:
  1. Write to cache.
  2. Write to database (synchronously).
  3. Return.

Pros:

• Cache always consistent with database (synchronous).
• No invalidation logic needed.

Cons:

• Write latency: every write waits for database (slow).
• Wasted cache space: infrequently-read items still occupy cache.
• Cascading failure: if database is down, all writes fail (no fallback).

Code sketch:

void updateUserName(int userId, String newName) {
  cache.set("user:" + userId, newName);
  database.updateUserName(userId, newName); // Wait for this
}

When to use: Small datasets, writes are critical and rare, database is reliable.

---

Write-Back (Write-Behind)

How it works:

Write request:
  1. Write to cache (immediately return).
  2. Queue write to database (asynchronously).
  3. Batch or flush writes in background.

Pros:

• Fast writes: no wait for database.
• Database batching: group writes, less I/O.

Cons:

• Durability risk: if cache crashes before flushing, data lost.
• Consistency lag: database is eventually consistent (stale for seconds).
• Complex recovery: if writes queue up, recovery is intricate.

Code sketch:

void updateUserName(int userId, String newName) {
  cache.set("user:" + userId, newName);
  writeQueue.enqueue(new WriteTask(userId, newName));
  return; // Don't wait
}

// Background thread:
void flushWrites() {
  while (true) {
    List<WriteTask> batch = writeQueue.drainBatch(1000);
    database.batchUpdate(batch);
  }
}

When to use: High write volume, acceptable data loss risk (logs, analytics), eventual consistency OK.

---

Eviction Policies

When cache is full, what gets removed?

LRU (Least Recently Used)

Remove item not accessed in longest time. Temporal locality: "if you accessed X recently, you'll access it again soon."

LFU (Least Frequently Used)

Remove item accessed least often. Works when access patterns are stable but misses temporal shifts.

FIFO

Remove oldest item. Simple but poor; ignores access patterns.

TTL (Time To Live)

Expire item after N seconds, regardless of access. Every cache uses this as a safety net.

---

Cache Stampede (Thundering Herd)

The problem:

Key "users:123" with 1-hour TTL cached in Redis.
At T=3600s, key expires.
100 concurrent requests arrive for users:123.
All 100 miss cache simultaneously.
All 100 query database.
Database gets 100x surge.

Fixes:

1. Probabilistic TTL

// Renew cache probabilistically before it expires
double renewProb = 1.0 / (timeToExpire + 1);
if (Math.random() < renewProb) {
  // Refresh cache early
  cache.set(key, fetchFromDB(), TTL);
}

2. Lock-based Refresh

String get(String key) {
  String value = cache.get(key);
  if (value != null) return value;

  // Cache miss or expired. Try to acquire lock.
  if (cache.tryLock(key + ":refresh", 5s)) {
    try {
      value = database.fetch(key);
      cache.set(key, value, TTL);
    } finally {
      cache.unlock(key + ":refresh");
    }
  } else {
    // Another thread is refreshing. Wait and retry.
    Thread.sleep(100);
    return get(key);
  }
  return value;
}

3. Xfetch / Probabilistic Early Recomputation

Set a "soft TTL" (expiration check) separate from hard TTL. If soft TTL hit, asynchronously refresh while still serving stale value.

---

Consistency Models

Write-through + read cache: Strong consistency. Cache always reflects database.

Cache-aside + invalidation: Eventual consistency. Between write and invalidate, stale data served.

Write-back: Weak consistency. Database lags cache by seconds.

---

In the interview

1. Align strategy to use case. "For user profiles (read-heavy), cache-aside with 1-hour TTL. For financial ledgers (consistency-critical), write-through."

2. Avoid stampede: "I'd use probabilistic early refresh or a lock to prevent 1,000 requests hitting the database simultaneously at TTL expiration."

3. Discuss invalidation: "On write, delete the cache key. If I want stronger consistency, I use write-through instead."

4. Trade-offs: "Cache-aside is simple but requires manual invalidation. Write-through is consistent but slower. Write-back is fast but risky."