4a755255f 566565 54c55a563 54f56055e 55f’564 56455a566558 565560563 562("")

“A thundering herd hits when 100k users request a missing cache key at once. Without protection, this triggers as many simultaneous database queries, risking total system downtime.”

Why This Matters

Caches assume atomic misses, but in practice millions of concurrent requests can bypass it at once. A single expired key during a flash sale can trigger hundreds of thousands of database queries in seconds, collapsing latency and cascading to full system failure.

Key Insights

• “Distributed locks: Only the first request holding the lock hits the DB, preventing concurrent stampedes.”
• “Jitter: Adding randomness to retry intervals spreads load and avoids synchronized replays.”
• “Request coalescing: Merges thousands of identical requests into one database call (e.g., used by API gateways).”
• “Early re-caching: Background processes refresh cache before TTL expires, reducing miss windows.”

Working Examples

Naive cache-aside pattern that causes thundering herd on cache miss.

let data = await cache.get(key);

if (!data) {
  data = await db.query(query); // The Bottleneck!
  await cache.set(key, data);
}

return data;

Practical Applications

References:

• From internal analysis