Cache Stampede Prevention

Understanding Cache Stampede

A cache stampede occurs when many requests simultaneously detect a missing or expired cache entry and attempt to regenerate it. This can overwhelm backend resources, causing latency spikes or outages.

Techniques for Prevention

• Use locking (mutex/semaphore): Allow only one request to rebuild the cache entry; other requests wait or use stale data;
• Apply request coalescing: Aggregate concurrent requests for the same key, serving all with a single backend call;
• Implement stale-while-revalidate: Serve expired (stale) cache data while a background process refreshes the cache;
• Use randomized expiration (jitter): Stagger cache expiration times to avoid synchronized cache misses.

Impact on Performance and Consistency

• Reduces backend load during cache misses, improving system stability and throughput;
• May temporarily serve stale data, which can slightly reduce consistency but ensures high availability and responsiveness;
• Ensures only one backend call per cache miss, minimizing redundant work and response time spikes.

By adopting these patterns, you can maintain reliable, performant systems even under heavy or unpredictable traffic.

Minimal Example: Preventing a Cache Stampede

Imagine a popular ticketing website where thousands of users check event availability. When the cache for the event list expires, every user's request could hit the backend database at once, causing a sudden spike in load. This is known as a cache stampede.

Cache stampede prevention works like a traffic cop at a busy intersection:

• When the cache expires, only the first user request is allowed to fetch fresh data from the backend;
• Other requests wait or receive stale data until the new data is cached;
• This prevents all requests from hitting the backend at the same time.

By controlling how and when cache refreshes occur, you keep your backend safe from sudden overload and ensure a smooth experience for your users.