A cache stampede (or thundering herd) is the failure mode where many concurrent requests for the same hot key all miss the cache at the same time, hit the origin simultaneously, and overwhelm it. Stampedes typically happen the instant a popular key expires, when a cache is cold-started, or when a downstream system briefly fails and recovers.
How it happens
Consider a homepage feed cached with…
An eviction policy is the rule a cache uses to decide which entry to remove when it reaches its memory limit and needs to make room for a new entry. Choice of policy directly affects hit rate and is one of the most consequential cache configuration decisions.
Common policies
Memcached is an open-source, high-performance, distributed in-memory key-value cache, originally designed to speed up dynamic web applications by reducing database load. It is one of the oldest and most widely deployed caching systems, often paired with relational databases for read-heavy workloads.
How it works
Memcached stores arbitrary string keys mapped to arbitrary byte values, with a…
A Content Delivery Network (CDN) is a distributed network of edge servers that cache and serve content from locations close to the user, instead of every request reaching the origin server. CDNs reduce latency, offload traffic from the origin, and improve resilience to traffic spikes and regional outages.
How it works
When a user requests an asset, DNS or anycast routing directs them to the…
Redis (Remote Dictionary Server) is an in-memory data store that holds its dataset in RAM for sub-millisecond access. Beyond simple key-value, Redis supports rich data structures including strings, hashes, lists, sets, sorted sets, streams, bitmaps, and HyperLogLog.
How it works
A Redis server accepts commands over a simple text protocol (RESP). Commands are executed single-threaded, which…
Deterministic TTL in caching systems create hidden synchronization points that eventually break under scale. This deep dive explains how mature systems design for expiration, not just performance.
A fast MongoDB system comes from modeling data around how your application reads and writes it. This guide breaks down how to structure documents, when to embed or reference, the patterns used in real production systems, and the indexing strategies that keep performance predictable as data grows.