Caching Strategies
> "There are only two hard things in Computer Science: cache invalidation and naming things."
> — Phil Karlton
Caching is one of the most impactful optimizations you can make. A database query might take 50ms. A cache hit? 1-5ms. That's a 10-50x improvement.
Why Cache?
The Cache-Aside Pattern
The most common caching pattern. The application manages both cache and database.
def get_user(user_id):
# 1. Check cache first
cached = cache.get(f"user:{user_id}")
if cached:
return cached # Cache hit!
# 2. Cache miss - query database
user = db.query("SELECT * FROM users WHERE id = ?", user_id)
# 3. Update cache for next time
cache.set(f"user:{user_id}", user, ttl=3600) # 1 hour TTL
return userWrite Patterns
Write-Through
Write to cache AND database simultaneously.
Pros
- Cache always has latest data
- Simple consistency model
Cons
- Higher write latency
- May cache rarely-read data
Write-Behind (Write-Back)
Write to cache immediately, asynchronously sync to database.
Write → Cache → (async) → DatabaseCache Invalidation Strategies
The hardest problem. When data changes in the DB, how do you update the cache?
| Strategy | How It Works | Trade-off |
|---|---|---|
| TTL (Time To Live) | Data expires after N seconds | Simple but may serve stale data |
| Explicit Invalidation | Delete cache key on DB update | Requires coordination |
| Refresh-Ahead | Proactively refresh before TTL | More complex, fewer misses |
CDN: Caching for Static Content
For static files (images, CSS, JS), use a Content Delivery Network.
CDNs have servers distributed globally. Users get content from the nearest server.
User in Tokyo → Server in Virginia → 200ms latency
With CDN:User in Tokyo → CDN edge in Tokyo → 20ms latency
Real-World: Meta's Memcached at Scale
Meta operates one of the world's largest Memcached deployments:
They moved GET operations to UDP to reduce network traffic, and improved cache consistency from 99.9999% (six nines) to 99.99999999% (ten nines).
Cache Eviction Policies
When cache is full, what gets removed?
LRU
Least Recently Used - evict what hasn't been accessed longest
LFU
Least Frequently Used - evict what's accessed least often
FIFO
First In First Out - evict oldest entries
Key Takeaways
- Cache-aside is the most common pattern: check cache → miss → DB → update cache
- TTL is the simplest invalidation strategy, but may serve stale data
- CDN for static content dramatically reduces latency for global users
- LRU is the most common eviction policy
Next up: Database Design & Scaling - SQL vs NoSQL, replication, and sharding.