Distributed Caching
🔥 0
⭐ 0
Lesson 5 of 6 20 min +225 XP

Redis vs Memcached

Two titans of the caching world. Both store data in memory. Both are blazing fast. But they're not the same.

At a Glance

Redis

  • Rich data structures (strings, lists, sets, hashes, sorted sets)
  • Persistence options (RDB, AOF)
  • Pub/Sub messaging
  • Lua scripting
  • Single-threaded (mostly)
  • Built-in replication

Memcached

  • Simple key-value (strings only)
  • No persistence
  • No pub/sub
  • No scripting
  • Multi-threaded
  • Simpler, more predictable

Performance Comparison

~100K
Redis ops/sec (single thread)
~200K
Memcached ops/sec (multi-threaded)
< 1ms
Both: typical latency
Performance Note

Memcached can be faster for simple GET/SET operations due to multi-threading. But Redis 6.0+ added multi-threaded I/O, narrowing the gap. For most applications, both are "fast enough."

Redis Data Structures

Redis's killer feature: rich data types beyond simple strings.

Strings

Basic key-value, but with atomic operations:

SET user:123:name "Alice"
GET user:123:name              # "Alice"

INCR page:views               # Atomic increment
SETEX session:abc 3600 "data" # Set with 1 hour TTL

Hashes

Store objects without serialization:

HSET user:123 name "Alice" email "alice@example.com" age 30

HGET user:123 name            # "Alice"
HGETALL user:123              # All fields
HINCRBY user:123 age 1        # Increment age atomically
Memory Efficiency

Hashes are more memory-efficient than storing each field as a separate key. One hash with 100 fields uses less memory than 100 individual keys.

Lists

Ordered collections, great for queues and feeds:

LPUSH notifications:user:123 "New message"
LPUSH notifications:user:123 "New follower"

LRANGE notifications:user:123 0 9  # Get latest 10
RPOP notifications:user:123         # Get oldest (FIFO queue)

Sets

Unordered unique elements:

SADD user:123:followers "user:456" "user:789"
SADD user:456:followers "user:123" "user:789"

SISMEMBER user:123:followers "user:456"  # Check membership
SINTER user:123:followers user:456:followers  # Common followers

Sorted Sets (ZSETs)

The power tool. Elements with scores, automatically sorted.

ZADD leaderboard 1500 "player:alice"
ZADD leaderboard 2000 "player:bob"
ZADD leaderboard 1800 "player:charlie"

ZREVRANGE leaderboard 0 2 WITHSCORES  # Top 3 with scores
# 1) "player:bob" 2) 2000
# 3) "player:charlie" 4) 1800
# 5) "player:alice" 6) 1500

ZRANK leaderboard "player:alice"  # Rank (0-indexed)
ZINCRBY leaderboard 500 "player:alice"  # Update score

Sorted Set Use Cases

  • Leaderboards - Score = points, member = player
  • Rate limiting - Score = timestamp, member = request ID
  • Priority queues - Score = priority, member = task
  • Time-series - Score = timestamp, member = event

Redis Persistence

Unlike Memcached, Redis can persist data to disk:

Method How It Works Trade-off
RDB (Snapshots) Point-in-time snapshots at intervals Faster recovery, potential data loss
AOF (Append-Only) Log every write operation More durable, slower recovery
RDB + AOF Both enabled Best durability, most resources 
# redis.conf
save 900 1       # Snapshot if 1 key changed in 15 min
save 300 10      # Snapshot if 10 keys changed in 5 min
save 60 10000    # Snapshot if 10000 keys changed in 1 min

appendonly yes   # Enable AOF
appendfsync everysec  # Sync to disk every second

Redis Pub/Sub

Built-in publish/subscribe messaging:

# Publisher
import redis
r = redis.Redis()
r.publish('notifications', 'New order received')

# Subscriber
pubsub = r.pubsub()
pubsub.subscribe('notifications')

for message in pubsub.listen():
    print(message['data'])
Pub/Sub Limitation

Redis Pub/Sub is "fire and forget." If a subscriber is disconnected when a message is published, it's lost. For durable messaging, use Redis Streams or a proper message queue.

When to Choose Redis

Choose Redis When:
  • You need data structures (lists, sets, sorted sets)
  • You need persistence
  • You need pub/sub or streams
  • You want atomic operations on complex data
  • You need Lua scripting
Choose Memcached When:
  • Simple key-value caching only
  • You need multi-threaded performance
  • Simpler operational model
  • Memory efficiency for small values

Redis Cluster

For horizontal scaling, Redis Cluster automatically shards data:

[Redis Cluster]
├── Node 1: Slots 0-5460
├── Node 2: Slots 5461-10922
└── Node 3: Slots 10923-16383

Key "user:123" → Hash → Slot 7892 → Node 2
from redis.cluster import RedisCluster

rc = RedisCluster(host='node1', port=6379)
rc.set('user:123', 'Alice')  # Automatically routes to correct node

Production Redis Configuration

# Essential settings
maxmemory 4gb
maxmemory-policy allkeys-lru

# Persistence
save 900 1
appendonly yes
appendfsync everysec

# Security
requirepass your-strong-password
bind 127.0.0.1

# Performance
tcp-keepalive 300
timeout 0

Real-World Usage

Twitter

Uses Redis for timeline caching. Each user's timeline is pre-computed and stored in Redis for instant retrieval.

GitHub

Uses Redis for background job queues (Resque/Sidekiq), rate limiting, and session storage.

Snapchat

Uses Redis for real-time messaging and ephemeral data storage, handling millions of messages per second.

Key Takeaways

  • Redis = Swiss Army knife - Data structures, persistence, pub/sub
  • Memcached = Simple and fast - Pure caching, multi-threaded
  • Sorted Sets are powerful - Leaderboards, rate limiting, time-series
  • Redis Cluster for scale - Automatic sharding across nodes

Next up: Caching Assessment - Test your distributed caching knowledge.

🧠 Quick Quiz

Test your understanding of this lesson.

1

What is a key difference between Redis and Memcached?

2

When would you choose Memcached over Redis?

3

What Redis data structure would you use for a leaderboard?

Cache Invalidation