Scaling: Vertical vs Horizontal
Your app is successful. Traffic is growing. Users are complaining about slow response times. Your single server is struggling. What do you do?
You have two choices: make your server bigger or add more servers.
The Single Server Architecture
Every application starts here. One server handles everything:
What Can a Single Server Handle?
A well-configured server with good hardware can handle:
- 1,000-5,000 concurrent connections
- 10,000-50,000 requests per second (simple APIs)
- 1-10 TB of storage
That sounds like a lot, but it's not enough when you're Twitter or Netflix.
Twitter started in 2006 as a Ruby on Rails app with a single MySQL database. When it got featured on TV, the site crashed. Repeatedly. The famous "Fail Whale" became a meme because one server couldn't handle the traffic spikes.
Vertical Scaling (Scaling Up)
The simple solution: get a bigger machine.
Before: Small Server
- 4 CPU cores
- 16 GB RAM
- 500 GB SSD
After: Big Server
- 64 CPU cores
- 512 GB RAM
- 10 TB NVMe
Pros of Vertical Scaling
No code changes required
Everything is on one machine
Single source of truth
Cons of Vertical Scaling
Can't exceed physical limits
High-end servers cost exponentially more
One machine goes down, everything goes down
Horizontal Scaling (Scaling Out)
The distributed solution: add more machines.
Pros of Horizontal Scaling
Keep adding servers as needed
One server dies, others continue
Many small machines cheaper than one giant
Cons of Horizontal Scaling
Requires architectural changes
Keeping data in sync is hard
Servers need to communicate
The Key to Horizontal Scaling: Stateless Design
For horizontal scaling to work seamlessly, your application servers must be stateless.
Stateful vs Stateless
Stateful Server
Stores session data locally (e.g., user login state in memory)
Stateless Server
No local session data. State stored in shared DB or cache.
Making Services Stateless
# ❌ Stateful: Session stored in server memory
user_sessions = {} # Dies when server restarts
@app.route('/login')
def login():
user_sessions[user_id] = session_data
return "Logged in"
# ✓ Stateless: Session stored in Redis (shared)
import redis
cache = redis.Redis()
@app.route('/login')
def login():
cache.set(f"session:{user_id}", session_data)
return "Logged in"Real-World Scale: Twitter's Infrastructure
In 2010, Twitter began migrating from a hosted data center to their own infrastructure. By 2017, they had:
Moving to Mesos allowed Twitter to "codify configurations and deploy slowly to preserve hit-rate while growing and scaling with higher confidence."
— Twitter Infrastructure Blog
When to Scale: Decision Framework
| Scenario | Recommendation |
|---|---|
| Early stage, traffic growing | Vertical scaling first (simpler) |
| High availability required | Horizontal (eliminates SPOF) |
| Traffic is spiky/unpredictable | Horizontal (auto-scaling) |
| Cost-sensitive, predictable load | Vertical until you hit limits |
| Global user base | Horizontal (multi-region) |
Key Takeaways
- Vertical scaling = bigger machine. Simple but limited.
- Horizontal scaling = more machines. Complex but unlimited.
- Stateless design is required for effective horizontal scaling.
- Start simple - scale vertically first, then horizontally when needed.
Next up: Load Balancing - How to distribute traffic across multiple servers effectively.