Why Load Balancing Matters
It's Black Friday. Your e-commerce site is getting 50,000 requests per second. A single server can handle maybe 2,000. Without load balancing, you're about to lose millions in sales.
Load balancing is the traffic cop of the internet - distributing incoming requests across multiple servers so no single machine gets overwhelmed.The Single Server Problem
WITHOUT LOAD BALANCING
50,000 requests/sec
│
▼
┌─────────┐
│ Server │ ← Overwhelmed!
│ (1) │ Crashes at 2K req/s
└─────────┘
│
▼
💥 DOWNWITH LOAD BALANCING
50,000 requests/sec
│
▼
┌──────────┐
│ Load │
│ Balancer │
└──────────┘
│ │ │
▼ ▼ ▼
┌──┐ ┌──┐ ┌──┐ ← 25 servers
│S1│ │S2│ │..│ 2K each = 50K total
└──┘ └──┘ └──┘What Load Balancers Do
Spread requests across multiple servers
Remove failing servers from the pool
Handle encryption at the edge
Route users to the same server
The Load Balancing Flow
Client Request → DNS → Load Balancer → Backend Server → Response
Step by step:
1. User visits myshop.com
2. DNS resolves to load balancer IP (e.g., 203.0.113.10)
3. Request hits load balancer
4. Load balancer picks a healthy backend server
5. Request forwarded to chosen server
6. Server processes and responds
7. Response flows back through load balancer to clientBenefits of Load Balancing
1. High Availability
If one server crashes, the load balancer routes traffic to healthy servers. Users never see downtime. You can even do rolling deployments - update servers one at a time while others handle traffic.
2. Horizontal Scalability
Traffic growing? Just add more servers.
Before: LB → [S1] [S2] [S3] = 6,000 req/s capacity
After: LB → [S1] [S2] [S3] [S4] [S5] = 10,000 req/s capacity
No code changes. No downtime. Just provision and add.3. Flexibility
| Scenario | Without LB | With LB |
|---|---|---|
| Server maintenance | Downtime required | Drain and maintain transparently |
| Traffic spike | Server crashes | Auto-scale and distribute |
| Server failure | Complete outage | Automatic failover |
| Deploy new version | Blue-green is complex | Rolling deploy or canary |
Real-World: Netflix's Load Balancing
Netflix uses a multi-tier load balancing approach. At the edge, they use their own CDN (Open Connect) with edge servers worldwide. For internal services, they use Eureka for service discovery and Ribbon for client-side load balancing. A single popular show launch can generate millions of concurrent streams - all seamlessly distributed.
Types of Load Balancers
Hardware Load Balancers
Physical appliances (F5, Citrix). Expensive but extremely fast. Common in enterprise data centers.
Software Load Balancers
NGINX, HAProxy, Envoy. Run on commodity hardware. Flexible and cost-effective.
Cloud Load Balancers
AWS ALB/NLB, GCP Load Balancer, Azure LB. Managed, auto-scaling, pay-per-use.
Where Load Balancers Sit
Internet
│
▼
┌─────────────────┐
│ DNS (Route 53) │
└────────┬────────┘
│
▼
┌─────────────────┐
│ Global LB (CDN) │ ← Cloudflare, AWS CloudFront
└────────┬────────┘
│
┌──────────────┼──────────────┐
│ │ │
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│Region A │ │Region B │ │Region C │
│ LB │ │ LB │ │ LB │ ← Regional LBs
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
┌───┴───┐ ┌───┴───┐ ┌───┴───┐
│Servers│ │Servers│ │Servers│
└───────┘ └───────┘ └───────┘Common Load Balancer Features
- Health Checks - Continuously monitor backend servers and remove unhealthy ones
- SSL/TLS Termination - Decrypt HTTPS at the load balancer, send HTTP to backends
- Session Persistence - Sticky sessions to keep users on the same server
- Connection Draining - Gracefully remove servers without dropping active connections
- Rate Limiting - Protect backends from traffic spikes or DDoS attacks
Key Takeaways
- Load balancers distribute traffic - No single server bears all the load
- Enable horizontal scaling - Add more servers to handle more traffic
- Improve availability - Automatic failover when servers go down
- Foundation for modern architecture - Essential for any high-traffic application
Next up: Load Balancing Algorithms - Round Robin, Least Connections, IP Hash, and more.