Load Balancing Algorithms

You have 5 servers behind a load balancer. A request comes in. Which server gets it? That decision is made by the load balancing algorithm.

Different algorithms optimize for different goals: simplicity, fairness, session affinity, or adaptive load distribution.

1. Round Robin

The simplest algorithm: distribute requests in a circular pattern.

Requests:  1   2   3   4   5   6   7   8   9
           │   │   │   │   │   │   │   │   │
           ▼   ▼   ▼   ▼   ▼   ▼   ▼   ▼   ▼
Server:    A   B   C   A   B   C   A   B   C

How It Works

class RoundRobinBalancer:
    def __init__(self, servers):
        self.servers = servers
        self.current = 0

    def next_server(self):
        server = self.servers[self.current]
        self.current = (self.current + 1) % len(self.servers)
        return server

# Usage
balancer = RoundRobinBalancer(['A', 'B', 'C'])
# next_server() returns: A, B, C, A, B, C, ...

2. Weighted Round Robin

Like Round Robin, but servers with higher capacity get more requests.

Out of every 10 requests:
- Server A gets 5 (weight 5)
- Server B gets 3 (weight 3)
- Server C gets 2 (weight 2)

3. Least Connections

Route to the server with the fewest active connections.

Current state:
┌─────────────┬─────────────┬─────────────┐
│  Server A   │  Server B   │  Server C   │
│  45 conns   │  12 conns ← │  38 conns   │
└─────────────┴─────────────┴─────────────┘

New request → Server B (fewest connections)

How It Works

class LeastConnectionsBalancer:
    def __init__(self, servers):
        self.servers = servers
        self.connections = {server: 0 for server in servers}

    def next_server(self):
        # Find server with minimum connections
        server = min(self.connections, key=self.connections.get)
        self.connections[server] += 1
        return server

    def release(self, server):
        self.connections[server] -= 1

4. Weighted Least Connections

Combines Least Connections with server weights.

Score = Active Connections / Weight

Server A: 45 connections, weight 5 → score = 9
Server B: 12 connections, weight 3 → score = 4 ← lowest!
Server C: 20 connections, weight 2 → score = 10

New request → Server B

5. IP Hash

Hash the client's IP address to determine the server. Same client always goes to the same server.

def ip_hash(client_ip, servers):
    hash_value = hash(client_ip)
    server_index = hash_value % len(servers)
    return servers[server_index]

# Client 192.168.1.100 → Always Server B
# Client 192.168.1.101 → Always Server C
# Client 192.168.1.102 → Always Server A

6. Consistent Hashing

Like IP Hash, but adding/removing servers only affects ~1/N of the keys.

Hash Ring (0 to 360 degrees):

        0°
        │
    S1 ─┼─ S2
   /    │    \
  /     │     \
270° ───┼─── 90°
  \     │     /
   \    │    /
    S3 ─┼─
        │
       180°

Client IP hashed to 45° → lands between S1 and S2 → goes to S2
Add S4 at 60°: Only clients between 45°-60° move from S2 to S4

# Regular hash: Adding a server rehashes ~80% of keys
servers_before = ['A', 'B', 'C', 'D', 'E']  # hash(key) % 5
servers_after = ['A', 'B', 'C', 'D', 'E', 'F']  # hash(key) % 6
# Most keys now map to different servers!

# Consistent hash: Adding a server only moves ~1/N keys
# Only keys in the new server's range move to it

7. Random

Simply pick a random server.

import random

def random_balancer(servers):
    return random.choice(servers)

Surprisingly effective! With large numbers of requests, random selection naturally distributes load fairly evenly.

Algorithm Comparison

Real-World: NGINX Configuration

# Round Robin (default)
upstream backend {
    server 10.0.0.1;
    server 10.0.0.2;
    server 10.0.0.3;
}

# Weighted
upstream backend_weighted {
    server 10.0.0.1 weight=5;
    server 10.0.0.2 weight=3;
    server 10.0.0.3 weight=2;
}

# Least Connections
upstream backend_least {
    least_conn;
    server 10.0.0.1;
    server 10.0.0.2;
    server 10.0.0.3;
}

# IP Hash
upstream backend_sticky {
    ip_hash;
    server 10.0.0.1;
    server 10.0.0.2;
    server 10.0.0.3;
}

Key Takeaways

Next up: Health Checks - How load balancers know which servers are alive.