Load Balancer (LB)

A load balancer distributes incoming requests across a pool of backend servers (upstream targets) to optimize resource utilization, maximize throughput, minimize response time, and avoid overloading any single server.

Types by OSI layer:

• Layer 4 (Transport): routes based on IP address and TCP/UDP port. Does not inspect application content. Fast and efficient. Examples: AWS NLB, HAProxy TCP mode.
• Layer 7 (Application): inspects HTTP headers, URL paths, cookies, and content. Can route based on application logic (path-based routing, host-based routing, cookie affinity). Examples: AWS ALB, Nginx, Envoy, Cloudflare.

Distribution algorithms:

Algorithm	Description
Round Robin	Rotate sequentially through servers
Weighted Round Robin	Servers with higher weight get more requests
Least Connections	Send to the server with fewest active connections
IP Hash	Hash the client IP to consistently route to the same server
Random	Random server selection (surprisingly effective at scale)

Health checks: the load balancer periodically probes each backend server (HTTP GET, TCP connect, or custom script). Unhealthy servers are removed from the pool and re-added when they recover.

Session persistence (sticky sessions): some applications require a client to consistently reach the same backend (e.g., shopping cart stored in server memory). Achieved via cookie injection, IP hash, or application-layer routing.

TLS termination: load balancers often decrypt HTTPS traffic (terminate TLS), inspect the plaintext HTTP, route to the appropriate backend, and optionally re-encrypt for the backend hop.

# /etc/nginx/conf.d/app.conf
upstream app_servers {
    least_conn;                    # Least connections algorithm
    server 10.0.1.10:3000 weight=3;
    server 10.0.1.11:3000 weight=2;
    server 10.0.1.12:3000 weight=1;
    server 10.0.1.13:3000 backup;  # Only used when others are down
}

server {
    listen 443 ssl;
    server_name app.bytesnation.com;

    ssl_certificate     /etc/ssl/certs/app.pem;
    ssl_certificate_key /etc/ssl/private/app.key;

    location / {
        proxy_pass http://app_servers;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    location /api/health {
        return 200 'ok';
    }
}