Load Balancing Algorithms – System Design Core Concept

What:

Load balancers act as central gateways, routing incoming traffic streams across a fleet of backend servers to prevent hotspots and ensure high availability.

Primary purpose:

Preventing single-server exhaustion, providing failover routing, and maximizing overall system QPS capacity.

Usually used for:

Stateless web server pools, API gateways, reverse proxies, and microservices internal meshes.

How should I think about this inside system architectures?

🔗 Layer 4 Transport

Low overhead routing. Looks only at IP addresses and TCP port numbers, forwarding raw packets to servers without opening them.

🧠 Layer 7 Smart Routing

High intelligence. Decrypts HTTP/TLS traffic to steer requests dynamically based on URL paths (`/feed`), cookies, or headers.

🏥 Active Health Checks

Continuously ping downstream nodes. Automatically eject crashed servers from the active routing pool within seconds.

Core Load Balancing Algorithms: Matching routing logic to workloads:

Routing Algorithm	Steering Mechanic	Primary Advantage	Severe Drawback
Round Robin	Steers requests sequentially down the server list.	Zero CPU overhead; completely predictable.	Ignores current server CPU load or request execution complexity.
Least Connections	Steers requests to the server with the fewest active TCP sockets.	Ideal for long-lived states (WebSockets, streaming channels).	Requires tracking active states, adding small lookup overheads.
IP / Source Hash	Routes requests by client IP hash modulo active node count.	Simple state persistence (sticky sessions) without central caches.	Re-routing occurs if servers scale up/down (forces Consistent Hashing).

L4 vs L7 Operational Layers Matrix: Structuring the gateway layer:

Layer Profile	Protocol Focus	CPU / Latency Overhead	Security Features
Layer 4 (L4) Transport	TCP / UDP packets routing.	Ultra-low (packets are routed immediately without reading HTTP bodies).	SSL decryption happens downstream at the application servers.
Layer 7 (L7) Application	HTTP / HTTP/2 / WebSocket headers parsing.	Moderate (requires full request decryption to read URL paths/headers).	Enables central SSL termination, header injection, and cookie sticky routing.

Benefit	Cost
L4 Latency Economy (routes raw network packets at wire-speed without decrypting HTTP headers, maximizing throughput)	No Content-Aware Routing (unable to route requests by URL path, query params, or client cookie types)
L7 Smart Routing (easily direct `/users` to the user service fleet, terminate SSL centrally, and enforce rate limits)	High CPU Overhead (decrypting TLS keys and inspecting HTTP bodies requires significant compute power)