Load Balancing: L4 vs L7, algorithms, health checks

A load balancer sits in front of your servers and distributes traffic. The question is: how much does it understand about what it's routing?

Layer 4 (TCP/UDP) Load Balancing

What it is: Route based on IP, port, and protocol. The load balancer looks at the TCP header and decides which backend server gets this connection.

Example:

Client → LB (looks at source IP, dest port)
         → Server 1 (TCP connection opens)
         → Server 2 (TCP connection opens)
         → Server 3 (TCP connection opens)

Pros:

Ultra-fast: only examines 4 fields (src IP, src port, dst IP, dst port).
Works for any protocol: TCP, UDP, DNS, custom binary protocols.
Low latency: minimal processing per packet.

Cons:

No application awareness: can't route based on URL, hostname, or request content.
Sticky sessions are hard: if the same connection must hit the same backend, you rely on source IP hashing (fails if client behind NAT).
Can't make smart decisions: e.g., route /api to API servers and /static to CDN.

When to use: High-throughput systems (millions of packets/sec), gaming, DNS, where you just need to distribute load evenly.

Layer 7 (Application) Load Balancing

What it is: Route based on application-level data: HTTP host, path, headers, cookies, request body.

Example:

GET /api/users → Route to API servers
GET /static/images → Route to image cache or CDN
GET /admin → Route to admin cluster (requires auth cookie)
POST /checkout → Route to payment servers

Pros:

Smart routing: send traffic where it belongs, not just round-robin.
Session awareness: route user123 always to the same backend based on user-id cookie (true sticky sessions).
Content-based routing: different backends handle different paths, hostnames, or request patterns.
Caching-friendly: can inspect headers to apply caching rules.

Cons:

Higher latency: must parse HTTP, extract headers, make routing decisions. Adds 1-5 ms per request.
Complexity: managing content routing rules, health checks per backend type.
Stateful: L7 LB must maintain connection state (HTTP/2 multiplexing, keep-alives).
Throughput cap: can't scale to as high QPS as L4 (more CPU per request).

When to use: Web APIs, microservices (different services on different paths), where routing logic matters.

Load Balancing Algorithms

Round-Robin

How: Servers take turns. Request 1 → Server 1, Request 2 → Server 2, Request 3 → Server 3, Request 4 → Server 1 (cycle).

Pros: Simple, fair, no configuration.

Cons: Doesn't account for server load. If Server 1 is slow, it still gets traffic.

Weighted Round-Robin

How: Assign weights. Server 1 (weight 2) gets 2 requests, Server 2 (weight 1) gets 1 request.

Pros: Can favor more powerful servers.

Cons: Weights are static; doesn't adapt to real-time load.

Least Connections

How: Route to the server with fewest active connections.

Pros: Adapts to real-world load imbalances.

Cons: High-throughput connections skew the metric (one file download blocks others).

IP Hash

How: Hash client IP. Same client always hits same server (useful for sticky sessions).

Pros: Stateless (no session replication needed), preserves per-client context.

Cons: Breaks when servers are added/removed (hash slots shift, clients route differently).

Consistent Hashing

How: Hash-based routing that survives adding/removing servers.

Pros: Adding a server only rehashes a fraction of keys (see consistent-hashing lesson).

Cons: Slightly more complex than IP hash.

Health Checks

A load balancer periodically probes backends. If a server is down, traffic is rerouted.

Simple Health Check

LB pings Server 1 every 5 seconds:
  GET /health → 200 OK (healthy)
  GET /health → 500 Error (unhealthy, mark as down)
  GET /health → timeout (unhealthy, mark as down)

Sophisticated Health Checks

Checks multiple things:
  1. TCP port open (can connect?)
  2. /health endpoint returns 200 (app alive?)
  3. Latency < 1s (responsive?)
  4. Database connected (deps working?)

If any check fails, mark server as down.

Critical tuning:

Check interval: Every 5-30 seconds (trade-off: fast detection vs overhead).
Unhealthy threshold: If 3 checks fail, mark down (avoid flapping).
Healthy threshold: If 2 checks pass, mark up.

L4 vs L7: When to Choose

| Scenario | L4 | L7 | |----------|----|----| | Web API (HTTP/HTTPS) | ✗ | ✓ | | Game server (UDP) | ✓ | ✗ | | DNS | ✓ | ✗ | | Microservices (path-based routing) | ✗ | ✓ | | High-frequency trading (TCP blobs) | ✓ | ✗ | | Content delivery (static vs dynamic) | ✗ | ✓ | | Million+ QPS with simple routing | ✓ | ✗ |

In the interview

Know the names: L4 = Layer 4 = TCP/UDP. L7 = Layer 7 = Application (HTTP). Interviewers expect this.
Tradeoff: "L4 is faster but dumb. L7 is smart but slower. For APIs, L7; for gaming, L4."
Health checks matter: "I'd poll /health every 10 seconds, mark down after 3 failures to avoid flapping, mark up after 2 successes."
Sticky sessions: "For stateful sessions, I'd use consistent hashing or L7 cookie-based routing rather than IP hash, which breaks when servers scale."