Sockets, Load Balancing, and CDNs

Once you understand the protocols, the next question is how real systems use them at scale. This lesson covers three building blocks that appear in almost every backend design: sockets (the programming interface to the network), load balancers (how you spread traffic across many servers), and CDNs (how you serve content close to users). Interviewers use these to bridge networking fundamentals into system design.

Sockets: the API to the network

A socket is the endpoint through which a program sends and receives data over the network. A TCP connection is uniquely identified by a 4-tuple:

(source IP, source port, destination IP, destination port)

This is why one server listening on port 443 can serve thousands of simultaneous clients: each connection differs in the client's IP/port, so the 4-tuples are all distinct.

The server-side socket lifecycle

socket()   -> create a socket
bind()     -> attach it to an IP:port (e.g. 0.0.0.0:8080)
listen()   -> mark it passive; OS queues incoming connections
accept()   -> pull one established connection off the queue (new socket)
read()/write() or recv()/send()
close()

The client side is simpler: socket(), then connect() (which triggers the TCP handshake), then read/write, then close.

Blocking vs non-blocking and the C10k problem

A naive server uses one thread per connection with blocking I/O — simple, but memory and context-switching costs explode at tens of thousands of connections (the classic C10k problem). High-concurrency servers instead use non-blocking sockets with an event loop backed by an OS readiness mechanism — epoll (Linux), kqueue (BSD/macOS), or IOCP (Windows) — so a single thread can watch thousands of sockets. This is the model behind Nginx, Node.js, and Redis.

Load balancing

A load balancer (LB) sits in front of a pool of servers and distributes incoming requests, providing horizontal scale, high availability, and a single stable entry point (a VIP, virtual IP). It also runs health checks and stops routing to unhealthy backends.

L4 vs L7 load balancers

| | L4 (Transport) | L7 (Application) | |---|---|---| | Operates on | IP + port (TCP/UDP) | HTTP: URL, headers, cookies | | Visibility | Cannot read payload | Reads and can route on content | | Speed | Very fast, low overhead | More CPU (parses/terminates TLS) | | Features | Simple pass-through | Path routing, TLS termination, sticky sessions, header rewrite | | Example | AWS NLB, LVS | AWS ALB, Nginx, HAProxy, Envoy |

An L4 balancer forwards packets by 4-tuple without understanding HTTP — extremely fast. An L7 balancer terminates the connection, reads the HTTP request, and can route /api to one pool and /images to another, do TLS termination, and rewrite headers.

Balancing algorithms

• Round robin — rotate through servers in order. Simple; ignores load.
• Weighted round robin — bigger servers get more traffic.
• Least connections — send to the server with the fewest active connections; good for uneven request durations.
• IP hash / consistent hashing — map a client (or key) to a server deterministically, giving session stickiness. Consistent hashing minimizes reshuffling when servers are added or removed — only ~1/N of keys move — which is why it underpins distributed caches and sharding.

Health checks and failover

The LB periodically probes each backend (e.g. GET /healthz). A backend that fails N checks is removed from rotation and re-added when it recovers. This is what turns a pool of fragile servers into a resilient service.

CDNs

A Content Delivery Network is a globally distributed set of edge servers that cache content close to users. Instead of every request traveling to a single origin (maybe an ocean away), users are served from a nearby PoP (point of presence), cutting latency dramatically and shielding the origin from load.

How a request finds the nearest edge

Two common mechanisms:

• DNS-based — the CDN's authoritative DNS returns the IP of a nearby edge based on the resolver's location.
• Anycast — the same IP is announced from many locations; BGP routes the client to the topologically nearest one.

What a CDN caches, and cache control

CDNs excel at static assets — images, CSS, JS, video, fonts — and increasingly cache dynamic/API responses at the edge. Behavior is driven by HTTP caching headers:

Cache-Control: public, max-age=31536000, immutable   (cache for a year)
Cache-Control: no-store                               (never cache)

• Cache hit — served straight from the edge; fastest.
• Cache miss — the edge fetches from the origin (or a mid-tier), caches it, then serves it.
• Invalidation — you purge content or, better, use cache-busting (fingerprinted filenames like app.9f2a.js) so a new deploy has new URLs and old caches are simply bypassed.

CDNs also absorb DDoS traffic and terminate TLS at the edge, adding a security and performance layer beyond simple caching.

Putting it together

A production request path often looks like: client → CDN edge (static hit) → L7 load balancer → app servers → cache → database. Each hop exists to add scale, resilience, or latency reduction.

In the interview

1. Define a socket as a 4-tuple and explain why one listening port serves many clients.
2. Contrast L4 vs L7 — the ability to route on HTTP content is the key L7 advantage.
3. Know least-connections and consistent hashing and when each is preferable.
4. Explain CDN hit/miss and cache-busting, and note edge benefits beyond caching (DDoS absorption, TLS termination).