A single server can only handle so much traffic before it starts dropping requests. A load balancer sits in front of a group of servers and decides which one should handle each incoming request. It’s the reason you can hit a website millions of times a day and still get a response in milliseconds.

But “distributing traffic” is a vague description. How does a load balancer actually pick a server? What happens when a server goes down? And why do some sessions need to stick to the same backend? Let’s break it all down.

What a Load Balancer Actually Does

At its core, a load balancer is a reverse proxy with a routing brain. A client sends a request, the load balancer receives it, picks a healthy backend server, and forwards the request there. The client never talks to the backend directly.

This gives you three things:

• Scalability — add more servers behind the load balancer to handle more traffic.
• Reliability — if one server dies, the load balancer stops sending it traffic. No 502 errors for your users.
• Flexibility — you can deploy, restart, or upgrade individual servers without downtime.

L4 vs L7 Load Balancing

Load balancers operate at different layers of the network stack, and the layer matters.

Layer 4 (Transport) load balancers make routing decisions based on TCP/UDP information — source IP, destination IP, and port numbers. They don’t inspect the actual content of the request. This makes them fast and efficient, but they can’t route based on URL paths, headers, or cookies.

Layer 7 (Application) load balancers understand HTTP. They can read the request URL, headers, cookies, and even the body. This lets you do things like:

• Route /api/* requests to your API servers and /static/* to a CDN origin.
• Send requests with a specific cookie to a particular backend.
• Reject malformed requests before they ever reach your servers.

L7 is more powerful but adds latency because the load balancer has to parse the full HTTP request. L4 is leaner and better suited for raw throughput — think database connections or game servers.

Load Balancing Algorithms

This is where the real decision-making happens. When a request arrives, the load balancer uses an algorithm to pick a backend.

Round Robin

The simplest approach. Requests go to servers in order: Server A, Server B, Server C, then back to A. Every server gets an equal share of traffic.

It works well when all your servers are identical and requests take roughly the same amount of time to process. It falls apart when servers have different capacities or some requests are much heavier than others.

Weighted Round Robin

Same idea as round robin, but you assign weights. If Server A has a weight of 3 and Server B has a weight of 1, A gets three requests for every one that B gets. Useful when your servers have different hardware specs — a 16-core machine should handle more traffic than a 4-core one.

Least Connections

Instead of rotating blindly, the load balancer tracks how many active connections each server has and sends the next request to whichever server has the fewest. This naturally adapts to servers that are slower or handling heavier requests — they accumulate connections, so they get fewer new ones.

This is often the best default choice for web applications where request processing times vary.

IP Hash

The client’s IP address is hashed, and the hash determines which server gets the request. The same client IP always maps to the same server (as long as the server pool doesn’t change). This is a simple way to achieve session persistence without cookies, though it can lead to uneven distribution if a large chunk of traffic comes from a single IP range — like users behind a corporate NAT.

Random

Pick a server at random. Surprisingly effective at scale. With enough requests and enough servers, random selection produces a fairly even distribution. Some implementations use “power of two choices” — pick two servers at random, then send the request to whichever has fewer connections. This simple tweak dramatically reduces the chance of overloading a single server.

Health Checks

A load balancer is only useful if it knows which servers are actually alive. Health checks are how it finds out.

There are two types:

• Passive health checks — the load balancer monitors real traffic. If a server starts returning errors or timing out, it gets marked as unhealthy.
• Active health checks — the load balancer periodically sends a probe request (like GET /health) to each server. If a server fails to respond correctly after a configured number of attempts, it’s pulled from the pool.

Most production setups use both. Active checks catch servers that are up but broken (returning 500s, for example). Passive checks react faster to sudden failures.

When a server is marked unhealthy, the load balancer stops routing traffic to it. Once it starts passing health checks again, it’s gradually reintroduced — often with a “slow start” period where it receives fewer requests to avoid being overwhelmed.

Sticky Sessions

Sometimes you need the same user to hit the same server for every request in a session. This is called session affinity or sticky sessions.

Common reasons:

• The server stores session data in local memory (not in a shared store like Redis).
• A multi-step workflow where state is kept server-side between requests.
• WebSocket connections that must persist on a single backend.

L7 load balancers typically implement sticky sessions using cookies. The load balancer injects a cookie (like SERVERID=backend2) into the response. On subsequent requests, the client sends the cookie back, and the load balancer routes to the same server.

The downside: if that server goes down, the session is lost. Sticky sessions trade resilience for simplicity. The better long-term fix is externalizing session state so any server can handle any request.

SSL Termination

Encrypting traffic with TLS is non-negotiable, but the encryption/decryption work has to happen somewhere. SSL termination means the load balancer handles TLS — it decrypts incoming HTTPS requests, then forwards plain HTTP to the backend servers.

This offloads CPU-intensive crypto work from your application servers and gives you a single place to manage certificates. The traffic between the load balancer and backends is unencrypted, which is fine if they’re on a private network. If you need end-to-end encryption, you can use SSL passthrough (L4) or re-encrypt the traffic between the load balancer and backends.

Common Tools

Nginx — started as a web server, now one of the most popular L7 load balancers. Configuration is straightforward, performance is excellent, and the open-source version covers most use cases.

HAProxy — purpose-built for load balancing. Supports both L4 and L7, has incredibly detailed health checking, and is the go-to choice for high-throughput environments. Powers some of the largest sites on the internet.

AWS ALB (Application Load Balancer) — managed L7 load balancer. Handles path-based routing, host-based routing, and integrates natively with ECS, EKS, and Lambda. Its L4 counterpart, NLB (Network Load Balancer), handles millions of requests per second with ultra-low latency.

Cloudflare — operates at the DNS level and edge network. Their load balancer routes traffic across multiple origins globally, with built-in health checks and failover. Great for multi-region setups.

For Kubernetes environments, Ingress controllers (often backed by Nginx or Envoy) act as the load balancer for services inside the cluster.

When Do You Need One?

You need a load balancer when:

• You’re running more than one instance of your application.
• You want zero-downtime deployments (drain connections from one server while deploying to it).
• You need to handle traffic spikes without a single server becoming a bottleneck.
• You want automatic failover when a server crashes.

You might not need one if you’re running a single server for a small project — though even then, a managed load balancer gives you a stable endpoint and health checking for very little cost.

The load balancer is one of those infrastructure pieces that’s invisible when it works and catastrophic when it doesn’t. Understanding how it routes, checks, and sticks gives you the knowledge to configure it properly — and debug it when things go sideways.