What is load balancing, what are the common types of load balancers, and why is it a fundamental component in system design?

Question

Accepted Answer

Load balancing distributes incoming network traffic across multiple servers to ensure no single server becomes overwhelmed. It is a foundational system design component because it enables both scalability and high availability.

At its core, a load balancer sits between clients and a pool of backend servers. When a request arrives, the load balancer selects a server based on an algorithm and forwards the request. Common algorithms include round robin, which cycles through servers sequentially; least connections, which sends traffic to the server handling the fewest active requests; and weighted distribution, which directs more traffic to more powerful servers.

There are several types of load balancers. Application Load Balancers operate at Layer 7, the HTTP level. They can inspect request content and route based on URL paths, hostnames, or HTTP headers. This makes them ideal for microservices architectures where different services handle different URL paths. Network Load Balancers operate at Layer 4, the TCP/UDP level. They handle millions of requests per second with ultra-low latency, making them suitable for real-time gaming, IoT, or any workload requiring raw TCP performance. Gateway Load Balancers route traffic through third-party virtual appliances like firewalls before it reaches your application.

Load balancing provides several architectural benefits. For availability, if one server fails its health check, the load balancer automatically stops sending traffic to it and redistributes across healthy servers. This means users experience no downtime even when individual servers crash. For scalability, you can add more servers behind the load balancer to handle increased traffic without changing the application. This is the foundation of horizontal scaling.

In cloud environments, managed load balancers like AWS ALB, Azure Application Gateway, or Google Cloud Load Balancing handle the undifferentiated heavy lifting of certificate management, health checking, and auto-scaling integration. They also provide features like SSL termination, where the load balancer handles encryption and decryption so backend servers do not need to, and sticky sessions, which route a user's requests to the same server when session state is stored locally.

A well-designed system places load balancers at multiple tiers: between users and web servers, between web servers and application servers, and between application servers and databases, creating defense in depth for availability.