Load Balancing Techniques

Why This Matters

Load balancing sits at the heart of cloud computing architecture—it's the traffic cop that determines whether your distributed system performs smoothly or collapses under pressure. You're being tested on your understanding of scalability, fault tolerance, availability, and resource optimization, and load balancing techniques demonstrate all of these principles in action. Every algorithm represents a different trade-off between simplicity, intelligence, and overhead.

Don't just memorize which algorithm does what—understand why you'd choose one over another. Exam questions will present scenarios and ask you to recommend the appropriate technique, or they'll probe whether you understand the underlying mechanisms: session persistence, connection tracking, health monitoring, and weighted distribution. Know the concept each technique illustrates, and you'll handle any question they throw at you.

---

Static Distribution Algorithms

These algorithms distribute traffic using predetermined rules without considering real-time server state. They're simple to implement but lack adaptability—the load balancer makes decisions without knowing how servers are actually performing.

Round Robin

• Cycles through servers sequentially—each new request goes to the next server in the rotation, then wraps back to the first
• Zero server awareness means no monitoring overhead, but assumes all servers have identical capacity and current load
• Best for homogeneous clusters where servers are truly equivalent; fails when capacity varies

Weighted Round Robin

• Assigns capacity weights to each server—a server with weight 3 receives three times the requests of a weight-1 server
• Handles heterogeneous environments where servers have different CPU, memory, or processing capabilities
• Requires manual configuration of weights, which can become stale if server performance changes over time

Random

• Selects servers using random distribution—statistically even over time but unpredictable per-request
• Minimal implementation complexity with no state tracking required between requests
• Probabilistically fair only at scale; short bursts may create temporary imbalances

---

Dynamic Load-Aware Algorithms

These algorithms make routing decisions based on real-time server metrics. They're more intelligent but require continuous monitoring infrastructure to track server state.

Least Connection

• Routes to the server with fewest active connections—dynamically adapts as connections open and close
• Ideal for long-lived connections like WebSocket or database sessions where connection count reflects actual load
• Requires connection tracking at the load balancer, adding state management overhead

Least Response Time

• Prioritizes servers responding fastest—combines connection count with latency measurements for smarter routing
• Optimizes end-user experience by minimizing perceived delay, not just balancing server load
• Demands continuous latency monitoring and may oscillate if response times fluctuate rapidly

Least Bandwidth

• Directs traffic to servers consuming least network bandwidth—measures actual data throughput, not just connections
• Critical for data-intensive applications like video streaming or large file transfers where bandwidth is the bottleneck
• Requires real-time bandwidth metrics which adds monitoring complexity but prevents network saturation

---

Session-Aware Algorithms

These algorithms ensure clients maintain consistent server relationships across multiple requests. They solve the session persistence problem—keeping user state intact when applications store session data locally on servers.

IP Hash

• Hashes client IP address to determine server assignment—same IP always maps to same server
• Guarantees session persistence without requiring shared session storage or cookies
• Vulnerable to uneven distribution if traffic comes from concentrated IP ranges (like corporate NATs or proxies)

---

Content-Aware Algorithms

These algorithms inspect request content to make intelligent routing decisions. They operate at Layer 7 (application layer) rather than Layer 4 (transport layer), enabling application-specific optimization.

URL-Based

• Routes based on requested URL path—different endpoints go to different server pools
• Enables microservices routing where /api/ hits application servers while /static/ hits content servers
• Can create hotspots if certain URLs receive disproportionate traffic without additional balancing

Content-Based

• Inspects request payload or headers to route based on content type, file format, or data characteristics
• Allows specialized server pools—image processing servers, video transcoding servers, API servers
• Requires deep packet inspection which adds latency and configuration complexity

---

Health and Availability Mechanisms

Health monitoring isn't a load balancing algorithm itself—it's the foundation that makes all other algorithms reliable. Without it, load balancers route traffic to failed servers.

Server Health Monitoring

• Performs continuous health checks—typically HTTP pings, TCP connections, or custom application probes at regular intervals
• Removes unhealthy servers from rotation automatically and re-adds them when health checks pass again
• Enables automatic failover to backup servers, which is essential for achieving high availability SLAs

---

Quick Reference Table

---

Self-Check Questions

1. Which two algorithms are both static (require no real-time monitoring) but differ in whether distribution is deterministic or probabilistic?

2. A web application stores user shopping carts in server memory without shared session storage. Which load balancing technique ensures users don't lose their carts, and what's its main drawback?

3. Compare and contrast Least Connection and Least Bandwidth—what metric does each optimize for, and when would you choose one over the other?

4. Your architecture uses microservices with separate server pools for /api/, /images/, and /video/. Which load balancing approach enables this routing, and at which OSI layer does it operate?

5. An FRQ describes a system where the load balancer continues sending traffic to a crashed server. What mechanism is missing, and how would adding it improve the system's fault tolerance?