Distribute incoming traffic across multiple backend instances.

Data Flow

                   [Load Balancer]
                          |
        +-----------------+-----------------+
        v                 v                 v
+-------+         +-------+         +-------+
| app_0 |         | app_1 |         | app_2 |
+-------+         +-------+         +-------+

Layer 4 vs Layer 7

• L4: hashes src_ip:port + dst_ip:port. Cheaper. Sticky on the URL or user.
• L7: inspects HTTP headers / path / cookies. Smarter routing; can do A/B tests, canary releases.

Algorithms

• Round-robin; weighted round-robin; least-connections; weighted least-connections.
• IP-hash (for sticky sessions); consistent hash (for cache affinity).
• Random with two choices (power-of-two-choices): pick two random servers, send to the one with fewer active connections. Provably near-optimal without any global view — the default behind modern LBs like Envoy.

Pitfalls

• No health checks: a dead instance still receives 50% of traffic.
• Sticky sessions when state isn’t actually kept server-side.
• Single LB is a SPOF: use two LBs in active/active or DNS-level failover.

Analogy

The receptionist at a busy clinic — routes patients to the doctor with the shortest queue, but always checks the doctor’s break status first.

Interview tip: Specify the routing algorithm + health-check mechanism + how the LB itself is fault-tolerant.