4 ms·

I always wonder just how much can a single load balancer do? Even if it is just forwarding request/replies, at something like hundreds of thousands of queries p

I always wonder just how much can a single load balancer do? Even if it is just forwarding request/replies, at something like hundreds of thousands of queries per second how does it work?

stingraycharles 5y ago
First of all, you’ll be amazed at what hardware load balancers can do. Secondly, if you want you can still have multiple load balancers listen to a single IP; it’s a bit more tricky to pull that off with HTTP load balancing rather than just network load balancing, but not impossible. Example of a load balancer appliance that allows multiple load balancers to share a single IP: https://docs.paloaltonetworks.com/pan-os/10-1/pan-os-admin/high-availability/ha-concepts/arp-load-sharing.html https://docs.paloaltonetworks.com/pan-os/10-1/pan-os-admin/h...
- 8eye 5y ago
  sounds cool, is there bottle neck? or not really noticeable?
grillvogel 5y ago
F5 load balancers have an absurd amount of features, and can load balance 100Gb traffic flows
matsur 5y ago
Cloudflare deploys software load balancers in all its datacenters, with every machine capable of acting as an LB for every other machine (and itself) within the datacenter. More here: https://blog.cloudflare.com/unimog-cloudflares-edge-load-balancer/ https://blog.cloudflare.com/unimog-cloudflares-edge-load-bal... Unimog is key to making the deployment described in the paper work. Another way of thinking about this is that we have a single virtual load balancer (backed by all the physical machines in the DC) sitting in front of services that transform/terminate/filter/etc customer traffic.
- omegalulw 5y ago
  Thanks! I am more curious about the physical aspect of it. How do you get multiple switches to share an IP? What determines with switch forwards the packets? Like, simply, what wires connect to what?
  - kijiki 5y ago
    Your switches run a routing protocol (almost always BPG) and your load balancers all announce the same IP to them. This results in a routing table with many next hops for the LB IP, and traffic is distributed across them using ECMP, typically with some resilient hashing scheme so that a given flow will always go to the same physical LB, even if there are topology changes due to LB or switch failures. At the edges, your switches or border routers announce the IP to all your peers and transit ISPs.
    - matsur 5y ago
      Yes, this. The beauty of the Unimog setup we run in production is we are resilient to ECMP rehash events — each server acting as a load balancer has access to state (backed by Consul) that tells it which flow belongs to which application server.
mrkstu 5y ago
Basically the upper limit is ephemeral port exhaustion: https://blog.box.com/ephemeral-port-exhaustion-and-web-services-at-scale https://blog.box.com/ephemeral-port-exhaustion-and-web-servi...

I always wonder just how much can a single load balancer do? Even if it is just forwarding request/replies, at something like hundreds of thousands of queries per second how does it work?

stingraycharles 5y ago

First of all, you’ll be amazed at what hardware load balancers can do. Secondly, if you want you can still have multiple load balancers listen to a single IP; it’s a bit more tricky to pull that off with HTTP load balancing rather than just network load balancing, but not impossible. Example of a load balancer appliance that allows multiple load balancers to share a single IP: https://docs.paloaltonetworks.com/pan-os/10-1/pan-os-admin/high-availability/ha-concepts/arp-load-sharing.html https://docs.paloaltonetworks.com/pan-os/10-1/pan-os-admin/h...

8eye 5y ago
sounds cool, is there bottle neck? or not really noticeable?

grillvogel 5y ago

F5 load balancers have an absurd amount of features, and can load balance 100Gb traffic flows

matsur 5y ago

Cloudflare deploys software load balancers in all its datacenters, with every machine capable of acting as an LB for every other machine (and itself) within the datacenter. More here: https://blog.cloudflare.com/unimog-cloudflares-edge-load-balancer/ https://blog.cloudflare.com/unimog-cloudflares-edge-load-bal... Unimog is key to making the deployment described in the paper work. Another way of thinking about this is that we have a single virtual load balancer (backed by all the physical machines in the DC) sitting in front of services that transform/terminate/filter/etc customer traffic.

omegalulw 5y ago
Thanks! I am more curious about the physical aspect of it. How do you get multiple switches to share an IP? What determines with switch forwards the packets? Like, simply, what wires connect to what?
- kijiki 5y ago
  Your switches run a routing protocol (almost always BPG) and your load balancers all announce the same IP to them. This results in a routing table with many next hops for the LB IP, and traffic is distributed across them using ECMP, typically with some resilient hashing scheme so that a given flow will always go to the same physical LB, even if there are topology changes due to LB or switch failures. At the edges, your switches or border routers announce the IP to all your peers and transit ISPs.
  - matsur 5y ago
    Yes, this. The beauty of the Unimog setup we run in production is we are resilient to ECMP rehash events — each server acting as a load balancer has access to state (backed by Consul) that tells it which flow belongs to which application server.

mrkstu 5y ago

Basically the upper limit is ephemeral port exhaustion: https://blog.box.com/ephemeral-port-exhaustion-and-web-services-at-scale https://blog.box.com/ephemeral-port-exhaustion-and-web-servi...