Well, in short, yes. > I always thought that for hundreds of backends and above 10K RPS those algorithms scale poorly https://www.haproxy.com/blog/haproxy-for

Well, in short, yes. > I always thought that for hundreds of backends and above 10K RPS those algorithms scale poorly https://www.haproxy.com/blog/haproxy-forwards-over-2-million-http-requests-per-second-on-a-single-aws-arm-instance/ https://www.haproxy.com/blog/haproxy-forwards-over-2-million... > First of all, scaling beyond single load balancer becomes a problem. If you are in DC or have L3 access to the underlying network ECMP is VERY easy way to scale to 4-16 loadbalancers. Over that just... use multiple IPs. We use ECMPed setup of 4 nodes since forever and it works very well. There are theoretical improvements that could be made by doing L4 loadabalancing into L7 loadbalancing, but you'd need a hell lot of traffic to ever need that > The cost of updating counters becomes significant. If responses are quick, the cost additional bookkeeping adds doesn't pay off. Essentially, yes but if you have some architecture that talks very little (say just few packets per connection, let's assume some IoT garbage), you probably just want different architecture altogether; like "first request gets you assigned server, then you talk to it directly without LB". L4 balancing is also an option. In essence, if somehow performance of leastconn algorithm is your bottleneck, you're both big enough and specialized enough to go with another approach than "just dump all traffic at few loadbalancers"