3 ms·
Least connections is intuitively very sensible algorithm, but it has one pitfall: if backend server starts returning errors for whatever reason (e.g. load shedd
by zokier 4y ago
Least connections is intuitively very sensible algorithm, but it has one pitfall: if backend server starts returning errors for whatever reason (e.g. load shedding) quicker than typical responses, it can lead to situation where disproportionate number of requests are directed to that one server.
Also typically you'd have more than one load balancer instance, and at least naive least connections requires shared state (backend connection counts) between the instances, while with round-robin you can avoid that?
- samwho 4y agoLeast connections doesn't need shared state between the instances, but it would need shared state between the load balancers if you had more than one for sure. You're totally right, I glossed right over errors. With PEWMA and weighted round robin you can make instances incur a "penalty" for serving errors, which can help you isolate bad servers. It would have been fun to visualise this, definitely.
- mst 4y agoIt also strikes me that it might be interesting to model least connections with two or three LBs with and without such shared state. My instinct is that you'd still get most of the advantages even with each LB keeping its own independent connection counts, but given the number of times I've fired up a profiler and gone "wait, seriously, -that-s the slow part?!" I'd suggest nobody believes me and measures it instead.
- sesm 4y agoThere is a talk about Fitbit load balancer on YouTube, where their algorithm is described. They used modified least connections algorithm, which also tracks failure rate of each node, decreasing the traffic when error rate is increased. The video is called "A Frontend Server, Front to Back by Zach Tellman", timestamp 33:34.
- ed_balls 4y agoyou can improve this be picking top 3 server with least connections and then pick one randomly from these 3 servers.
- lowflyingcondor 4y agoInverting this is actually better because it avoids the stale data / herd behavior. i.e. randomly choose X candidate servers and then pick the one with least load. also, 2 may be better than 3 https://brooker.co.za/blog/2012/01/17/two-random.html https://brooker.co.za/blog/2012/01/17/two-random.html
- francislavoie 4y agoCaddy has this with `lb_policy random_choose 2`. See https://caddyserver.com/docs/caddyfile/directives/reverse_proxy#load-balancing https://caddyserver.com/docs/caddyfile/directives/reverse_pr....
- goodpoint 4y agoCorrect! And there can be many failures where a server returns immediately but without an HTTP error.