4 ms·

> I've never understood using round-robin or random algorithms for load-balancers... LeastConns/FastestConn selection is very dangerous when a backend host fai

> I've never understood using round-robin or random algorithms for load-balancers... LeastConns/FastestConn selection is very dangerous when a backend host fails. Imagine a host has a partial failure, allowing health checks to pass. This host now fast fails and returns a 500 faster than other hosts in the pool generate 200s. This poison host will have less active connections and your LB will route more requests to it. A single host failure just became a major outage. I like WRR selection for back ends, then use a queue or fast fail when your max active conns is exceeded. Humans prefer latency to errors, so let the lb queue (to a limit) on human centric VIPs. Automated clients deal better with errors so have your lb throw a 500 directly, or RST, or direct to a pool that serves static content.

shabble 14y ago
You'd have some sort of error threshold/rate limit[1] at which point the server is marked dead and falls out of the backends list, surely? Or even, an alarm threshold if responses are averaging /too fast/, based on your expected load & response times. I've not done any deployment/ops beyond hte trivial/theoretical though, so I don't know how this would work in reality.
- donavanm 14y ago
  No, lbs don't inspect established streams. The lb will periodically send requests to a known uri as a health heck instead. The problem is when the health check uri isn't indicative of availabilty. (Hint: it never is) Nope, don't do this either. Unless you like getting pages because things are working?
  - jeremyjh 14y ago
    You can configure a LB to inspect layer 7 - in HAProxy this is done with an observe directive. Error 500s would then take the member out of a pool. You are right that the health check of a static file may put it right back into the pool, but you can give it a slow rise value so that it waits a few minutes to do that. I'm not saying this is easy to get right but it is definitely possible to at least reduce the frequency of selection of a server raising 500 errors.
    - donavanm 14y ago
      Yes, all things are possible. You'll also have to keep state and track rates. Otherwise a very low error rate could down all of your backend hosts. But you're now running a stateful l7 application proxy. That's waaaaaay more expensive than a tcp proxy with asynchronous checks.

shabble 14y ago

You'd have some sort of error threshold/rate limit[1] at which point the server is marked dead and falls out of the backends list, surely? Or even, an alarm threshold if responses are averaging /too fast/, based on your expected load & response times. I've not done any deployment/ops beyond hte trivial/theoretical though, so I don't know how this would work in reality.

donavanm 14y ago
No, lbs don't inspect established streams. The lb will periodically send requests to a known uri as a health heck instead. The problem is when the health check uri isn't indicative of availabilty. (Hint: it never is) Nope, don't do this either. Unless you like getting pages because things are working?
- jeremyjh 14y ago
  You can configure a LB to inspect layer 7 - in HAProxy this is done with an observe directive. Error 500s would then take the member out of a pool. You are right that the health check of a static file may put it right back into the pool, but you can give it a slow rise value so that it waits a few minutes to do that. I'm not saying this is easy to get right but it is definitely possible to at least reduce the frequency of selection of a server raising 500 errors.
  - donavanm 14y ago
    Yes, all things are possible. You'll also have to keep state and track rates. Otherwise a very low error rate could down all of your backend hosts. But you're now running a stateful l7 application proxy. That's waaaaaay more expensive than a tcp proxy with asynchronous checks.