5 ms·

AWS Application Load Balancers seem to default to round-robin, would be interested to hear how many people change this to their "least connections" equivalent c

AWS Application Load Balancers seem to default to round-robin, would be interested to hear how many people change this to their "least connections" equivalent called "LOR" [1]? And why they don't support any other options? I'd guess AWS default to round robin because it's the least complex, and doesn't have issues like if a worker responds extremely quickly due to an error (eg. returns a 500 immediately) then using LOR it seems it would consume all the requests until it's taken out of service by any health checks. But maybe there are other potential downsides? [1] - https://aws.amazon.com/about-aws/whats-new/2019/11/application-load-balancer-now-supports-least-outstanding-requests-algorithm-for-load-balancing-requests/ https://aws.amazon.com/about-aws/whats-new/2019/11/applicati...

AdamN 4y ago
You point out a good gotcha that most people don't notice - many host-specific errors are often faster than the standard response and more traffic will route to those hosts with a performance-based heuristic. A well configured LB would have hosts with responses that take them out of rotation for known fault conditions - but at scale that's hard for companies to validate.
- zamnos 4y ago
  At scale, with also having scaled systems engineers, it's not, like, impossibly hard. A sidecar like Envoy can be configured to emit health stats which can then be read by the load balancer to consider a given server unhealthy. Again, at scale, but each team is already responsible for a dashboard with health metrics for their service, so the load balancer team doesn't have to try and determine everybodies health metrics, only their own.
base 4y ago
Depending on the app might not have better performance. The reason is that a host with more connections might not be using more resources than one with less connections. The busy connections might be waiting for a reply from an external service.
zamnos 4y ago
Conspiratorially speaking, the way to handle AWS' load balancer algorithm shortcomings is to run more EC2 instances, which, naturally AWS profits from.

AdamN 4y ago

You point out a good gotcha that most people don't notice - many host-specific errors are often faster than the standard response and more traffic will route to those hosts with a performance-based heuristic. A well configured LB would have hosts with responses that take them out of rotation for known fault conditions - but at scale that's hard for companies to validate.

zamnos 4y ago
At scale, with also having scaled systems engineers, it's not, like, impossibly hard. A sidecar like Envoy can be configured to emit health stats which can then be read by the load balancer to consider a given server unhealthy. Again, at scale, but each team is already responsible for a dashboard with health metrics for their service, so the load balancer team doesn't have to try and determine everybodies health metrics, only their own.

base 4y ago

Depending on the app might not have better performance. The reason is that a host with more connections might not be using more resources than one with less connections. The busy connections might be waiting for a reply from an external service.

zamnos 4y ago

Conspiratorially speaking, the way to handle AWS' load balancer algorithm shortcomings is to run more EC2 instances, which, naturally AWS profits from.