3 ms·
All things which have too few bins under random are at risk of asymmetry. Some LB situations take some approximation to load/responsiveness, and back off the co
by ggm 3y ago
All things which have too few bins under random are at risk of asymmetry. Some LB situations take some approximation to load/responsiveness, and back off the counts as a function of observed pace of response. TCP RTT will do, size of the pending Queue for async responses too, time in queue, there's lots of measures. Your liveness check to the backends probably provides this simply by how responsive to heartbeat it is.
Given tasks do not run in identical time, there's going to be variance come what may.
I would suggest a pool of 3 should tend to be more stable. But, your platform people may be recommending that 3 and below need to be sized to work as 1, only 4 and above can start to assume a minimum cohort (2?) exist.
Sharding is not LB. It's just a hash approximation to grouping things. Maybe you're telling a story about using a random() to emulate how far out of 50/50 a 2 part sharding story can get?
Routers doing LB may well do simple prefix triage. You may be stuck on one side of the LB because of who you are, and nothings going to change it.
(I don't do this for a living, wiser people may tell you I'm wrong)
- beardedwizard 3y agoTypically network LB will look at the entire layer 4 tuple to overcome things like NAT clustering traffic from many users into one place. Not to say IP lb doesn't happen, but with NAT being de-facto almost nobody does it IME.
- lazide 3y agoIt sounds like part of the issue is that people think random == evenly (but unpredictably) distributed on a specific (and relatively short) timescale. For a truly random source, the odds of getting the same number in the next roll are identical to getting it on any prior roll. For example, flipping a coin twice is as likely to get you heads twice as it is to get you heads/tails. eventually things even out. But if it’s truly random, when that is going to be the case should also be completely unpredictable. It might end up being after the service is deprecated, for instance. You might end up never getting a specific ‘desired’ answer ever. For instance, if the test is flipping a coin and we do it 10 times, there is a small chance we never get tails and get 10 heads in a row from a truly random source. That is working as expected. Usually (at least with load balancing) they instead want a smoother distribution, where it’s less likely to get the same answers they just got, when they just got them. So for example, they’d prefer that if they just got a heads they’re more likely to get a tails next (but it won’t always happen). Notably, this isn’t really random. It’s also a really hard thing to do in practice without also getting all nodes in lockstep, being predictable to an outside attacker, or having internode communication. Maybe impossible, I don’t remember.