4 ms·
This was a great and readable explanation! Also it was a great example as to why you should never use “random” as your load balancing algorithm unless you plan
by jedberg 7y ago
This was a great and readable explanation!
Also it was a great example as to why you should never use “random” as your load balancing algorithm unless you plan to always have 1/3 extra capacity.
Or conversely why you should always have 1/3 extra capacity if you must use random.
- fyp 7y agoI thought the example in the article was a little artificial at first. Like why would you only have 12 requests if you have 12 backends? Reframing it in terms of capacity cleared that up. If the rate of incoming requests is higher than the total rate your backends can process, your queues will grow infinitely! So something like an average of 12 incoming requests per second with each backend capable of processing 1 request per second is actually fairly realistic. And I think the math still works out the same there.
- skizm 7y agoThat condition is only met if you send N requests to N routers. If you send 1,000,000*N requests to N routers, they will almost always be evenly distributed.
- jedberg 7y agoBut then you’re under capacity. The assumption is that it takes N servers to service N requests simultaneously.