4 ms·
Load balancing as a strategy is used in far more than just web-applications! This article only discusses web-based load balancing, which is absolutely importan
by dragontamer 6y ago
Load balancing as a strategy is used in far more than just web-applications!
This article only discusses web-based load balancing, which is absolutely important, but doesn't discuss supercomputer scheduling load-balancing. Its arguably a different subject... but the concept is the same.
When you have 4000 nodes on a supercomputer, how do you distribute the problem such that all the nodes have something to do? Supercomputer problems are sometimes predictable (ie: matrix-multiplications), and you can sometimes "perfectly load balance without communication".
But in the case of web-applications, there's probably no way to really predict the "cost of performance" before you start processing the service request (what if its a Facebook request to a really old photograph? Facebook may have to pull it out of long-term storage before it can service that request. There's no real way to know at the load-balancer whether a picture-request would be in the cache or not... at least, not before you process the request to begin with!)
-----------
In any case, I think adding "Predict the computational cost, calculate the costs you distributed to different nodes, and then distribute the new load to the node with lowest computational cost given so far" is a good method that works in some applications. (All blocks in a dense matrix multiplication have the same cost, so just keep passing out subblocks to all nodes as you're working on the problem)
- saranshk 6y agoConcepts are definitely transferrable across domains. And then adapted according to desired outputs. Cost of performance is interesting and somewhat similar to the least-time approach but adapted to the supercomputers domain.
- snoshy 6y agoArguably, one of the most important characteristics of a load balancer is to have extremely low latency. If you're balancing loads, you want to be very quick about making a decision. When generating predictions about the computational cost, that can itself add in a computational cost that might result in a non-negligible amount of computational cost overhead. Inherently, the idea that you're talking about boils down to having a way to characterize the nature of the request flows in such a manner that they can be evenly distributed. The ideal way to characterize them then, would be to know this information beforehand such that it does not require any computation at all to normalize the costs. As such, the best strategy would be to actually segregate traffic flows such that they're forwarded to "dumb" load balancers that use one of the strategies from TFA like weighted round robin. Of course, there are many such optimizations available, but TFA seems to be targeting a beginner level introduction to a rather complex topic. As you describe, load balancing and scheduling algorithms have a pretty high overlap in terms of their theoretical foundations, and these concepts manifest themselves throughout any large scale system.
- toast0 6y ago> But in the case of web-applications, there's probably no way to really predict the "cost of performance" before you start processing the service request (what if its a Facebook request to a really old photograph? Facebook may have to pull it out of long-term storage before it can service that request. There's no real way to know at the load-balancer whether a picture-request would be in the cache or not... at least, not before you process the request to begin with!) I worked at Facebook, but didn't touch anything related to this; so this is all conjecture. The load balancer can't (or shouldn't) predict the cost to service a request; but the thing that generated the url for the image could; and that prediction could be passed in the url for the balancer to act on. If you really need to balance by performance, it's probably simpler and accurate enough to provide frequent load feedback to the balancer. As long as you have a lot of requests, simple things work pretty well.