3 ms·
Cool read! Always fun to think about this stuff. > The official implementation uses a Per Call Load Balancing and not a per connection one. So each call is be
by dmoy 5y ago
Cool read! Always fun to think about this stuff.
> The official implementation uses a Per Call Load Balancing and not a per connection one. So each call is being load balanced separately. This is the ideal and the desired case and it will avoid having heavy sticky connections.
I wonder how much of this is just leaking from how stubby & GSLB is done internally at Google. I'll admit, I don't know much about gRPC.
Separate point: if you control both the clients and the servers completely, you can have servers respond with load shedding signals if they start to get overloaded, and use that as a hint for the client to go look elsewhere. This doesn't solve all problems, but it does make some cases easier to handle. Also it is obviously not a viable solution if you don't fully control all clients - some of them will just not respect your load shedding and continue to beat an individual server into the ground.
- Sparkyte 5y agoThe trick would be to use the LB as an initial session starter. Don't use LB metrics to scale, just manage what hosts are available for a session.
- alpb 5y agoI am not even sure if the problem described in the article is specific to gRPC. I think it's a HTTP/2 problem. Any sane HTTP/2 client making N queries to the same host/IP ideally should use 1 connection since h2 supports multiplexing (which is bad for load balancing if you have an L4 LB). So even if you don't use gRPC, consider putting a per-call (L7) LB in front of your app if you use HTTP/2 protocol between your apps.