There are quite a few proxies that can do this; Envoy's probably the most popular. You can also load-balance on the client side. Clients connect to a manageme

There are quite a few proxies that can do this; Envoy's probably the most popular. You can also load-balance on the client side. Clients connect to a management server, and the server tells them what percentage of connections to send to which upstream. I don't think anyone has written any publicly available software to manage this, but it can use the same underlying protocol as Envoy's configuration servers (which are also rare in public; I wrote one, but it sends clients Endpoints and Clusters and the gRPC implementation happens to want Listeners). The xDS implementation is a newer version of grpclb, which maybe had more support but I've never heard of anyone using it. (Google internally uses the client side load balancing. Worked great when I used it, so was always surprised it wasn't more popular in the real world.) The article is really negative about client-side load balancing, but you can't complain about the latency properties -- no proxy between you and the other end. I always wished the Internet standards did a better job here; you can serve multiple A records for a DNS lookup, and browsers kind of sort of try to load balance (by selecting one of the endpoints randomly), but it doesn't work too well if one of the backends is broken. If you could just supply a pre-defined health-check function and maybe select a simple balancing algorithm, HTTP would be just a little bit smoother for everyone. (If you put topology information in the DNS lookup, you could even balance geographically without having to run special DNS servers that serve different replies to different users. A lot of upstream DNS caching hacks could then go away. It would be great! People are onto the value of this; Envoy Mobile lets you serve this information to your mobile app, so it can easily survive regional failures in your server-side infrastructure. Probably not a lot of adoption, but something that browsers should start doing.)