3 ms·
Seems simpler to do this at the load balancer level. That way you can use the same strategy for any type of application.
by joevandyk 8y ago
Seems simpler to do this at the load balancer level. That way you can use the same strategy for any type of application.
- SteveNuts 8y agoWhat if your Go application IS a load balancer?
- joevandyk 8y agoThen you should use nginx or haproxy ;)
- ebikelaw 8y agoEven in that case you don't need graceful in-place restarts. If you have multiple replicas of the load balancer and ECMP from higher on your stack, all you need is for your balancer process to 1) remove itself as a destination for new flows; 2) stop accepting new connections and complete all existing connections; 3) shut down and restart; 4) add itself back to the set of new flow destinations. If you don't have multiple instances of your load balancer, that's weird.
- kevin_nisbet 8y agoI don't believe this approach would work for a layer 7 load balancer (where the connection is terminated on the host). This is in regards to the original article, which is discussing how to maintain socket state through a process upgrade. When you remove the host from the routing table, the ECMP algorithm will redistribute the traffic to the other hosts still in the pool. For TCP connections, this will generate reset's, and drop any active connections moved between hosts. Also, at times I've not gotten clear answers from router vendors on whether they support a consistent hash in their ECMP implementation, which means that connections through the load balancers not being reloaded may also be rebalanced to other hosts and be affected by the routing table change. Even if we consider going down the stack to a layer 4 load balancer, outside the scope of this article, it may still be desirable to combine approaches and offer hitless reloads (a common case) and not just recover from a node failure (a less common case). The reason is, that you would need state replication of the load balancing tables to support removing the load balancer from the routing tables, and having another load balancer take over it's work while maintaining the connections through the load balancer. This is under the assumption that the service being load balanced requires an uninterrupted connection of course. Disclaimer: I work with the author of the article and contributed to it.
- ebikelaw 8y agoI've worked with ECMP routers where you can inform that router you'd like new flows to be sent elsewhere while existing flows continue to be sent to their established endpoints. JunOS for example has this feature.
- AdamJacobMuller 8y agoDo you recall what option/platform that is? I assume that must have been something like an SRX, not an MX/QFX/EX.