This is similar to how we direct traffic to MySQL at Eventbrite. We also have multiple MySQL clusters with complex topologies for different services. Every serv

This is similar to how we direct traffic to MySQL at Eventbrite. We also have multiple MySQL clusters with complex topologies for different services. Every server that can access MySQL has a local HAProxy that it connects to. This local HAProxy is checking the health of a pool of dedicated HAProxy servers. Those HAProxy servers are hitting a script run by xinetd on the MySQL servers to determine whether a server should be in its pool. Here is where we differ the most from Github. Instead of having two backends per frontend, we only have one, but the health check itself is context-aware. In addition to takings its own lag into account, it also queries HAProxy to determine the number of healthy servers in its backend pool. As the percentage of healthy servers goes down, the threshold for declaring itself unhealthy based on lag goes up. Additionally, there are some extra safeguards to make sure we never dip below a certain number of servers in the pool. We use variations on this approach for other services like Solr and Redis as well. BTW, if managing this sort of infrastructure excites you, we're hiring! https://jobs.lever.co/eventbrite/cd7d6b95-0226-4e03-b9fa-c157f7920b6e https://jobs.lever.co/eventbrite/cd7d6b95-0226-4e03-b9fa-c15...