I personally prefer running a large number of logically separate HAProxy instances (a pair for each service). This makes it easier to forecast / tune request vo

I personally prefer running a large number of logically separate HAProxy instances (a pair for each service). This makes it easier to forecast / tune request volume of the proxy servers on a per-service level, and it limits the impact that reconfiguration of one service can have on another (which helps immensely in root-cause analysis of production incidents once you've scaled beyond one team). It makes firewall rules a pain, but really no more painful than the general orchestration / configuration pain you feel at this scale. The good part about HAProxy is that it's flexible and can be configured any number of ways, but I've found that flexibility can weigh some teams by using it to solve problems that it's really not designed for. Sometimes it's better to use a brute-force approach with a simple configuration and deal with the complexity in a tool better designed for complexity like Mesos.