5 ms·

Using consul (https://www.consul.io/ https://www.consul.io/) in conjunction with HAProxy is another great way for service discovery and routing.

imdsm 11y ago
Yep, Consul-template + HAProxy is great for a dynamic HAProxy solution.
- jolynch 11y ago
  Has anyone actually done this at scale and seen how it works? We've talked about it a few times because Consul looks really cool but it doesn't appear to have the full capabilities of SmartStack yet, in particular Synapse (https://github.com/airbnb/synapse https://github.com/airbnb/synapse) for intelligently doing stats socket updates, intelligent reloads, or controlling how HAProxy reloads over time. Also consul only has two levels of services right, "local" and "remote"? I'm not sure that is sufficiently powerful to describe SOA's which need to be able to differentiate services on the machine, rack, colo, datacenter, region locality? I suppose you could do what we do in SmartStack and just register the same service instance at a few different places since I think that consul can register arbitrary services. Maybe all we need is a consul watcher in Synapse, and a consul-template for the Synapse configs, and we can get the best of all worlds. With Linux 4.4 SYN handling in Linux might even be fixed by Eric Dumazet being super hardcore (e.g. https://lwn.net/Articles/659199/ https://lwn.net/Articles/659199/) and we won't even have to worry about reloading HAProxy.
  - awinder 11y ago
    Services in consul are datacenter aware and there's a tagging concept that you could utilize for something rack or region.
- magiconair 11y ago
  https://github.com/eBay/fabio https://github.com/eBay/fabio might also be an option for you. Disclosure: I'm the author.
  - sandstrom 11y ago
    Looks really interesting. Will try it out!
  - imdsm 11y ago
    That looks interesting. I'll check it out. Thanks!
- znep 11y ago
  As long as you don't have too many microservices... the polling and templating doesn't scale too well. This is related to https://github.com/hashicorp/consul-template/issues/165#issuecomment-70862099 https://github.com/hashicorp/consul-template/issues/165#issu... although that is even more extreme.
  - imdsm 11y ago
    Well it depends on how you implement it. I wouldn't imagine that each microservice has it's own haproxy-consul-template router. If it did, then sure, you're going to swamp it. Maybe I'm not understanding the situation correctly, and granted, I've not ran it on a cluster of more than 5 machines with more than 10 microservices as we decided to use Rancher instead.

imdsm 11y ago

Yep, Consul-template + HAProxy is great for a dynamic HAProxy solution.

jolynch 11y ago
Has anyone actually done this at scale and seen how it works? We've talked about it a few times because Consul looks really cool but it doesn't appear to have the full capabilities of SmartStack yet, in particular Synapse (https://github.com/airbnb/synapse https://github.com/airbnb/synapse) for intelligently doing stats socket updates, intelligent reloads, or controlling how HAProxy reloads over time. Also consul only has two levels of services right, "local" and "remote"? I'm not sure that is sufficiently powerful to describe SOA's which need to be able to differentiate services on the machine, rack, colo, datacenter, region locality? I suppose you could do what we do in SmartStack and just register the same service instance at a few different places since I think that consul can register arbitrary services. Maybe all we need is a consul watcher in Synapse, and a consul-template for the Synapse configs, and we can get the best of all worlds. With Linux 4.4 SYN handling in Linux might even be fixed by Eric Dumazet being super hardcore (e.g. https://lwn.net/Articles/659199/ https://lwn.net/Articles/659199/) and we won't even have to worry about reloading HAProxy.
- awinder 11y ago
  Services in consul are datacenter aware and there's a tagging concept that you could utilize for something rack or region.
magiconair 11y ago
https://github.com/eBay/fabio https://github.com/eBay/fabio might also be an option for you. Disclosure: I'm the author.
- sandstrom 11y ago
  Looks really interesting. Will try it out!
- imdsm 11y ago
  That looks interesting. I'll check it out. Thanks!
znep 11y ago
As long as you don't have too many microservices... the polling and templating doesn't scale too well. This is related to https://github.com/hashicorp/consul-template/issues/165#issuecomment-70862099 https://github.com/hashicorp/consul-template/issues/165#issu... although that is even more extreme.
- imdsm 11y ago
  Well it depends on how you implement it. I wouldn't imagine that each microservice has it's own haproxy-consul-template router. If it did, then sure, you're going to swamp it. Maybe I'm not understanding the situation correctly, and granted, I've not ran it on a cluster of more than 5 machines with more than 10 microservices as we decided to use Rancher instead.