9 ms·

Zero Downtime Frontend Deploys with Vulcand on CoreOS

88 points by robszumski 12y ago

bkirkbri 12y ago
What great timing! I was just about to roll my own using etcd, haproxy and a bunch of perl duct tape. Very excited to take a good look at vulcand.
- robszumski 12y ago
  If you're still interested in doing this for any reason, check out confd [0] for populating templates based on etcd values. [0] https://github.com/kelseyhightower/confd https://github.com/kelseyhightower/confd
  - bkirkbri 12y ago
    Yes, confd was on my radar and looks promising. In my case, we've already got a wad of perl duct-tape out there and adding etcd was a baby step. Using confd would have been a later step. Vulcand will likely allow me to replace the whole shebang in one step.
    - bketelsen 12y ago
      it doesn't look like vulcan supports HTTPS, so it's a no-go for anything serious. Here's a post on my confd/nginx/docker setup for zero-downtime using etcd. http://brianketelsen.com/2014/02/25/using-nginx-confd-and-docker-for-zero-downtime-web-updates/ http://brianketelsen.com/2014/02/25/using-nginx-confd-and-do...
      - bkirkbri 12y ago
        I actually learned about confd via your post. Thanks for that. In our case, HTTPS isn't an issue but there are good arguments for using a tried-and-true proxy. I see Vulcan as a quick way to test out docker containers for our architecture. In the end I'll probably use confd to rewrite a VCL file for Varnish. It's just a matter of deciding how many containers and what level of sidekicking/ambassadoring I'm willing to put up with to "properly" organize things...
      - alexk 12y ago
        Yep, until this won't be closed: https://github.com/mailgun/vulcan/issues/31 https://github.com/mailgun/vulcan/issues/31 Vulcand should not be used as for anything frontend-related
bjt 12y ago
etcd, fleet, deis, and now this are getting me more and more tempted to move from Velociraptor to the Docker ecosystem. If Deis could run my uptests[0] before Vulcan gets called to re-route things, that'd be awesome. https://bitbucket.org/yougov/velociraptor/src/2bcef54253a9e9238aee92de92b6d088a5d33c60/docs/uptests.rst https://bitbucket.org/yougov/velociraptor/src/2bcef54253a9e9...
- gabrtv 12y ago
  Come find us in #deis on Freenode and we'll see if we can help.
gabrtv 12y ago
Top notch technical writeup. We've had our eyes on Vulcand for some time now, though we'd really like to see native TCP support.
- alexk 12y ago
  This is in pipeline! TCP would be next step once I make sure the existing feature set is production ready and robust.
jksmith 12y ago
This is a golang project, no?
- alexk 12y ago
  yep
moe 12y ago
The architecture is sound. But half-assing a loadbalancer impl (vulcand) instead of embedding HAProxy is a terrible idea.
- oddurmagg 12y ago
  +1 confd + haproxy would be a better choice. I have plans do roll the cocktail out, but running the HA prox y on each machine, much like the airbnb smartstack: http://nerds.airbnb.com/smartstack-service-discovery-cloud/ http://nerds.airbnb.com/smartstack-service-discovery-cloud/
- jksmith 12y ago
  So it's half assed if it's not HAProxy? How so?
  - moe 12y ago
    HAProxy is 14 years old, under active development by very competent people, used in thousands of production deployments and many hardware appliances. Pretty much every large site you visit every day has HAProxy in the stack. It's one of the cornerstones of OSS infrastructure software. Are you seriously asking how a 2 month old hack (vulcand) is half-assed in comparison?
    - jksmith 12y ago
      If you're going to call something half-assed because it's not HAProxy then I for one would consider it more productive if this was framed in terms of metrics instead of something like "half-assed." Does HAProxy have a suite of metrics that can serve as benchmarks for all the projects out there that would dare to be less half-assed?
      - felixgallo 12y ago
        Why didn't you ask the question the other way around -- does vulcand have anything to show that it is better in any way than the current best-practice load balancer? Reading the docs, the only thing that stands out is that it's tightly coupled with etcd, which seems like a really bad idea architecturally.
mahmoudimus 12y ago
We have something very similar at Balanced called Rump (routing umpire). It's written in python has a pretty cool rule based engine and a swappable backend. It currently uses ZK and Chef, but it would be pretty interesting to use something like Etcd/Consul since no one wants to maintain a ZK cluster. If there are individuals who want to hack on it, shoot me a msg on twitter and I'll see what I can do to speed up the process to open source it.
jbrechtel 12y ago
I'll ask about the reasoning instead of attacking the decision (since that didn't get a response). Why not embed HAProxy?

88 points by robszumski 12y ago

What great timing! I was just about to roll my own using etcd, haproxy and a bunch of perl duct tape. Very excited to take a good look at vulcand.

robszumski 12y ago
If you're still interested in doing this for any reason, check out confd [0] for populating templates based on etcd values. [0] https://github.com/kelseyhightower/confd https://github.com/kelseyhightower/confd
- bkirkbri 12y ago
  Yes, confd was on my radar and looks promising. In my case, we've already got a wad of perl duct-tape out there and adding etcd was a baby step. Using confd would have been a later step. Vulcand will likely allow me to replace the whole shebang in one step.
  - bketelsen 12y ago
    it doesn't look like vulcan supports HTTPS, so it's a no-go for anything serious. Here's a post on my confd/nginx/docker setup for zero-downtime using etcd. http://brianketelsen.com/2014/02/25/using-nginx-confd-and-docker-for-zero-downtime-web-updates/ http://brianketelsen.com/2014/02/25/using-nginx-confd-and-do...
    - bkirkbri 12y ago
      I actually learned about confd via your post. Thanks for that. In our case, HTTPS isn't an issue but there are good arguments for using a tried-and-true proxy. I see Vulcan as a quick way to test out docker containers for our architecture. In the end I'll probably use confd to rewrite a VCL file for Varnish. It's just a matter of deciding how many containers and what level of sidekicking/ambassadoring I'm willing to put up with to "properly" organize things...
    - alexk 12y ago
      Yep, until this won't be closed: https://github.com/mailgun/vulcan/issues/31 https://github.com/mailgun/vulcan/issues/31 Vulcand should not be used as for anything frontend-related

bjt 12y ago

etcd, fleet, deis, and now this are getting me more and more tempted to move from Velociraptor to the Docker ecosystem. If Deis could run my uptests[0] before Vulcan gets called to re-route things, that'd be awesome. https://bitbucket.org/yougov/velociraptor/src/2bcef54253a9e9238aee92de92b6d088a5d33c60/docs/uptests.rst https://bitbucket.org/yougov/velociraptor/src/2bcef54253a9e9...

gabrtv 12y ago
Come find us in #deis on Freenode and we'll see if we can help.

gabrtv 12y ago

Top notch technical writeup. We've had our eyes on Vulcand for some time now, though we'd really like to see native TCP support.

alexk 12y ago
This is in pipeline! TCP would be next step once I make sure the existing feature set is production ready and robust.

jksmith 12y ago

This is a golang project, no?

alexk 12y ago
yep

moe 12y ago

The architecture is sound. But half-assing a loadbalancer impl (vulcand) instead of embedding HAProxy is a terrible idea.

oddurmagg 12y ago
+1 confd + haproxy would be a better choice. I have plans do roll the cocktail out, but running the HA prox y on each machine, much like the airbnb smartstack: http://nerds.airbnb.com/smartstack-service-discovery-cloud/ http://nerds.airbnb.com/smartstack-service-discovery-cloud/
jksmith 12y ago
So it's half assed if it's not HAProxy? How so?
- moe 12y ago
  HAProxy is 14 years old, under active development by very competent people, used in thousands of production deployments and many hardware appliances. Pretty much every large site you visit every day has HAProxy in the stack. It's one of the cornerstones of OSS infrastructure software. Are you seriously asking how a 2 month old hack (vulcand) is half-assed in comparison?
  - jksmith 12y ago
    If you're going to call something half-assed because it's not HAProxy then I for one would consider it more productive if this was framed in terms of metrics instead of something like "half-assed." Does HAProxy have a suite of metrics that can serve as benchmarks for all the projects out there that would dare to be less half-assed?
    - felixgallo 12y ago
      Why didn't you ask the question the other way around -- does vulcand have anything to show that it is better in any way than the current best-practice load balancer? Reading the docs, the only thing that stands out is that it's tightly coupled with etcd, which seems like a really bad idea architecturally.

mahmoudimus 12y ago

We have something very similar at Balanced called Rump (routing umpire). It's written in python has a pretty cool rule based engine and a swappable backend. It currently uses ZK and Chef, but it would be pretty interesting to use something like Etcd/Consul since no one wants to maintain a ZK cluster. If there are individuals who want to hack on it, shoot me a msg on twitter and I'll see what I can do to speed up the process to open source it.

jbrechtel 12y ago

I'll ask about the reasoning instead of attacking the decision (since that didn't get a response). Why not embed HAProxy?