3 ms·

Folks should take a look at Baker Street (bakerstreet.io) which is a HAProxy-based system for routing traffic between microservices. Baker Street is designed to

Folks should take a look at Baker Street (bakerstreet.io) which is a HAProxy-based system for routing traffic between microservices. Baker Street is designed to be really easy to get up and running (the install is super quick) and is based on the Yelp popularized SmartStack. However, one of the beautiful aspects of Baker Street is that unlike SmartStack it does not require running and maintaining a ZooKeeper cluster which can be a PITA for teams just starting to work with microservices. Full disclosure: I am the lead developer of Baker Street :) Website: http://bakerstreet.io http://bakerstreet.io Source: https://github.com/datawire/bakerstreet https://github.com/datawire/bakerstreet

toomuchtodo 11y ago
Hey there! Just a small nitpick: Running Zookeeper is insanely simple. We deployed it in 2-3 days, and it has had literally 0 issues over the last year. I highly recommend https://github.com/mbabineau/docker-zk-exhibitor https://github.com/mbabineau/docker-zk-exhibitor to get started (Docker container that wraps together Zookeepher and Netflix's Exhibitor manager for it). Remember, pick boring technologies that just work.
- meddlepal 11y ago
  Thanks for the response! I agree that ZooKeeper is not exceptionally difficult to run, but it does bring in some additional overhead and complexity for a development team that is just trying to get started with microservices, for example sorting out the deployment and quorum bootstrapping. For a team that might not have an experienced systems or infrastructure engineer it could become a nightmare. By bringing in Exhibitor you've added even more initial complexity to the deployment just so you can get started gluing services together. That's what we're primarily trying to solve with Baker Street at this time: super simple deployment and usage; minutes versus days.
- felixgallo 11y ago
  zookeeper is not a boring technology that just works: http://blog.cloudera.com/blog/2014/03/zookeeper-resilience-at-pinterest/ http://blog.cloudera.com/blog/2014/03/zookeeper-resilience-a... http://arstechnica.com/information-technology/2015/05/the-discovery-of-apache-zookeepers-poison-packet/ http://arstechnica.com/information-technology/2015/05/the-di... https://tech.knewton.com/blog/2014/12/eureka-shouldnt-use-zookeeper-service-discovery/ https://tech.knewton.com/blog/2014/12/eureka-shouldnt-use-zo... Here's ibm and netflix with some advice about how non-boring running a ZK cluster can be: https://www-01.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.pd.doc/doc/containerstreamszookeeper.html https://www-01.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/... http://techblog.netflix.com/2011/11/introducing-curator-netflix-zookeeper.html http://techblog.netflix.com/2011/11/introducing-curator-netf...
sandstrom 11y ago
Looks neat! Any plans on adding support for Consul as a directory/backend? (understand if you'd hold the same arguments against it as for ZooKeeper)
- meddlepal 11y ago
  Thanks! We have had very abstract conversations about adding alternative back ends, but presently we are not working on support for that feature because as you correctly noted, at least for Consul and ZooKeeper it is somewhat against the operating philosophy. We're not opposed to it though and if you open a feature request we'll track interest which will help us decide direction. Baker Street is not a secondary component for some other system in our company. We consider Baker Street a core product even at this early stage in its development.
vidarh 11y ago
This, from your page, is worth highlighting: > because we wanted a simple, highly available service, not a strongly consistent one. I absolutely agree with this design choice. This is the big flaw for me with how a lot of setups use Etcd, Consul, Zookeeper and the like. A lot of the time they're being used because we need high-availability, not at all for their consistency guarantees, and their consistency guarantees has great potential to reduce the availability. I've not run Zookeeper in production, but both with Etcd and Consul I've had situations where the clusters have gotten into totally broken states because the cluster membership has changed in bad ways. It makes sense for the guarantees they are providing. It does not make sense if you don't need the consistency guarantees. The way I see it, lost service registrations are a non-issue as you generally want to continuously renew them combined with a TTL to avoid stale registrations, so the occasionally lost one due to e.g. a part of the service directory cluster failing at a bad time is no big deal. Conversely, if a service stays in the service directory when it shouldn't, the worst that happens is that you run into an unhealthy service when you try to issue requests, but that will happen during normal operation while using services with consistency guarantees anyway, since the service may have failed between health checks anyway, which means you still need to handle retries. I'd rather use a service-discovery solution with fairly significant consistency issues that keeps replying "no matter what" than one which focuses on consistency but may stop answering.

toomuchtodo 11y ago

Hey there! Just a small nitpick: Running Zookeeper is insanely simple. We deployed it in 2-3 days, and it has had literally 0 issues over the last year. I highly recommend https://github.com/mbabineau/docker-zk-exhibitor https://github.com/mbabineau/docker-zk-exhibitor to get started (Docker container that wraps together Zookeepher and Netflix's Exhibitor manager for it). Remember, pick boring technologies that just work.

meddlepal 11y ago
Thanks for the response! I agree that ZooKeeper is not exceptionally difficult to run, but it does bring in some additional overhead and complexity for a development team that is just trying to get started with microservices, for example sorting out the deployment and quorum bootstrapping. For a team that might not have an experienced systems or infrastructure engineer it could become a nightmare. By bringing in Exhibitor you've added even more initial complexity to the deployment just so you can get started gluing services together. That's what we're primarily trying to solve with Baker Street at this time: super simple deployment and usage; minutes versus days.
felixgallo 11y ago
zookeeper is not a boring technology that just works: http://blog.cloudera.com/blog/2014/03/zookeeper-resilience-at-pinterest/ http://blog.cloudera.com/blog/2014/03/zookeeper-resilience-a... http://arstechnica.com/information-technology/2015/05/the-discovery-of-apache-zookeepers-poison-packet/ http://arstechnica.com/information-technology/2015/05/the-di... https://tech.knewton.com/blog/2014/12/eureka-shouldnt-use-zookeeper-service-discovery/ https://tech.knewton.com/blog/2014/12/eureka-shouldnt-use-zo... Here's ibm and netflix with some advice about how non-boring running a ZK cluster can be: https://www-01.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.pd.doc/doc/containerstreamszookeeper.html https://www-01.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/... http://techblog.netflix.com/2011/11/introducing-curator-netflix-zookeeper.html http://techblog.netflix.com/2011/11/introducing-curator-netf...

sandstrom 11y ago

Looks neat! Any plans on adding support for Consul as a directory/backend? (understand if you'd hold the same arguments against it as for ZooKeeper)

meddlepal 11y ago
Thanks! We have had very abstract conversations about adding alternative back ends, but presently we are not working on support for that feature because as you correctly noted, at least for Consul and ZooKeeper it is somewhat against the operating philosophy. We're not opposed to it though and if you open a feature request we'll track interest which will help us decide direction. Baker Street is not a secondary component for some other system in our company. We consider Baker Street a core product even at this early stage in its development.

vidarh 11y ago

This, from your page, is worth highlighting: > because we wanted a simple, highly available service, not a strongly consistent one. I absolutely agree with this design choice. This is the big flaw for me with how a lot of setups use Etcd, Consul, Zookeeper and the like. A lot of the time they're being used because we need high-availability, not at all for their consistency guarantees, and their consistency guarantees has great potential to reduce the availability. I've not run Zookeeper in production, but both with Etcd and Consul I've had situations where the clusters have gotten into totally broken states because the cluster membership has changed in bad ways. It makes sense for the guarantees they are providing. It does not make sense if you don't need the consistency guarantees. The way I see it, lost service registrations are a non-issue as you generally want to continuously renew them combined with a TTL to avoid stale registrations, so the occasionally lost one due to e.g. a part of the service directory cluster failing at a bad time is no big deal. Conversely, if a service stays in the service directory when it shouldn't, the worst that happens is that you run into an unhealthy service when you try to issue requests, but that will happen during normal operation while using services with consistency guarantees anyway, since the service may have failed between health checks anyway, which means you still need to handle retries. I'd rather use a service-discovery solution with fairly significant consistency issues that keeps replying "no matter what" than one which focuses on consistency but may stop answering.