9 ms·
Consul 1.0 Released
- neduma 9y agoChangelog - https://github.com/hashicorp/consul/blob/v1.0.0/CHANGELOG.md https://github.com/hashicorp/consul/blob/v1.0.0/CHANGELOG.md
- xatnys 9y agoThis is only tangentially related but is anyone using Spring Cloud + Consul? Is the integration of the two in a good spot now?
- t1o5 9y agoWe are a Java shop and use Consul (as a docker container) for our service discovery. Spring Cloud Consul plays well with Consul. The REST APIs are what makes Consul stand out. We have an in-house monitoring service querying consul apis for service removals & healthy service counts and send an alert if there are no more healthy services, so that our docker can spin up one just in time. Another feature that we use is the KV configuration dynamic reloading. So all your yml configurations are in Consul and can be updated and propagated to all of your services on the fly. Now the problem we face is Spring Cloud Consul project is maintained by a very few people, so if there is an issue, we have to workaround than waiting for a fix.
- tombert 9y agoI use Consul at work, and I do like it, but I have to ask a kind of obvious question; why would I use Consul over something that's more industry-standard like Zookeeper? I'm not asking this passive aggressively, I mostly want to know what features it adds over a vanilla Zookeeper setup.
- nunez 9y agoConsul is like ZK but specifically tailored to maintaining the state of infrastructure and application services. It removes a lot of the work required to do this on top of ZK.
- jake-low 9y agoI found this article [0] a couple weeks ago and found it informative. Basically Consul has service discovery built in, while Zookeeper et al. expect you to build service discovery on top of the primitives they provide (assuming you need it). [0]: https://www.consul.io/intro/vs/zookeeper.html https://www.consul.io/intro/vs/zookeeper.html
- kevinconaway 9y agoAlthough its not part of Zookeeper core, there is an Apache Curator module[1] for doing service discovery on Zookeeper [1] https://curator.apache.org/curator-x-discovery/ https://curator.apache.org/curator-x-discovery/
- takeda 9y agoThis article is biased because it was written by Consul author. I'm big fan of Zookeeper, and agree with parent that is not only industry standard, but it is very robust and battle tested. Having said that, this is not what Zookeeper was meant to. ZK is coordination service, if you have a distributed service that you need to coordinate and you don't want any single point of failure ZK is the choice. ZK can be used for service discovery, but that's not the proper use, you could equally set up a single database node, or MongoDB cluster to accomplish essentially the same thing. So specialized service is better, but I also don't like how Consul solves service discovery. If you think about it, SD doesn't require strong consistency, it is not a big deal if all nodes won't learn about each other at the same time, arguably that's actually preferred. By removing strong consistency (the only reason why ZK is even compared to Consul) we could have a better SD that's also more scalable.
- scarface74 9y agoIf you think about it, SD doesn't require strong consistency, it is not a big deal if all nodes won't learn about each other at the same time, arguably that's actually preferred. Can you go into more detail about this? Each computer has its own Consul agent that monitors the health of services running on it. You also get configuration from the agent on the individual computer. All of the agents communicate peer to peer and with the server don’t they? Isn’t that eventually consistent? I’m just starting to use Consul for configuration and haven’t started working on the service discovery part yet.
- scarface74 9y agoI use it because of the built in service discovery and the built in integration with Hashicorp’s other tools that I use - Nomad and Vault.
- jorge-fundido 9y agoLack of gossip in zookeeper makes changes in cluster topology much more painful.
- poooogles 9y agoZK 3.5 brings this.
- didip 9y agoThrough most of my careers, ZK cause me so much grief. So I welcome a competition in this space.
- takeda 9y agoIf ZK caused a lot of grief then you must be doing something horribly wrong with me. From my experience ZK is one of the tools that's most ops friendly (i.e. you set it and forget it), right next to Postgres and Solr.
- snuxoll 9y ago> right next to Postgres and Solr. Never used Solr, but even as someone who loves Pg over every other RDBMS out there I'd hardly call it "ops friendly", with the Bring-Your-Own-Tooling mentality it turns a lot of operations people off when they realize features like replication, log archiving, etc. require bolt-on tools to actually do anything useful. ZK is set and forget though, outside of occasional tweaking of JVM parameters based on growth.
- takeda 9y agoI'm not sure what you mean (I suspect that perhaps that was the way before Postgres 9?), my current company exists since times when MySQL was ruling the IT world, so it is heavily MySQL based. Since about a year ago it was accepted to also introduce PostgreSQL. We currently have 9.6 with barman for streaming backups and repmgr to control replication. Compared to any of our MySQL setups is a breeze to manage, my team is owning both and the bad thing is that there was only one issue so far[1] on call (bad, because if system doesn't break often no one will learn about it). PostgreSQL + barman + repmgr is much nicer than anything else we have for MySQL. Few things: barman offers streaming backup, meaning it is not just once a day, but it is done continuously. On top of that contains a single command that can verify it operates correctly (it has enough backups, streaming operation works etc). I also like that everything is robust. In MySQL if replication breaks, I need to manually restart it, if it gets too far behind (too many changes, or it is not fixed fast enough) the wal logs will be erased and I would need to set it up from the scratch again[2]. Postgres is smart enough to keep track of all wal logs and only deletes them if all standby nodes received them. If connection is broken (for example a network partition) it will reconnect and resume replication. It self heals whenever it cans. It is a breeze to manage. Another thing MySQL lost a lot of points for me was at my previous workplace. An MySQL host suddenly started throwing errors to clients. Turns out that few tables got corrupted, the fixing was easy, just shutting down and running repair tool. What was puzzling is that there was no obvious reason (the disk had plenty of free space, no corruption) the machine was not abruptly terminates (~100 days uptime) I never found why data got corrupted. Also, I never experienced things like that with Postgres. [1] the issue was that barman host had a bad disk, so that triggered an alert and after disk replacement things continued to work. [2] there is also one crazy system that MySQL allowed to happen. Essentially in master-standby replication someone had genius idea to do some processing on the data in master and write it in standby. Over time that data became also important, so actually the standby node needed to be backed up instead of the master. This is "great" because if replication breaks, whomever is on call will have fun time doing some acrobatics to restore the whole thing. Postgres correctly doesn't allow writes to standby during replication.
- shmish111 9y agoIt's a lot easier to take care of than zookeeper for one thing. Also Consul is getting pretty close to industry standard these days for newer businesses.
- ah- 9y agoI can confirm this. Consul is much easier to run in production without issues. Zookeeper is rather picky around disk responsiveness etc. And the Consul API is rather nice.
- schmichael 9y agoZookeeper and Consul have very different scopes and therefore architectures. Consul servers are very much like Zookeeper: an odd number of nodes running a consensus protocol to provide a consistent API to a key/value store. However, Consul builds tons more on top of the servers. Consul has a first class notion of services and health checks. It has agents that run on every node to register services, perform health checks, and provide service discovery to local apps via HTTP or DNS. For every Consul feature there is probably a similar library or tool to do the same thing with Zookeeper. Consul just chose to focus on the service discovery problem and address it as a first class feature of the project. Full disclosure: I work for HashiCorp, but not on Consul.
- corford 9y agoFor me it's the ease of use. One simple go binary, all the features you need (yet manages to feel lightweight), well documented and rock solid.
- skywhopper 9y agoI don't know much about Zookeeper, having not used it personally, but I do manage a large Consul installation as part of my Ops role for for a large cloud-based SaaS provider. Not knowing anything about differences in functionality, one thing that stands out to me is platform differences. If you're in a Java shop and have all the tools and expertise for managing Java applications, then Zookeeper will fit right in. If you _aren't_ a Java shop, then that's an additional layer of complexity you will be forced to manage. So for the most part, we aren't a Java shop at my job, so Consul's native binary is very appealing because it's simple and straightforward. Consider that the Consul binary distribution is a zip file containing a single file. Compare that to Zookeeper's tarball which contains 1538 files. That maybe isn't a fair point on which to make a comparison, but I think it's an important signal about their respective approaches. Various people might have different reactions to those numbers, so I'm not trying to pass judgment here. Just giving an idea of one way to think about differences.
- kod 9y agoWhat additonal layer of complexity are you talking about? Installing a jvm?
- vesak 9y agoZookeeper is industry standard? I guess that explains why it's such a mess.
- DeathRabbit 9y agoFrom the standpoint of CAP theorem, Consul is AP and ZK is CP. AP and CP distributed systems have their use cases, so it's important to think them through up-front. For example, I personally consider service discovery to favor AP. I arrived at that personal assertion by building a SD setup using our conventions + Curator + ZK. In disaster scenarios we forced into the system when testing, I was able to get ZK to basically lock up because, really, that's what it's supposed to do, guarantee consistency--if it can't guarantee consistency, then it tells you about it. Consul for us fit the bill better. Note: This was some time ago and I do not claim to be a ZK expert; it's quite possible the root issue was me. YMMV, etc etc.
- Chyzwar 9y agoHCL - why inventing another config/markup language? There TOML, yaml and dozen others.
- nunez 9y agoI think HCL strikes a balance between being too generic (YAML, JSON) and too flexible (Chef, Puppet code) and seems to be better suited for infrastructure automation.
- ykler 9y agoYes, or a full-featured programming language. Even though I don't really know or like Ruby, I still like that Vagrant uses it for config since it is at least possible to do complex things if you need to.
- zenlikethat 9y agoIt makes it impossible for the system to reason about the outputs though (due to the halting problem and a lot of other practical issues). Therefore a Vagrant setup just "does its best" to execute the given prescription to set a machine up, whereas something like Terraform will actually intelligently figure out if work needs to be done.
- eropple 9y agoThis is a strange assertion--you run the DSL to generate, at runtime, a data model that can then be checked. Hell, I wrote a Terraform competitor that was certainly no worse at planning than Terraform is. (Still not great, because the problem is a hard one, but.) And doing so gives you such niceties as if-statements and loops (I mean, have you seen the straight nonsense people have to pull to make a simple 'if' in HCL?) as well as a seamless way to introduce external data sources. This is why Chef remains the only part of my workflow that has not changed over the last five years (and why first Puppet and, eventually, Ansible will fall by the wayside): because, as a programming language and set of libraries, I can bring it with me by extending it at the user level without having to extend the runtime to do something. Patching the runtime means I need CM for my CM; doing it at the user level means it goes with me and plays with the base tools.
- mrmrcoleman 9y agoCongrats.
- aprdm 9y agoBig thanks to the Consul team! We have been using it for a year with basically 0 maintenance. * Decent REST API * K/V store * Service discovery * Single Go Binary * Geographical failover * Load balancing A real Swiss knife. Haven't tried different things (e.g: ZooKeeper) but don't feel very compelled to since everything just works.
- deleted 9y ago[deleted]
- erikb 9y agoWe used it heavily, but are switching away since it's not that stable inside kubernetes. Also some features are unnecessary if you have k8s already. Like you have a k/v store and you have service discovery already. I wasn't part of that discussion though. Thus I'm curious. Anybody got a different point of view on Consul in k8s?
- slackpad 9y agoDid you try 0.9.3 under k8s? We did some work specifically targeting k8s by enabling pods with servers to get restarted with new IPs, even all of the servers at once. That required Consul to be configured with -raft-protocol=3 to benefit in 0.9.3. That's the default now in 1.0, so both of those versions should be much more stable under k8s. I'd be curious if you ran into other kinds of issues on k8s and if you tried 0.9.3 with the required raft protocol version set before you gave up.
- iofiiiiiiiii 9y agoHow many people are actively working on it and maintaining it? From my experiences with Vagrant and Packer, I get the feeling that there is not much development effort (by Hashicorp employees) allocated to the free Hashicorp products so I hesitate to even consider using Consul. Vagrant and Packer feel like early alphas and while there is little competition, I do not feel comfortable using them. Because of this bad experience in the past, I have not even tried out Consul and Vault and am unlikely to do so in the future.
- jnsaff2 9y agoI don't know the answer to your question but while Vagrant and Packer both have been pretty bad experiences for me, Consul, Nomad, Vault and Terraform are very different products and all pretty actively worked on. I am using them all at the moment and am very happy. Consul seems like a very mature product and development has maybe slowed down, but I think this is mostly because it is mature and looks like HashiCorp does not feel the need for feature bloat.
- navlelo 9y agoConsul seems to get some love when looking at the releases, but I am really getting this impression when looking at Nomad. Seems like its been 0.6 forever.
- schmichael 9y agoI'm not sure if exact employee numbers are public, but I can assure you a lot of hiring has taken place over the last year and all of these products have dedicated engineering teams as well as supporting staff (customer success, sales, management, etc). (I'm a HashiCorp employee on the Nomad team.)
- oweiler 9y agoFor Service Discovery, why should I use Consul over Eureka?
- slackpad 9y agoConsul's documentation has a comparison at https://www.consul.io/intro/vs/eureka.html https://www.consul.io/intro/vs/eureka.html.
- anubhavmishra 9y agoConsul is a dream for most Operations Engineers. At work we have used it in production for almost 3 years with basically no maintenance except dropping a new binary for the old one when we update the versions. We use it for service discovery, feature flags, consul lock for deployments and use the KV as a config store. It is also our backend for Vault and stores all the encrypted data. Thank you HashiCorp and the Consul team for an amazing achievement.