7 ms·
Etcd v3: increased scale and new APIs
- hn_rate_limiter 10y agoWill CORS ever be enabled for the https://discovery.etcd.io/new https://discovery.etcd.io/new endpoint?
- philips 10y agoThis is the GitHub issue for this service. Please +1 the thing on GitHub and we will try and get it fixed in prod: https://github.com/coreos/discovery.etcd.io/issues/12 https://github.com/coreos/discovery.etcd.io/issues/12
- meta_AU 10y agoWait... there are maintainers that encourage +1s all over their GitHub issues?
- jvoorhis 10y agoProbably referring to the new "reactions" feature. https://github.com/blog/2119-add-reactions-to-pull-requests-issues-and-comments https://github.com/blog/2119-add-reactions-to-pull-requests-...
- kpcyrd 10y agoIt's possible to +1 comments now without adding a new comment (and notifying maintainers)
- AYBABTME 10y agoThis cracks me up. An announcement/article about a company/product props up, and instead of commenting on the announcement/article, people come up with tangential comments.
- pg_is_a_butt 10y agoThis cracks me up. PR drone thinks they control response. idiot.
- hn_rate_limited 10y agoCracks you up? This a feature that has been missing since etcd's introduction and this article on a major new version makes no mention of it. Are we really so speech-restricted here that we can only discuss a press release in the company's own terms? The question is highly relevant to the etcd product and its future.
- dang 10y agoWe detached this subthread from https://news.ycombinator.com/item?id=12011247 https://news.ycombinator.com/item?id=12011247 and marked it off-topic.
- sandstrom 10y agoSounds interesting! What are the benefits of using Etcd over Consul? (https://www.consul.io/ https://www.consul.io/)
- meddlepal 10y agoI've used Consul but I haven't used Etcd directly, besides using via Kubernetes. One benefit IMO is the number of high-profile eyeballs on the project due to the success of K8s and the pedigree of engineers working on it.
- ideal0227 10y agoI think the two projects focus on different things. Consul provides features like health checking, failure detection besides its consistent key-value store. It aims to provide an all-in-one solution[0]. etcd focuses on the consistent key-value store. The key-value store has more advanced features like multi-version keys, reliable watches, and provides better performance. People build additional features on top of etcd's key-value store/raft. (I work on etcd) [0] https://www.consul.io/intro/vs/index.html https://www.consul.io/intro/vs/index.html
- otterley 10y agoDo you have proof of the "better performance" claim?
- ideal0227 10y agoConsul: https://github.com/hashicorp/consul/blob/master/bench/results-0.3.md https://github.com/hashicorp/consul/blob/master/bench/result... etcd: https://github.com/coreos/etcd/blob/master/Documentation/op-guide/performance.md#benchmarks https://github.com/coreos/etcd/blob/master/Documentation/op-... Note that the testing environments are not exact the same in the doc I listed, but comparable. Also Consul performance is improved after a few releases. So we did the benchmark internally on the same environment. The result is still comparable to what I listed in the two official docs. The best way to compare performance is still probably to run the benchmark on your own environment.
- Ygor 10y agoEtcd looks more and more promising as its usage and development activity increases. Anyone using it internally as a standalone part in the system (e.g. not just for k8s or coreos)? Using e.g. gRPC shows great promise, but systems like ZooKeeper still play nicer in more traditional Java shops, or do they? How hard is it to use etcd from the JVM?
- ideal0227 10y agoIt should not be hard. For v2 API, there are several Java bindings (https://github.com/jurmous/etcd4j https://github.com/jurmous/etcd4j). For the gRPC API, it is easy to generate a Java gRPC client based on the defined service. We have plans to make that experience better.
- jzelinskie 10y agoThere's a project called zetcd[0] that acts as a translation proxy in front of etcd to let you use the ZooKeeper API. I don't know if it's production ready, but I do know it works and is a pretty cool idea. [0]: https://github.com/chzchzchz/zetcd https://github.com/chzchzchz/zetcd
- cbsmith 10y agoSeems pretty easy to use etcd from a JVM, particularly now that it is gRPC based.
- bogomipz 10y agoEtcd looks more promising? Zookeeper has been around now for over 5 years with an extremely large install base. Kafka, Hadoop, Solr, Mesos and Hbase projects that leverage Zookeeper distributed coordination. Zookeeper has already delivered.
- harlowja 10y agoI would tend to agree, knowing what zookeeper has been doing and actually using zookeeper (and etcd) I can say that the API and the primitives offered by zookeeper are IMHO better (although this multi-version concurrency control model is interesting) and more mature. It feels like etcd is 'still discovering itself' for lack of better words. Btw: https://issues.apache.org/jira/browse/ZOOKEEPER-2169 https://issues.apache.org/jira/browse/ZOOKEEPER-2169 (this is the equivalent of TTLs for zookeeper).
- gshx 10y agoDoes someone have any benchmarks or comparisons with Zk? We have run it for many yrs without a problem and are very happy with it. Would be interested in hearing from anyone who switched from zk->etcd for distributed locking, presence, leader election type idioms and ran it in prod for a few months/yrs and their takeaways.
- deleted 10y ago[deleted]
- ideal0227 10y agoetcd3 has similar performance compared to ZK for small scale. For large dataset, etcd3 does better since it does incremental snapshot/ smaller memory footprint, when ZK does full snapshot that takes a lot of resources. For watches, etcd3 does streaming + TCP multiplexing to save memory.
- embiggen 10y agoI too have been using ZK for many years now, and it's pretty great. Etcd can provide faster election notifications but it comes at the cost of etcd still being pretty new, so be prepared to get cut by the bleeding edge :)
- Randgalt 10y agoetcd is a key value store that has features of ZK. ZooKeeper is strictly distributed coordination. Apples and oranges, no?
- atombender 10y agoApples all the way. Etcd is pretty much a clone of ZooKeeper in Go; they both support hiearchical keys, atomic autoincrement, watches, though Etcd uses the Raft consensus algorithm, whereas ZooKeeper uses its own homegrown algorithm, and there are other minor differences. Both are intended for configuration management and coordination. (Of course, both are ultimately clones of Google's internal tool, Chubby.)
- 10y ago
- agentultra 10y agoDo they publish formal specifications of these distributed algorithms? Has anyone verified the implementation? Zookeeper is at least based on Paxos which has a TLA+ model one can check.
- emirozer 10y agoIt's based on Raft : https://github.com/coreos/etcd/tree/master/raft https://github.com/coreos/etcd/tree/master/raft https://raft.github.io/ https://raft.github.io/
- philips 10y agoWe also do a ton of functional testing as code is merged to test lots of different partitions and faults. You can read more on this post: https://coreos.com/blog/new-functional-testing-in-etcd/ https://coreos.com/blog/new-functional-testing-in-etcd/
- ideal0227 10y agoetcd is based on Raft. Raft has a TLA+ spec. But do note that the implementation usually diverges from its algorithm [0]. For etcd, we try to keep the core algorithm as self-contained and deterministic (no I/O, no timer) as possible. So it can be very close to the pure algorithm. We are very confident about it since we throughly tested it, and the implementation is shared with other consistent large scale database systems too (cockroachdb, tikv). ZooKeeper uses ZAB [1] under the hood. I do not think there is a TLA+ for ZAB. [0] https://www.cs.utexas.edu/users/lorenzo/corsi/cs380d/papers/paper2-1.pdf https://www.cs.utexas.edu/users/lorenzo/corsi/cs380d/papers/... [1] http://www-users.cselabs.umn.edu/classes/Spring-2016/csci8211/Papers/Distributed%20Systems%20Zab-%20High-performance%20broadcast%20for%20primary-backup%20systems.pdf http://www-users.cselabs.umn.edu/classes/Spring-2016/csci821...
- agentultra 10y agoRaft - cool! I'm aware that implementations diverge or do not directly synthesize from their specifications. At least not yet. I'll be poking around more to see if the etcd team has published their own specs for the novel parts of the system. I'm particularly interested in seeing if/how OSS projects are adopting more rigorous "engineering" practices.
- qwertyuiop924 10y agoThis doesn't look like feature creep, but coreos bought into systemd big time, and with etcd being used in more places, the temptation of feature creep grows... and that's worrying.
- Randgalt 10y agoAre there high level APIs for etcd like there are for ZooKeeper? I'm the main author of Apache Curator and I know that writing "recipes" is not trivial.
- ideal0227 10y agoYes. Here: https://godoc.org/github.com/coreos/etcd/clientv3/concurrency https://godoc.org/github.com/coreos/etcd/clientv3/concurrenc... It would be great if you can provide opinions, comments or help on these high level APIs. We also might move these to an internal proxy layer, so that other clients in other language can use it more easily. (some more here: https://github.com/coreos/etcd/tree/master/contrib/recipes https://github.com/coreos/etcd/tree/master/contrib/recipes)
- Randgalt 10y agoI assume there's a Java API for etcd? If so, it would be interesting to try to port Curator.
- ideal0227 10y agoWe are working on it (https://github.com/coreos/etcd/issues/5067 https://github.com/coreos/etcd/issues/5067). Probably we could work together on the Java client first? It should not be hard given that gRPC supports Java.
- Perceptes 10y agoIt's not mentioned in the blog post—is there a document that explains the migration plan for the etcd that ships on the host in CoreOS? Edit: They haven't been published to the docs on the CoreOS website yet, but there are two documents listed under "upgrading and compatibility" at the bottom of https://github.com/coreos/etcd/blob/v3.0.0/Documentation/docs.md https://github.com/coreos/etcd/blob/v3.0.0/Documentation/doc...
- philips 10y agoGreat question. The upgrade path is a rolling upgrade from v2.3.y series to v3.0.0 series. This is how all of the etcd upgrades have worked since the start of v2.x.y. Doc is here: https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md https://github.com/coreos/etcd/blob/master/Documentation/upg...
- Perceptes 10y agoSeems worth noting as well that this only upgrades the version of the cluster. Data populated via the v2 API will not magically be available via the v3 API as they have separate data stores/keyspaces. https://github.com/coreos/etcd/blob/v3.0.0/Documentation/op-guide/v2-migration.md https://github.com/coreos/etcd/blob/v3.0.0/Documentation/op-... talks about how to migrate data that was stored with v2 to v3's data store.
- jondubois 10y agoLast time I looked into Etcd, I got the impression that it was great at handling a high-volume of read operations (theoretically read-scalability) but bad for handling high-volume of write operations (since every write has to be propagated to every node in the cluster via Raft)? Is this still the case in v3?