6 ms·

Red Hat contributes etcd to the Cloud Native Computing Foundation

256 points by pplonski86 8y ago

ross2990 8y ago
Glad to see that Red Hat is still committed to the Open Source movement after so many naysayers predicted that all contributions going forward would be stymied due to the announcement of the acquisition.
- jakevn 8y ago
  Not saying it will happen (just that it usually does), but I almost never see acquired companies immediately turn into the parent. Usually the acquired maintains its course until attrition and cross-pollination replace its original culture with that of the parent. The acquired company eventually exists only as a collection of intellectual property and history. Could take many years depending on how tightly IBM squeezes. That and the acquisition isn't even finalized yet.
  - wlesieutre 8y ago
    I wonder if "We should get this taken care of before IBM takes over" was part of the thought process here.
    - Proven 8y ago
      What makes you think federated DB2 couldn’t replace etcd? Uhm, yeah.
- halfastack 8y ago
  The acquisition hasn’t even happened yet, so there’s no influence to see just yet. In fact, it’s illegal for RH to change for IBM prior to the acquisition.
barbecue_sauce 8y ago
I was wondering when this was going to happen. It's such a critical piece of Kubernetes.
philips 8y ago
etcd has succeeded as a piece of distributed systems infrastructure beyond our wildest expectation. When Alex Polvi, Xiang Li, and I started the project as a README in the summer of 2013 we identified that their still was no consensus database that was developer friendly, easily secured, production ready, and based on a well understood consensus algorithm. And largely we got lucky with good market timing, the invention of the Raft algorithm, and the explosion of good tooling around the Go language. This lead to early success as etcd was used in locksmith, skydns, and vulcan load balancer. As the years went on we got lucky again that the Kubernetes project chose etcd as its primary key-value database. This helped to establish the project as a must use piece of infrastructure software. Which went on to influence the technology selection of storage, database, networking, and many other projects. Just check out all of the stickers of projects relying on etcd that I could find at KubeCon here in Seattle. https://twitter.com/BrandonPhilips/status/1073701369872187393 https://twitter.com/BrandonPhilips/status/107370136987218739... For a sense of all of the projects that use checkout this list we maintain in the project: https://github.com/etcd-io/etcd/blob/master/Documentation/integrations.md https://github.com/etcd-io/etcd/blob/master/Documentation/in... Some notable projects include: Kubernetes, Rook, CoreDNS, Uber M3, Trillian, Vitess, TiDB, and many many others. Moving into the CNCF will help to bring a few things to the project: - Funding and resources to complete regular third-party security audits and correctness audits - On-call rotation and team for the discovery.etcd.io system - Assistance in maintaining a documentation website - Resources to fund face to face meetup groups and maintainer meetings As a closing remark I want to thank the over 450 contributors and the entire maintainer team for bringing the project to this point. We are solving an important distributed systems problem with a focused piece of technology. In fact, in Seattle this week we all got together as a maintainer team for the first time ever: https://twitter.com/sp_zala/status/1073239003330015233 https://twitter.com/sp_zala/status/1073239003330015233 If you want to learn more about the history of the project checkout this other post blog post: https://coreos.com/blog/history-etcd https://coreos.com/blog/history-etcd
- pstuart 8y ago
  It is indeed work that you and your team should be proud of. Any thoughts on rkt?
RcouF1uZ4gsC 8y ago
I wonder if this the core of Redhat knowing that the clock is ticking, and make sure that critical software that they worked on is available in an open fashion. One only has to look at Sun and MySQL to see what can happen to a once vibrant open source offering after acquisition.
- shykes 8y ago
  I don't understand this line of thinking. IBM has plenty of people contributing to open-source projects. I wouldn't be surprised if they contributed to etcd even before the acquisition. When it comes to their open-source track record, IBM and Oracle are nothing alike.
- moosingin3space 8y ago
  Red Hat does not own anything that's valuable aside from their developers, who chose to work at Red Hat due to their pro-FOSS positioning. If IBM chose to start shutting projects like Fedora down or move in the direction of closed-source, these developers would have no desire to remain, and would leave, making that $40B acquisition worthless.
  - rbetts 8y ago
    Red Hat has extensive customer relationships, customer databases, contracts for future revenue, partnerships, operational processes, . . . product IP is just a slice of the pie.
- crb 8y ago
  It was proposed in July (https://github.com/cncf/toc/issues/136 https://github.com/cncf/toc/issues/136) and voted on in September (https://github.com/cncf/toc/pull/143 https://github.com/cncf/toc/pull/143), all before the IBM/Red Hat deal was announced or known to the staff. The announcement was held for publicity at KubeCon.
Rapzid 8y ago
> it demonstrated its quality early on I find that slightly revisionist though I suppose it depends on your definition of "early on". To my recollection etcd had a very rough patch early on until they overhauled their raft sub-system. Hashicorp caught some flak at the time for giving the raft implementation a hard pass and writing their own. Go ecosystem fragmentation was particularly bad and a very hot topic at the time. I believe that etcd's new found stability and subsequent track record after the new implementation vindicated Hashicorp somewhat.
- ideal0227 8y ago
  Let me share a little bit more story here. The original go-raft is one of the first raft implementations. At that time, the raft paper was not even officially published. Many other attempts around that time were not very successful, including go-raft. Making a production ready consensus algorithm is not easy: https://www.cs.utexas.edu/users/lorenzo/corsi/cs380d/papers/paper2-1.pdf https://www.cs.utexas.edu/users/lorenzo/corsi/cs380d/papers/.... Things like pipelining, batching, flow control, asynchronous snapshot were not extensively explored in the context of raft. And not much effort has been put into testing due to the immaturity of the applications of raft at the time. We realized the problem a few months after etcd alpha was initially released and became popular. However, I went back to CMU to continue my master degree for 1yr, which slowed down the progress. After I came back from school, together with Blake, Yicheng from CoreOS and later on Ben from Cockroach Labs, we built a solid raft impl as our first priority. Once we put etcd/raft inside etcd2, the stability of etcd greatly improved. That is about 1.5 yr after the initial release. Now etcd/raft powers many production level distributed systems: tikv, cockroachdb, dgraph and many others. Over the last couples of years, the focus of etcd/raft is always stability and nothing else (although people are blaming us for usability :P).
spullara 8y ago
This is exactly the kind of thing that should be a layer on top of FoundationDB. Too many state stores all doing largely the same thing.
- pram 8y ago
  FoundationDB has only been open source for 8 months?? etcd is like 5 years old.
  - reilly3000 8y ago
    FoundationDB started in 2009 and released their 1.0 in 2013. Source: https://en.wikipedia.org/wiki/FoundationDB https://en.wikipedia.org/wiki/FoundationDB
    - unfunco 8y ago
      From the same source: > On April 19, 2018, Apple open sourced the software, releasing it under the Apache 2.0 license.
    - deleted 8y ago
      [deleted]
  - spullara 8y ago
    I meant going forward, not saying that etcd should have originally been built on top of it.
polskibus 8y ago
How does etcd compare to zookeeper? What made etcd the choice for kubernetes and cncf?
- guipsp 8y ago
  zookeeper uses a coordination kernel, while etcd uses a replicated state machine.
  - polskibus 8y ago
    What are the pros and cons? Should etcd be the default whenever one is thinking about using something like that?
    - dkoston 8y ago
      It really depends on your use case but one of the main "pros" of etcd is the narrow latency band when writing. This article is likely biased to the good parts of etcd as it's written by coreOS but you can see how the latency of writes in etcd is very consistent compared to the wide range of latencies experienced writing to ZooKeeper or Consul: https://coreos.com/blog/performance-of-etcd.html https://coreos.com/blog/performance-of-etcd.html There are other "pros" related to the fact that it's been designed for "cloud native" architectures like kubernetes. For example, FoundationDB can perform on average at sub-milisecond latency for writes (https://apple.github.io/foundationdb/benchmarking.html https://apple.github.io/foundationdb/benchmarking.html) versus 1.6ms on etcd however configuring FoudationDB to run programmatically is challenging as it was designed in an environment where ops people rack physical servers. All key/value stores have good points and bad points but that's in relation to your use case. If write or read throughput isn't the most important, say it's consistency or availability, you may make a different choice about what are "pros" and what are "cons". Another "pro" or "con" may be the language its written in or how it runs or deploys. If you run a Java shop and have tons of experience writing and deploying Java code, it may be in your best interest to be able to have more control by using a project written in Java. conversely, if you have all go engineers, you may want a project written in go. If you only have junior engineers, you may want whatever is easiest to operate and deploy.
fulafel 8y ago
How is etcd security these days? Authentication still off by default?

256 points by pplonski86 8y ago

Glad to see that Red Hat is still committed to the Open Source movement after so many naysayers predicted that all contributions going forward would be stymied due to the announcement of the acquisition.

jakevn 8y ago
Not saying it will happen (just that it usually does), but I almost never see acquired companies immediately turn into the parent. Usually the acquired maintains its course until attrition and cross-pollination replace its original culture with that of the parent. The acquired company eventually exists only as a collection of intellectual property and history. Could take many years depending on how tightly IBM squeezes. That and the acquisition isn't even finalized yet.
- wlesieutre 8y ago
  I wonder if "We should get this taken care of before IBM takes over" was part of the thought process here.
  - Proven 8y ago
    What makes you think federated DB2 couldn’t replace etcd? Uhm, yeah.
halfastack 8y ago
The acquisition hasn’t even happened yet, so there’s no influence to see just yet. In fact, it’s illegal for RH to change for IBM prior to the acquisition.

barbecue_sauce 8y ago

I was wondering when this was going to happen. It's such a critical piece of Kubernetes.

philips 8y ago

etcd has succeeded as a piece of distributed systems infrastructure beyond our wildest expectation. When Alex Polvi, Xiang Li, and I started the project as a README in the summer of 2013 we identified that their still was no consensus database that was developer friendly, easily secured, production ready, and based on a well understood consensus algorithm. And largely we got lucky with good market timing, the invention of the Raft algorithm, and the explosion of good tooling around the Go language. This lead to early success as etcd was used in locksmith, skydns, and vulcan load balancer. As the years went on we got lucky again that the Kubernetes project chose etcd as its primary key-value database. This helped to establish the project as a must use piece of infrastructure software. Which went on to influence the technology selection of storage, database, networking, and many other projects. Just check out all of the stickers of projects relying on etcd that I could find at KubeCon here in Seattle. https://twitter.com/BrandonPhilips/status/1073701369872187393 https://twitter.com/BrandonPhilips/status/107370136987218739... For a sense of all of the projects that use checkout this list we maintain in the project: https://github.com/etcd-io/etcd/blob/master/Documentation/integrations.md https://github.com/etcd-io/etcd/blob/master/Documentation/in... Some notable projects include: Kubernetes, Rook, CoreDNS, Uber M3, Trillian, Vitess, TiDB, and many many others. Moving into the CNCF will help to bring a few things to the project: - Funding and resources to complete regular third-party security audits and correctness audits - On-call rotation and team for the discovery.etcd.io system - Assistance in maintaining a documentation website - Resources to fund face to face meetup groups and maintainer meetings As a closing remark I want to thank the over 450 contributors and the entire maintainer team for bringing the project to this point. We are solving an important distributed systems problem with a focused piece of technology. In fact, in Seattle this week we all got together as a maintainer team for the first time ever: https://twitter.com/sp_zala/status/1073239003330015233 https://twitter.com/sp_zala/status/1073239003330015233 If you want to learn more about the history of the project checkout this other post blog post: https://coreos.com/blog/history-etcd https://coreos.com/blog/history-etcd

pstuart 8y ago
It is indeed work that you and your team should be proud of. Any thoughts on rkt?

RcouF1uZ4gsC 8y ago

I wonder if this the core of Redhat knowing that the clock is ticking, and make sure that critical software that they worked on is available in an open fashion. One only has to look at Sun and MySQL to see what can happen to a once vibrant open source offering after acquisition.

shykes 8y ago
I don't understand this line of thinking. IBM has plenty of people contributing to open-source projects. I wouldn't be surprised if they contributed to etcd even before the acquisition. When it comes to their open-source track record, IBM and Oracle are nothing alike.
moosingin3space 8y ago
Red Hat does not own anything that's valuable aside from their developers, who chose to work at Red Hat due to their pro-FOSS positioning. If IBM chose to start shutting projects like Fedora down or move in the direction of closed-source, these developers would have no desire to remain, and would leave, making that $40B acquisition worthless.
- rbetts 8y ago
  Red Hat has extensive customer relationships, customer databases, contracts for future revenue, partnerships, operational processes, . . . product IP is just a slice of the pie.
crb 8y ago
It was proposed in July (https://github.com/cncf/toc/issues/136 https://github.com/cncf/toc/issues/136) and voted on in September (https://github.com/cncf/toc/pull/143 https://github.com/cncf/toc/pull/143), all before the IBM/Red Hat deal was announced or known to the staff. The announcement was held for publicity at KubeCon.

Rapzid 8y ago

> it demonstrated its quality early on I find that slightly revisionist though I suppose it depends on your definition of "early on". To my recollection etcd had a very rough patch early on until they overhauled their raft sub-system. Hashicorp caught some flak at the time for giving the raft implementation a hard pass and writing their own. Go ecosystem fragmentation was particularly bad and a very hot topic at the time. I believe that etcd's new found stability and subsequent track record after the new implementation vindicated Hashicorp somewhat.

ideal0227 8y ago
Let me share a little bit more story here. The original go-raft is one of the first raft implementations. At that time, the raft paper was not even officially published. Many other attempts around that time were not very successful, including go-raft. Making a production ready consensus algorithm is not easy: https://www.cs.utexas.edu/users/lorenzo/corsi/cs380d/papers/paper2-1.pdf https://www.cs.utexas.edu/users/lorenzo/corsi/cs380d/papers/.... Things like pipelining, batching, flow control, asynchronous snapshot were not extensively explored in the context of raft. And not much effort has been put into testing due to the immaturity of the applications of raft at the time. We realized the problem a few months after etcd alpha was initially released and became popular. However, I went back to CMU to continue my master degree for 1yr, which slowed down the progress. After I came back from school, together with Blake, Yicheng from CoreOS and later on Ben from Cockroach Labs, we built a solid raft impl as our first priority. Once we put etcd/raft inside etcd2, the stability of etcd greatly improved. That is about 1.5 yr after the initial release. Now etcd/raft powers many production level distributed systems: tikv, cockroachdb, dgraph and many others. Over the last couples of years, the focus of etcd/raft is always stability and nothing else (although people are blaming us for usability :P).

spullara 8y ago

This is exactly the kind of thing that should be a layer on top of FoundationDB. Too many state stores all doing largely the same thing.

pram 8y ago
FoundationDB has only been open source for 8 months?? etcd is like 5 years old.
- reilly3000 8y ago
  FoundationDB started in 2009 and released their 1.0 in 2013. Source: https://en.wikipedia.org/wiki/FoundationDB https://en.wikipedia.org/wiki/FoundationDB
  - unfunco 8y ago
    From the same source: > On April 19, 2018, Apple open sourced the software, releasing it under the Apache 2.0 license.
  - deleted 8y ago
    [deleted]
- spullara 8y ago
  I meant going forward, not saying that etcd should have originally been built on top of it.

polskibus 8y ago

How does etcd compare to zookeeper? What made etcd the choice for kubernetes and cncf?

guipsp 8y ago
zookeeper uses a coordination kernel, while etcd uses a replicated state machine.
- polskibus 8y ago
  What are the pros and cons? Should etcd be the default whenever one is thinking about using something like that?
  - dkoston 8y ago
    It really depends on your use case but one of the main "pros" of etcd is the narrow latency band when writing. This article is likely biased to the good parts of etcd as it's written by coreOS but you can see how the latency of writes in etcd is very consistent compared to the wide range of latencies experienced writing to ZooKeeper or Consul: https://coreos.com/blog/performance-of-etcd.html https://coreos.com/blog/performance-of-etcd.html There are other "pros" related to the fact that it's been designed for "cloud native" architectures like kubernetes. For example, FoundationDB can perform on average at sub-milisecond latency for writes (https://apple.github.io/foundationdb/benchmarking.html https://apple.github.io/foundationdb/benchmarking.html) versus 1.6ms on etcd however configuring FoudationDB to run programmatically is challenging as it was designed in an environment where ops people rack physical servers. All key/value stores have good points and bad points but that's in relation to your use case. If write or read throughput isn't the most important, say it's consistency or availability, you may make a different choice about what are "pros" and what are "cons". Another "pro" or "con" may be the language its written in or how it runs or deploys. If you run a Java shop and have tons of experience writing and deploying Java code, it may be in your best interest to be able to have more control by using a project written in Java. conversely, if you have all go engineers, you may want a project written in go. If you only have junior engineers, you may want whatever is easiest to operate and deploy.

fulafel 8y ago

How is etcd security these days? Authentication still off by default?