5 ms·

Is there a good open-source Paxos implementation? There are many open-source Raft implementations, some of which are good. goraft powers etcd, for example. I

Is there a good open-source Paxos implementation? There are many open-source Raft implementations, some of which are good. goraft powers etcd, for example. I believe most developers just want to consume a working library. It seems likely that the plethora of Raft libraries and dearth of Paxos libraries is because it is easier to write a Raft library, but I also don't think it matters _why_, it only matters that there are good Raft libraries.

tedsuo 12y ago
CoreOs (etcd team) is currently rewriting their raft implementation because it had a lot of problems. I don't believe there is a "good" implementation out there yet, because it's all very new. And should an implantation be "good", AKA implement the spec without introducing incidental errors, will it still have edge cases because the spec has edge cases? This is all currently unknown. It's precisely these assumptions that I'm pointing out. A quick google search brings up libpaxos and other open paxos implementations, so it appears that they exist.
- justinsb 12y ago
  Interesting - thanks for the libpaxos link. Is it any good? I'm curious: what are these edge cases you're talking about? I've worked on a Raft implementation, and I don't remember any particular edge cases (though it's been a while). There's definitely some interesting choices for when to replace nodes in the face of failure, but I don't think that's a Raft algorithm shortcoming.
- alexnewman 12y ago
  If you are looking for a good one. Checkout ours https://github.com/cloud-software-foundation/c5-replicator https://github.com/cloud-software-foundation/c5-replicator
  - justinsb 12y ago
    Wow - this looks like a great implementation. Who is cloud-software-foundation? Is this the software that powers WANdisco?
    - alexnewman 12y ago
      This does not power wandisco. WanDISCO uses DConE.
- philips 12y ago
  The new etcd raft implementation is working well and has a solid set of deterministic tests. We recently released an alpha of etcd that utilizes this improved raft implementation: https://github.com/coreos/etcd/releases https://github.com/coreos/etcd/releases. The core raft state machine comes in under 600 LOC last time I checked and the docs are here: https://godoc.org/github.com/coreos/etcd/raft https://godoc.org/github.com/coreos/etcd/raft. There are a few non-etcd projects that have started working with it, most notably the cockroachdb folks have started to use it to implement the "multi-raft" that they need for their DB. You can find the discussion and work on github.com/cockroachdb. It turns out getting a raft implementation that is easy to audit with a copy of the paper in hand and easy to test in a fast and consistent way requires a significant engineering effort. I hope that based on the lessons learned from goraft that this implementation is something that a number of Go projects can use and leverage. Particularly since we carefully separated the WAL, RPC and state machine from each other.
rdtsc 12y ago
Yes there is -- Basho Riak's riak_ensemble module implements a version of Multi-Paxos. Description: https://github.com/basho/riak_ensemble/blob/develop/doc/Readme.md https://github.com/basho/riak_ensemble/blob/develop/doc/Read... Code: https://github.com/basho/riak_ensemble https://github.com/basho/riak_ensemble Ensemble module, the way I understand it, provides ability to have "consistent" CP (in CAP theorem sense) updates in the otherwise default AP database setup in Riak. Disclaimer: I am not affiliated in any way with Basho, just follow riak_core module development because I think it is a piece of very cool engineering.
contingencies 12y ago
Is there a good open-source Paxos implementation? Try Corosync/Pacemaker. I believe most developers just want to consume a working library. Right. Maybe they'd even prefer not to have to bother, they just want the availability thing solved for parallel service instances. Perhaps the more interesting question is, "how do you neatly fit raft/paxos like functionality at deployment time in to the average SOA service development process?" (subtext: without making devs learn either algorithm) I think the answer there is an improved devops process that discourages any form of assumption around infrastructure type and can easily run failure tests at various levels of the deployment topology. Some of my thinking in the area is documented at http://stani.sh/walter/pfcts http://stani.sh/walter/pfcts

tedsuo 12y ago

CoreOs (etcd team) is currently rewriting their raft implementation because it had a lot of problems. I don't believe there is a "good" implementation out there yet, because it's all very new. And should an implantation be "good", AKA implement the spec without introducing incidental errors, will it still have edge cases because the spec has edge cases? This is all currently unknown. It's precisely these assumptions that I'm pointing out. A quick google search brings up libpaxos and other open paxos implementations, so it appears that they exist.

justinsb 12y ago
Interesting - thanks for the libpaxos link. Is it any good? I'm curious: what are these edge cases you're talking about? I've worked on a Raft implementation, and I don't remember any particular edge cases (though it's been a while). There's definitely some interesting choices for when to replace nodes in the face of failure, but I don't think that's a Raft algorithm shortcoming.
alexnewman 12y ago
If you are looking for a good one. Checkout ours https://github.com/cloud-software-foundation/c5-replicator https://github.com/cloud-software-foundation/c5-replicator
- justinsb 12y ago
  Wow - this looks like a great implementation. Who is cloud-software-foundation? Is this the software that powers WANdisco?
  - alexnewman 12y ago
    This does not power wandisco. WanDISCO uses DConE.
philips 12y ago
The new etcd raft implementation is working well and has a solid set of deterministic tests. We recently released an alpha of etcd that utilizes this improved raft implementation: https://github.com/coreos/etcd/releases https://github.com/coreos/etcd/releases. The core raft state machine comes in under 600 LOC last time I checked and the docs are here: https://godoc.org/github.com/coreos/etcd/raft https://godoc.org/github.com/coreos/etcd/raft. There are a few non-etcd projects that have started working with it, most notably the cockroachdb folks have started to use it to implement the "multi-raft" that they need for their DB. You can find the discussion and work on github.com/cockroachdb. It turns out getting a raft implementation that is easy to audit with a copy of the paper in hand and easy to test in a fast and consistent way requires a significant engineering effort. I hope that based on the lessons learned from goraft that this implementation is something that a number of Go projects can use and leverage. Particularly since we carefully separated the WAL, RPC and state machine from each other.

rdtsc 12y ago

Yes there is -- Basho Riak's riak_ensemble module implements a version of Multi-Paxos. Description: https://github.com/basho/riak_ensemble/blob/develop/doc/Readme.md https://github.com/basho/riak_ensemble/blob/develop/doc/Read... Code: https://github.com/basho/riak_ensemble https://github.com/basho/riak_ensemble Ensemble module, the way I understand it, provides ability to have "consistent" CP (in CAP theorem sense) updates in the otherwise default AP database setup in Riak. Disclaimer: I am not affiliated in any way with Basho, just follow riak_core module development because I think it is a piece of very cool engineering.

contingencies 12y ago

Is there a good open-source Paxos implementation? Try Corosync/Pacemaker. I believe most developers just want to consume a working library. Right. Maybe they'd even prefer not to have to bother, they just want the availability thing solved for parallel service instances. Perhaps the more interesting question is, "how do you neatly fit raft/paxos like functionality at deployment time in to the average SOA service development process?" (subtext: without making devs learn either algorithm) I think the answer there is an improved devops process that discourages any form of assumption around infrastructure type and can easily run failure tests at various levels of the deployment topology. Some of my thinking in the area is documented at http://stani.sh/walter/pfcts http://stani.sh/walter/pfcts