5 ms·
FoundationDB vs. Jepsen
- AdrianRossouw 13y agohas there been one for couchdb yet?
- rancor 13y agoThere has not to my knowledge. As a CouchDB user I'm definitely interested in the results, but it's not clear to me what cluster configuration to test.
- jrullmann 13y agoYou'll need two pieces to run Jepsen against a database: 1. Code to provision a five cluster database (using Aphyr's Salticid deployment system) 2. Code to do some reads and writes to the database (this is run by Jepsen) It might be helpful to look through the pull request I submitted to Aphyr for FoundationDB's test, so you can see the pieces: https://github.com/aphyr/jepsen/pull/10 https://github.com/aphyr/jepsen/pull/10
- willholley 13y agoWe (Cloudant) ran a Jepsen-style test against our clustered CouchDB implementation (similar to BigCouch). The write-up is at https://cloudant.com/blog/run-dmc-explains-network-partitions/ https://cloudant.com/blog/run-dmc-explains-network-partition.... The clustering layer is getting merged into CouchDB and will provide semantics closer to other Dynamo-style systems (e.g. Cassandra, Riak).
- rdtsc 13y agoWith couchdb you can set up your own couch cluster replication topology: * master-slave * a tree * a ring * a ring with overlaps * completely connected graph Well basically any graph topology you like. Replications can be pull or push. The downside is you are in charge of handling and setting up proxies and deciding how memebership is handled. Whether you have a proxy or not and if so how do client pick were to write etc. Also CouchDB only offers atomic document transaction. If you need atomic multi-document transactions you have to use a transaction log based on single docs. But you get continuous replication, a nice browser GUI that helps you debug as you develop, and most of all, a well defined conflict model when there are conflicts in your database. It would seem to be Jepsen like test doesn't make sense for CDB as it doesn't ship with a default single cluster setup.
- jrullmann 13y agoHey guys, I'm the author of the post. I'm happy to answer any questions or comments you may have about it.
- ForgotMyUName 13y agoThis was a good post in general, though you misrepresented Riak by only talking about the situations where allow_mult=false in the Jepsen post. If allowing for siblings, Jepsen and Riak play nice together: https://pbs.twimg.com/media/BjmZeBYCEAAqCk3.png:large https://pbs.twimg.com/media/BjmZeBYCEAAqCk3.png:large
- jrullmann 13y agoThanks for pointing this out. That's totally a mistake on my part. I've updated the table to include the CRDT result, showing Riak's 0% data loss.
- ForgotMyUName 13y agoThank you.
- deleted 13y ago[deleted]
- misframer 13y agoWhat's the significance of the 1 unacknowledged write in the last run?
- teraflop 13y agoThat means a client submitted a write that was successfully committed, but the client never found out that it was successful. It's more or less inevitable in the face of network partitions. You don't want the database to report that a transaction was successful before the data is actually committed, or else you lose consistency. But if you wait until after the commit happens, there's a chance that the client might no longer be reachable. There's no way around this aside from waiting an indefinite period of time for the partition to be resolved.
- misframer 13y agoThat makes perfect sense. If it is the case, I think it's worth mentioning that the acknowledge write happened when the partition happened. I assumed that it happened when everything was going well, which is odd.
- jrullmann 13y agoGood idea - I've updated the blog post. Thanks for the feedback!
- jrullmann 13y agoRight. Most of the time you would simply retry the transaction - that's what FoundationDB's automatic transaction retry loop does. The first two tests I ran do that, but the third (foundationdb-append-noretry) does not.
- jcoene 13y agoWhen a network partition occurs it's possible that transactions be committed but not acknowledged due to the inability to receive the ack.
- rebelidealist 13y agoHow does this compare with http://rethinkdb.com/ http://rethinkdb.com/ ?
- misframer 13y agoRethinkDB doesn't support ACID transactions.
- nlavezzo 13y agoYep, that's a big difference. RethinkDB is also a document store, whereas FoundationDB is an ordered K/V store at its core (although transactions / layers allow for richer data modeling through stateless translation "layers"). I don't think there's any data on RethinkDB and Jepsen. Would be interesting to see someone do the work to make that happen.
- sporkland 13y agoIt is worth noting that an ordered K/V store is a very simple and flexible data structure in terms of the types of operations that can be efficiently layered on top of it. The simplicity gives you guys a chance to do something insanely hard like distributed transactions. The flexibility lets people build a ton of different abstractions on top of it.
- jessaustin 13y agoIt might have been interesting to arrange the partition so that both the master and "cluster controller" went to the minority side. Then a new CC would have had to have been elected before the CC could choose a new master.
- jrullmann 13y agoI've been playing around with the idea of doing a couple of these posts - for example, I could do one with multiple datacenters. This would be a good scenario too.