4 ms·

Jepsen goes into detail about general CRDTs here: https://aphyr.com/posts/285-jepsen-riak https://aphyr.com/posts/285-jepsen-riak

lostcolony 9y ago
Yes, but devil's in the details. The fact CRDTs are provably able to have specific consistency guarantees doesn't mean a particular implementation does it correctly in all contexts. Without testing, "We built this (city) on CRDTs" isn't very useful.
- elvinyung 9y ago
  Sure. My point is that it's a rough idea of what you could expect. If you look at the Jepsen posts for Cassandra and Riak, for example, you can see that the general characteristics are fairly similar.
- KirinDave 9y ago
  What's nice about building databases on compositions of CRDTs is that if you can validate the individual CRDTs via automated testing, you have a very high degree of confidence the composition of those CRDTs will do something similar. No one's arguing that a Jepsen test shouldn't be done. Just that it'll probably be very different in character from more invented industry technology.
  - lostcolony 9y ago
    Yes; if you can prove a system was built atop an academically sound paradigm, it's more likely to adhere to the guarantees that paradigm is intended to provide than a system built atop an unproven paradigm. Kinda goes without saying, though. :P Unsnarkily, I agree that implementation and composition of CRDTs is comparatively straightforward compared with other approaches people have taken. But if correct behavior is a requirement, full testing is too, regardless of how easy it -should- be in theory.
OhDagny 9y ago
yeah, this appears to be Riak, after Riak went kaboom, possibly with anti-customer CRDTs turned on by default rather than the other marketing-friendly modes. Until we get John Galt to release his all-knowing database that can authoritatively resolve all conflicts and avoid the evil socialist resolution of AP read conflicts from his hidden valley of 1000 men and 1 woman, this will continue to be a hard problem.
- ramchip 9y ago
  I don’t think it’s Riak. It just uses riak_core, which is a kind of distribution / sharding framework. Riak itself has new owners and is not going away.
- cmeiklejohn 9y ago
  (former Basho engineer and committer on Riak and Antidote) No, it's not Riak. [Full disclosure: work was started in 2013/2014, with Basho, as an alternative to Riak and shares Riak Core as a base. At the time, Basho was a member of SyncFree.] It's built on Riak Core, but it's a lot different from Riak in many ways: no replication in the DC, transactions, more data-types, operation-based CRDTs and not state-based CRDTs, log vs. in-place replacement in the storage backend. I mean, the analogy is poor: it's even less close than saying that Riak is Cassandra and vice-versa.

lostcolony 9y ago

Yes, but devil's in the details. The fact CRDTs are provably able to have specific consistency guarantees doesn't mean a particular implementation does it correctly in all contexts. Without testing, "We built this (city) on CRDTs" isn't very useful.

elvinyung 9y ago
Sure. My point is that it's a rough idea of what you could expect. If you look at the Jepsen posts for Cassandra and Riak, for example, you can see that the general characteristics are fairly similar.
KirinDave 9y ago
What's nice about building databases on compositions of CRDTs is that if you can validate the individual CRDTs via automated testing, you have a very high degree of confidence the composition of those CRDTs will do something similar. No one's arguing that a Jepsen test shouldn't be done. Just that it'll probably be very different in character from more invented industry technology.
- lostcolony 9y ago
  Yes; if you can prove a system was built atop an academically sound paradigm, it's more likely to adhere to the guarantees that paradigm is intended to provide than a system built atop an unproven paradigm. Kinda goes without saying, though. :P Unsnarkily, I agree that implementation and composition of CRDTs is comparatively straightforward compared with other approaches people have taken. But if correct behavior is a requirement, full testing is too, regardless of how easy it -should- be in theory.

OhDagny 9y ago

yeah, this appears to be Riak, after Riak went kaboom, possibly with anti-customer CRDTs turned on by default rather than the other marketing-friendly modes. Until we get John Galt to release his all-knowing database that can authoritatively resolve all conflicts and avoid the evil socialist resolution of AP read conflicts from his hidden valley of 1000 men and 1 woman, this will continue to be a hard problem.

ramchip 9y ago
I don’t think it’s Riak. It just uses riak_core, which is a kind of distribution / sharding framework. Riak itself has new owners and is not going away.
cmeiklejohn 9y ago
(former Basho engineer and committer on Riak and Antidote) No, it's not Riak. [Full disclosure: work was started in 2013/2014, with Basho, as an alternative to Riak and shares Riak Core as a base. At the time, Basho was a member of SyncFree.] It's built on Riak Core, but it's a lot different from Riak in many ways: no replication in the DC, transactions, more data-types, operation-based CRDTs and not state-based CRDTs, log vs. in-place replacement in the storage backend. I mean, the analogy is poor: it's even less close than saying that Riak is Cassandra and vice-versa.