4 ms·

There are equations/properties for data that syncs nicely and allows mutli-master writes. Ie, if your merges follow a few simple properties, this stuff will wor

There are equations/properties for data that syncs nicely and allows mutli-master writes. Ie, if your merges follow a few simple properties, this stuff will work. These are basically what CRDTs are. And I know what you're thinking; "but I'm not writing a mutli user text editor!" or "automerge won't scale!". But CRDTs aren't a library, they're a set of properties. if your whole data system obeys the properties - you have a CRDT.

klysm 2y ago
I’m not really sure what you’re saying. Do you mean you achieve the gaurantees of CRDTs without using CRDTs? What technology are you implicitly referring to? Whenever I hear someone say “sync” data, I instantly get scared. Consensus is fraught with peril and very very very difficult to implement correctly.
- LAC-Tech 2y ago
  It's more shifting the view point. CRDTs are not just something you 'use'. They are laws a system must be obey to achieve strong eventual consistency. Eventual Consistency: If Node A and Node B have received the same set of events (ie in any order), they will eventually have the same state. Strong eventual consistency: Exactly the same, but replace "eventually" with "immediately". Meaning as long as all nodes get the same events, they'll have the same state, straight away. That's sync that works. As long as your merge algorithm is commutative, associative, and atomic, your sync will work. That's what the CRDT people uncovered. A data structure can obey these laws (the aforementioned CRDT libraries) A database can obey these laws (the original Amazon Dynamo did this, it was a CRDT). Any arbitrary system can obey these laws (there's 1982 paper described this for a distributed file system)
  - klysm 2y ago
    Okay I’m with you, but if I’m not mistaken I believe you have slightly mis-characterized what strong eventual consistency is. > Strong eventual consistency: Exactly the same, but replace "eventually" with "immediately". I believe this isn’t quite correct. I was under the impression that the delivery doesn’t have to be immediate, but rather that any two nodes with the same set of events, regardless of received order, must be in the same state.
    - LAC-Tech 2y ago
      https://inria.hal.science/hal-00932836/file/CRDTs_SSS-2011.pdf https://inria.hal.science/hal-00932836/file/CRDTs_SSS-2011.p... I'm taking my definitions from section 2.2 here. I feel like I've summarised it fairly accurately, but if I've made a mistake would be happy to be corrected.

klysm 2y ago

I’m not really sure what you’re saying. Do you mean you achieve the gaurantees of CRDTs without using CRDTs? What technology are you implicitly referring to? Whenever I hear someone say “sync” data, I instantly get scared. Consensus is fraught with peril and very very very difficult to implement correctly.

LAC-Tech 2y ago
It's more shifting the view point. CRDTs are not just something you 'use'. They are laws a system must be obey to achieve strong eventual consistency. Eventual Consistency: If Node A and Node B have received the same set of events (ie in any order), they will eventually have the same state. Strong eventual consistency: Exactly the same, but replace "eventually" with "immediately". Meaning as long as all nodes get the same events, they'll have the same state, straight away. That's sync that works. As long as your merge algorithm is commutative, associative, and atomic, your sync will work. That's what the CRDT people uncovered. A data structure can obey these laws (the aforementioned CRDT libraries) A database can obey these laws (the original Amazon Dynamo did this, it was a CRDT). Any arbitrary system can obey these laws (there's 1982 paper described this for a distributed file system)
- klysm 2y ago
  Okay I’m with you, but if I’m not mistaken I believe you have slightly mis-characterized what strong eventual consistency is. > Strong eventual consistency: Exactly the same, but replace "eventually" with "immediately". I believe this isn’t quite correct. I was under the impression that the delivery doesn’t have to be immediate, but rather that any two nodes with the same set of events, regardless of received order, must be in the same state.
  - LAC-Tech 2y ago
    https://inria.hal.science/hal-00932836/file/CRDTs_SSS-2011.pdf https://inria.hal.science/hal-00932836/file/CRDTs_SSS-2011.p... I'm taking my definitions from section 2.2 here. I feel like I've summarised it fairly accurately, but if I've made a mistake would be happy to be corrected.