4 ms·

Strong Eventual Consistency – The Big Idea Behind CRDTs

35 points by todsacerdoti 1y ago

cryptonector 1y ago
What is the monoid for table primary keys?
- giovannibonetti 1y ago
  Vector clocks, since they keep a separate counter for each client
- BoiledCabbage 1y ago
  Im guessing generating unique ids instead of incremental numbers? Maybe not as long as uuids, but long enough to be comfortable they won't conflict withing your table/db. Those will merge fine as two separate "rows".
  - cryptonector 1y ago
    No, because primary keys are things like names, not random numbers.
ktall 1y ago
[dead]
an0malous 1y ago
We need a true peer-to-peer Internet instead of the centralized, feudal system we currently have. CRDTs would allow for peers to own their data and collaborate without centralized servers owned by Big Tech, but even the network itself goes through a small number of pipes owned by telecom companies that can decide what gets through and when. Decentralization is the real big idea behind CRDTs.
- gobdovan 1y ago
  That's like saying hash tables will fix cloud computing monopolies. They're really neat for collaborative tools at application level, but it doesn't solve any of the infrastructure problems. They're not even the missing piece in decentralized infra either. There are alternatives that work as efficiently.
  - an0malous 1y ago
    > That's like saying hash tables will fix cloud computing monopolies. No it’s not because there’s nothing about hash tables that enables peer to peer collaboration over centralization. CRDTs do because you don’t need a “master” or “central” source of truth anymore. > They're not even the missing piece in decentralized infra either. Then what is? I didn’t say they were anyway though, that’s why I mentioned network infrastructure as another constraint.
Jtsummers 1y ago
This one isn't a dupe, it came back through the second chance pool but in the meantime a second submission did take and got a lot of comments: https://news.ycombinator.com/item?id=45177518 https://news.ycombinator.com/item?id=45177518 - 87 comments
- trosenbaum 1y ago
  This sequence of events ended up very relevant, given the subject matter. :D
lamp_book 1y ago
We really ought to come up with some new terms in this space... it is just confusing to mix "strong consistency" in with "eventual consistency" in any way.
- toomim 1y ago
  Why is that confusing? This "Strong Eventual Considtency" is the defining property of the CRDT. Do you have a less-confusing way to think about that property?
pradn 1y ago
A common pattern I see is data-plane nodes receiving versioned metadata updates from the control-plane. As long as the version is higher than the node's previous one, it's correct to use. So, the metadata is a sort of monotonic counter with a bag of data attached to it. This pattern produces a monotonic counter, which I assume is a naive sort of CRDT - though the data itself doesn't benefit from CRDT merge semantics. In this world, as long as a node gets an update, its going to incorporate it into its state. In the article's terms, the system has Strong Convergence. I'm trying to figure out under what practical circumstances updates would result in Eventual Convergence, not Strong Convergence. Wouldn't a node incorporate an update as soon as you receive it? What's causes the "eventual" behavior even after a node gets an update? It seems to me the trouble is actually getting the update, not the data model as such. Yes, I realize partial orders are possible, making it impossible to merge certain sequences of updates for certain models. CRDTs solve that, as they're designed to do. (Though I hear that, for some CRDTs, merges might result in bad "human" results even if the merge operation follows all the CRDT rules.)

35 points by todsacerdoti 1y ago

cryptonector 1y ago

What is the monoid for table primary keys?

giovannibonetti 1y ago
Vector clocks, since they keep a separate counter for each client
BoiledCabbage 1y ago
Im guessing generating unique ids instead of incremental numbers? Maybe not as long as uuids, but long enough to be comfortable they won't conflict withing your table/db. Those will merge fine as two separate "rows".
- cryptonector 1y ago
  No, because primary keys are things like names, not random numbers.

ktall 1y ago

[dead]

an0malous 1y ago

We need a true peer-to-peer Internet instead of the centralized, feudal system we currently have. CRDTs would allow for peers to own their data and collaborate without centralized servers owned by Big Tech, but even the network itself goes through a small number of pipes owned by telecom companies that can decide what gets through and when. Decentralization is the real big idea behind CRDTs.

gobdovan 1y ago
That's like saying hash tables will fix cloud computing monopolies. They're really neat for collaborative tools at application level, but it doesn't solve any of the infrastructure problems. They're not even the missing piece in decentralized infra either. There are alternatives that work as efficiently.
- an0malous 1y ago
  > That's like saying hash tables will fix cloud computing monopolies. No it’s not because there’s nothing about hash tables that enables peer to peer collaboration over centralization. CRDTs do because you don’t need a “master” or “central” source of truth anymore. > They're not even the missing piece in decentralized infra either. Then what is? I didn’t say they were anyway though, that’s why I mentioned network infrastructure as another constraint.

Jtsummers 1y ago

This one isn't a dupe, it came back through the second chance pool but in the meantime a second submission did take and got a lot of comments: https://news.ycombinator.com/item?id=45177518 https://news.ycombinator.com/item?id=45177518 - 87 comments

trosenbaum 1y ago
This sequence of events ended up very relevant, given the subject matter. :D

lamp_book 1y ago

We really ought to come up with some new terms in this space... it is just confusing to mix "strong consistency" in with "eventual consistency" in any way.

toomim 1y ago
Why is that confusing? This "Strong Eventual Considtency" is the defining property of the CRDT. Do you have a less-confusing way to think about that property?

pradn 1y ago

A common pattern I see is data-plane nodes receiving versioned metadata updates from the control-plane. As long as the version is higher than the node's previous one, it's correct to use. So, the metadata is a sort of monotonic counter with a bag of data attached to it. This pattern produces a monotonic counter, which I assume is a naive sort of CRDT - though the data itself doesn't benefit from CRDT merge semantics. In this world, as long as a node gets an update, its going to incorporate it into its state. In the article's terms, the system has Strong Convergence. I'm trying to figure out under what practical circumstances updates would result in Eventual Convergence, not Strong Convergence. Wouldn't a node incorporate an update as soon as you receive it? What's causes the "eventual" behavior even after a node gets an update? It seems to me the trouble is actually getting the update, not the data model as such. Yes, I realize partial orders are possible, making it impossible to merge certain sequences of updates for certain models. CRDTs solve that, as they're designed to do. (Though I hear that, for some CRDTs, merges might result in bad "human" results even if the merge operation follows all the CRDT rules.)