4 ms·

I might be missing something, but I have trouble seeing how CRDTs can work for regular CRUD style applications. Just the first example that pops into my head:

I might be missing something, but I have trouble seeing how CRDTs can work for regular CRUD style applications. Just the first example that pops into my head: edit A sets an invoice status to paid, edit b changes the invoice amount from 100 to 120. The merge is a paid invoice with an incorrect amount. A workaround would be to record a separate PaidInvoice that wont be changed by the application logic. But that's just a really trivial example that only involves scalar fields inside a single object, and also relies on the application logic considering all ways that the CRDT might behave. There are countless ways to end up with data that violates the constraints of the domain. Is there any theoretical groundwork happening on how CRDTs can preserve domain semantics?

olah_1 5y ago
I think CRDTs necessitate a robust permission system to be more than a toy. If you want a decentralized CRDT, you'd probably end up re-inventing smart contracts (which is what I found myself doing once).
- sagichmal 5y ago
  CRDTs and smart contracts are almost totally unrelated to each other.
ollysb 5y ago
Your general point withstanding, in this case wouldn't the merge be a partially paid invoice?
- jeremyjh 5y ago
  No, its over-payment that no one knows was over-paid.
josephg 5y ago
Thats a great example. In this specific case, you'd want to associate the invoice status (paid) with the point in time when that change was made so concurrent changes to the invoice result in merge conflicts. I don't know anyone who's building a CRDT like that; but its definitely possible. Basho's RIAK databases had a CRDT implementation that would provide a grounding to handle cases like that. When concurrent changes happen, the database simply stored each conflicting value. It was up to the application to decide how the conflicting values merged together. On top of something like that, you can imagine a bunch of different merging strategies which would be appropriate in different situations: Optimistic (recursively merge all children), Arbitrary-writer-wins, or Conflict-and-bother-the-user. In this case, generating a conflict state is appropriate. Conflicts also make sense when merging branches in git; and I'd love to see a CRDT which managed to reproduce that functionality. But no, I don't know of any work happening in that space. Maybe the inventor could be you!
- infogulch 5y ago
  > When concurrent changes happen, the database simply stored each conflicting value. Yes, this is the key! 1. store everything, 2. give the domain modelers an array of strategies to merge particular cases, with the default being conflict-and-bother-the-user. Every existing merge strategy can be represented on top of this model, so it should be the base layer. The conflict resolution strategies can be sliced up, layered, hidden, or exposed to the user according to the design criteria because conflict resolution is part the problem domain. I would go so far as to say that assuming one particular merge strategy for all data structures as 'correct' is actively harmful. What is the worst case outcome in the example above? That nobody looks at it. In a general case like this choosing any default action at all is the wrong call because both edits were made with different assumptions and contexts, and that's the level where the conflict should be resolved, not down here in the invoice.
TeMPOraL 5y ago
I know nothing about CRDTs, but I feel the ultimate problem with your invoice example might be a representation issue. > edit A sets an invoice status to paid, edit b changes the invoice amount from 100 to 120. The merge is a paid invoice with an incorrect amount. There is no such thing as "paid status" of an invoice. That's a simplification. If you unroll it, what you get is accounts that must balance. The invoice starts with "accounts payable"[0] set to 100$, and no payment covering it. Edit A adds a transaction from company account to AP to the value of $100. Edit B changes the liability in AP to $120. The result of a merge should be a transaction record for $100 and liability of $120, meaning a partially-paid invoice. -- [0] - Or whatever it is called, I keep getting confused by this terminology, as it's not something regular people use in my part of the world.
sagichmal 5y ago
> Is there any theoretical groundwork happening on how CRDTs can preserve domain semantics? Effectively using CRDTs requires a fundamentally different model of state. Typical CRUD stuff simply doesn't work. If you're unwilling or unable to refactor your state layer, then CRDTs are gonna seem totally infeasible.

olah_1 5y ago

I think CRDTs necessitate a robust permission system to be more than a toy. If you want a decentralized CRDT, you'd probably end up re-inventing smart contracts (which is what I found myself doing once).

sagichmal 5y ago
CRDTs and smart contracts are almost totally unrelated to each other.

ollysb 5y ago

Your general point withstanding, in this case wouldn't the merge be a partially paid invoice?

jeremyjh 5y ago
No, its over-payment that no one knows was over-paid.

josephg 5y ago

Thats a great example. In this specific case, you'd want to associate the invoice status (paid) with the point in time when that change was made so concurrent changes to the invoice result in merge conflicts. I don't know anyone who's building a CRDT like that; but its definitely possible. Basho's RIAK databases had a CRDT implementation that would provide a grounding to handle cases like that. When concurrent changes happen, the database simply stored each conflicting value. It was up to the application to decide how the conflicting values merged together. On top of something like that, you can imagine a bunch of different merging strategies which would be appropriate in different situations: Optimistic (recursively merge all children), Arbitrary-writer-wins, or Conflict-and-bother-the-user. In this case, generating a conflict state is appropriate. Conflicts also make sense when merging branches in git; and I'd love to see a CRDT which managed to reproduce that functionality. But no, I don't know of any work happening in that space. Maybe the inventor could be you!

infogulch 5y ago
> When concurrent changes happen, the database simply stored each conflicting value. Yes, this is the key! 1. store everything, 2. give the domain modelers an array of strategies to merge particular cases, with the default being conflict-and-bother-the-user. Every existing merge strategy can be represented on top of this model, so it should be the base layer. The conflict resolution strategies can be sliced up, layered, hidden, or exposed to the user according to the design criteria because conflict resolution is part the problem domain. I would go so far as to say that assuming one particular merge strategy for all data structures as 'correct' is actively harmful. What is the worst case outcome in the example above? That nobody looks at it. In a general case like this choosing any default action at all is the wrong call because both edits were made with different assumptions and contexts, and that's the level where the conflict should be resolved, not down here in the invoice.

TeMPOraL 5y ago

I know nothing about CRDTs, but I feel the ultimate problem with your invoice example might be a representation issue. > edit A sets an invoice status to paid, edit b changes the invoice amount from 100 to 120. The merge is a paid invoice with an incorrect amount. There is no such thing as "paid status" of an invoice. That's a simplification. If you unroll it, what you get is accounts that must balance. The invoice starts with "accounts payable"[0] set to 100$, and no payment covering it. Edit A adds a transaction from company account to AP to the value of $100. Edit B changes the liability in AP to $120. The result of a merge should be a transaction record for $100 and liability of $120, meaning a partially-paid invoice. -- [0] - Or whatever it is called, I keep getting confused by this terminology, as it's not something regular people use in my part of the world.

sagichmal 5y ago

> Is there any theoretical groundwork happening on how CRDTs can preserve domain semantics? Effectively using CRDTs requires a fundamentally different model of state. Typical CRUD stuff simply doesn't work. If you're unwilling or unable to refactor your state layer, then CRDTs are gonna seem totally infeasible.