3 ms·

There are many ways to solve each individual problem. The point is that you always have to think about merging behaviour for every piece of state.

fauigerzigerk 1y ago
Yes, sort of like you have to think about your transaction boundaries in server-side code for every single task. The difference is that coming up with a correct CRDT solution for application specific consistency requirements can be a research project. In many cases, no CRDT solution can exist.
- josephg 1y ago
  Can you give some examples? In my experience, 95% of applications are handled just fine by the sort of JSON types built in to Yjs or automerge. The problems I hear people complain about are things like performance, size on disk and library ergonomics. And the long tail of features - like ephemeral data support and binary assets. But data mapping seems mostly fine? I know of a couple of exceptions. Arbitrary nested tree reparenting can be a nightmare. And there aren’t many good rich text implementations out there. What problems are you actually running into?
  - fauigerzigerk 1y ago
    There are two rather different issues. One large class of problems I'm thinking of is simply outside the scope of CRDTs. The whole idea of _eventual_ consistency doesn't really work for things like payment systems or booking systems. A lot of OLTP applications have to be consistent at all times (hence the O). Money must not be double spent. Rooms or seats must not be double booked. The other class of problems is more debatable. CRDTs can guarantee that collaborative text editing results in the same sequence of letters on all nodes. They cannot guarantee that this sequence makes sense. Authors can step on each other's toes. Whether or not this is a problem depends on the specific workflow and I think it could be mitigated by choosing better units of storage/work (such as paragraphs rather than letters).
    - josephg 1y ago
      > One large class of problems I'm thinking of is simply outside the scope of CRDTs. The whole idea of _eventual_ consistency doesn't really work for things like payment systems or booking systems. Yes! I think of it as owned data and shared data. Owned data is data that is owned by one process or node. Eg my bank balance, the position of my mouse cursor, the temperature of my CPU. For this stuff, you don’t want a crdt. Use a database. Or a variable in memory or a file on disk. Broadcast updates if you want, but route all write requests through the data’s owner. Then there’s shared data - like the source code for a project or an apple note. There, CRDTs might make sense - especially if you get branching and merging support along for the ride. > Authors can step on each other's toes. Yeah when merging long lived branches, the workflow most people want is what git provides - of humans manually resolving conflicts. There’s no reason a crdt couldn’t provide this. CRDTs have a superset of the information available to git. It’s weird nobody has coded a system like that up yet.
      - fauigerzigerk 1y ago
        >CRDTs have a superset of the information available to git. It’s weird nobody has coded a system like that up yet. That's an interesting idea. I have to think about this.
      - johnecheck 1y ago
        Conflict-free is right in the name, layering conflicts on top of it would be blasphemy :p
      - sethev 1y ago
        Pijul is a version control system based on a CRDT: https://pijul.org/manual/theory.html#conflicts-and-crdts https://pijul.org/manual/theory.html#conflicts-and-crdts It works like you describe, with humans manually resolving conflicts. The conflicts are represented in the data model, so the data model itself converges without conflicts...if that makes sense.
      - evelant 1y ago
        See my comment below, I prototyped something like this. https://news.ycombinator.com/item?id=45180325 https://news.ycombinator.com/item?id=45180325
naasking 1y ago
> The point is that you always have to think about merging behaviour for every piece of state. CRDTs can't eliminate the requirement to think about what the consistent states are.

fauigerzigerk 1y ago

Yes, sort of like you have to think about your transaction boundaries in server-side code for every single task. The difference is that coming up with a correct CRDT solution for application specific consistency requirements can be a research project. In many cases, no CRDT solution can exist.

josephg 1y ago
Can you give some examples? In my experience, 95% of applications are handled just fine by the sort of JSON types built in to Yjs or automerge. The problems I hear people complain about are things like performance, size on disk and library ergonomics. And the long tail of features - like ephemeral data support and binary assets. But data mapping seems mostly fine? I know of a couple of exceptions. Arbitrary nested tree reparenting can be a nightmare. And there aren’t many good rich text implementations out there. What problems are you actually running into?
- fauigerzigerk 1y ago
  There are two rather different issues. One large class of problems I'm thinking of is simply outside the scope of CRDTs. The whole idea of _eventual_ consistency doesn't really work for things like payment systems or booking systems. A lot of OLTP applications have to be consistent at all times (hence the O). Money must not be double spent. Rooms or seats must not be double booked. The other class of problems is more debatable. CRDTs can guarantee that collaborative text editing results in the same sequence of letters on all nodes. They cannot guarantee that this sequence makes sense. Authors can step on each other's toes. Whether or not this is a problem depends on the specific workflow and I think it could be mitigated by choosing better units of storage/work (such as paragraphs rather than letters).
  - josephg 1y ago
    > One large class of problems I'm thinking of is simply outside the scope of CRDTs. The whole idea of _eventual_ consistency doesn't really work for things like payment systems or booking systems. Yes! I think of it as owned data and shared data. Owned data is data that is owned by one process or node. Eg my bank balance, the position of my mouse cursor, the temperature of my CPU. For this stuff, you don’t want a crdt. Use a database. Or a variable in memory or a file on disk. Broadcast updates if you want, but route all write requests through the data’s owner. Then there’s shared data - like the source code for a project or an apple note. There, CRDTs might make sense - especially if you get branching and merging support along for the ride. > Authors can step on each other's toes. Yeah when merging long lived branches, the workflow most people want is what git provides - of humans manually resolving conflicts. There’s no reason a crdt couldn’t provide this. CRDTs have a superset of the information available to git. It’s weird nobody has coded a system like that up yet.
    - fauigerzigerk 1y ago
      >CRDTs have a superset of the information available to git. It’s weird nobody has coded a system like that up yet. That's an interesting idea. I have to think about this.
    - johnecheck 1y ago
      Conflict-free is right in the name, layering conflicts on top of it would be blasphemy :p
    - sethev 1y ago
      Pijul is a version control system based on a CRDT: https://pijul.org/manual/theory.html#conflicts-and-crdts https://pijul.org/manual/theory.html#conflicts-and-crdts It works like you describe, with humans manually resolving conflicts. The conflicts are represented in the data model, so the data model itself converges without conflicts...if that makes sense.
    - evelant 1y ago
      See my comment below, I prototyped something like this. https://news.ycombinator.com/item?id=45180325 https://news.ycombinator.com/item?id=45180325

naasking 1y ago

> The point is that you always have to think about merging behaviour for every piece of state. CRDTs can't eliminate the requirement to think about what the consistent states are.