4 ms·

Sync is still an unsolved problem. The solutions that claim to do it aren't very satisfactory.

swiftcoder 2y ago
Fully-generic character-level textual sync is an unsolved problem (and likely, always will be). That's also not typically the best way to sync one's concrete data models, and if your data has any sort of structure, you can likely produce a merge operation that works Well Enough™ in practice
- sausagefeet 2y ago
  IME, and experiences may vary, "Well Enough" merge function usually means corrupt data, confusing outputs, incoherent semantics, and frustrating debugging sessions at any kind of scale beyond playing around. YMMV.
  - anonzzzies 2y ago
    I find actual lock on edit or last-save-wins to work best of what I have seen for structured data. You can add nice visual tools that show who is editing (so you can wait for them) and pass the object lock to someone else etc, but this way at least there is no merge mechanism (that generally knows very little about the ins/outs of the data) which will merge two semantically incompatible fields into one record. For offline-first, the user would be alerted that someone changed the record(s) while you changed them as well and ask what to do. In attempts to make these things for an offline first healthcare app which is mostly structured records based, I found that it is very rare for 2 (or more) to work on the same data, but I guess it really depends on the use case and with unstructured data it's not a nice way of working.
    - dontlaugh 2y ago
      This is what people actually do in practice when collaborating on large non-code documents. The example I’m familiar with is assets in the video games industry. Everyone used Perforce (or maybe SVN) and artists/designers lock the 3D model or whatever they’re editing. People ask each other to give up a lock all the time.
    - yencabulator 2y ago
      > For offline-first, the user would be alerted that someone changed the record(s) while you changed them as well and ask what to do. There are few sync strategies worse than "you worked hard on this, but now you have to choose to either throw away your change or everyone else's changes". May any product doing that fail miserably.
      - anonzzzies 2y ago
        And your great solution? As others say; there are none. Talking about structured content in an offline first context. What is your strategy that won't break anything while doing a merge?
  - InsideOutSanta 2y ago
    Just wait until the vibe coders use an LLM to decide which data in a merge conflict it likes better.
mentalgear 2y ago
Reliable Sync from a syntactic (e.g. rich text) or data structure level is possible already with CRDTs, there are many excellent solutions (y.js, automerge etc.). Semantic sync (for example when 2 people have modified a draft of a novel and want to sync): how to auto sync both versions so the resulting text makes sense and has no duplicates, is indeed much harder, and requires manual revisions (like git revisions), possibly with LLM suggestions.
- immibis 2y ago
  Text editing CRDTs fall in the category of things that are technically possible, in an incredibly useless way - just like it's possible to automatically "resolve" every Git merge conflict by randomly splatting the conflicting lines around in the file.
  - baq 2y ago
    CRDTs with LLM suggested conflict resolution might just be the thing. Next best thing is keeping conflicts unresolved, but marked as such as a first-class citizen of the document content (see pijul, jj).
    - threatofrain 2y ago
      But then you don’t get convergence which is a requirement and LLMs will be just as good for Git as they will be for CRDTs.
      - baq 2y ago
        I postulate they may handle written language quite well, within reasonable bounds. Re convergence, turns out it's much more convenient to let conflict resolution be deferred.
      - yencabulator 2y ago
        You can get convergence with the same mechanism that OT uses for compaction: assign a special server role that does it.
isaachinman 2y ago
Not at all true. One good solution exists.

Fully-generic character-level textual sync is an unsolved problem (and likely, always will be). That's also not typically the best way to sync one's concrete data models, and if your data has any sort of structure, you can likely produce a merge operation that works Well Enough™ in practice

sausagefeet 2y ago
IME, and experiences may vary, "Well Enough" merge function usually means corrupt data, confusing outputs, incoherent semantics, and frustrating debugging sessions at any kind of scale beyond playing around. YMMV.
- anonzzzies 2y ago
  I find actual lock on edit or last-save-wins to work best of what I have seen for structured data. You can add nice visual tools that show who is editing (so you can wait for them) and pass the object lock to someone else etc, but this way at least there is no merge mechanism (that generally knows very little about the ins/outs of the data) which will merge two semantically incompatible fields into one record. For offline-first, the user would be alerted that someone changed the record(s) while you changed them as well and ask what to do. In attempts to make these things for an offline first healthcare app which is mostly structured records based, I found that it is very rare for 2 (or more) to work on the same data, but I guess it really depends on the use case and with unstructured data it's not a nice way of working.
  - dontlaugh 2y ago
    This is what people actually do in practice when collaborating on large non-code documents. The example I’m familiar with is assets in the video games industry. Everyone used Perforce (or maybe SVN) and artists/designers lock the 3D model or whatever they’re editing. People ask each other to give up a lock all the time.
  - yencabulator 2y ago
    > For offline-first, the user would be alerted that someone changed the record(s) while you changed them as well and ask what to do. There are few sync strategies worse than "you worked hard on this, but now you have to choose to either throw away your change or everyone else's changes". May any product doing that fail miserably.
    - anonzzzies 2y ago
      And your great solution? As others say; there are none. Talking about structured content in an offline first context. What is your strategy that won't break anything while doing a merge?
- InsideOutSanta 2y ago
  Just wait until the vibe coders use an LLM to decide which data in a merge conflict it likes better.

mentalgear 2y ago

Reliable Sync from a syntactic (e.g. rich text) or data structure level is possible already with CRDTs, there are many excellent solutions (y.js, automerge etc.). Semantic sync (for example when 2 people have modified a draft of a novel and want to sync): how to auto sync both versions so the resulting text makes sense and has no duplicates, is indeed much harder, and requires manual revisions (like git revisions), possibly with LLM suggestions.

immibis 2y ago
Text editing CRDTs fall in the category of things that are technically possible, in an incredibly useless way - just like it's possible to automatically "resolve" every Git merge conflict by randomly splatting the conflicting lines around in the file.
- baq 2y ago
  CRDTs with LLM suggested conflict resolution might just be the thing. Next best thing is keeping conflicts unresolved, but marked as such as a first-class citizen of the document content (see pijul, jj).
  - threatofrain 2y ago
    But then you don’t get convergence which is a requirement and LLMs will be just as good for Git as they will be for CRDTs.
    - baq 2y ago
      I postulate they may handle written language quite well, within reasonable bounds. Re convergence, turns out it's much more convenient to let conflict resolution be deferred.
    - yencabulator 2y ago
      You can get convergence with the same mechanism that OT uses for compaction: assign a special server role that does it.

isaachinman 2y ago

Not at all true. One good solution exists.