4 ms·

Now that there are LLMs available, why is this still a problem? You just have to detect the conflicts and show a powerful enough LLM the two versions and tell i

Now that there are LLMs available, why is this still a problem? You just have to detect the conflicts and show a powerful enough LLM the two versions and tell it to do its best job at merging. This solves exactly the kind of issue described in the post. Oh, bonus point: you don't have to take any metadata.

miltonlost 2y ago
> You just have to detect the conflicts and show a powerful enough LLM the two versions and tell it to do its best job at merging. This solves exactly the kind of issue described in the post. How does "LLM tries its best" solve the problem of exactly syncing documents offline? "Tries its best" implies it could fail which is what the problem already is. An LLM only adds a new layer of abstraction, but now the downside is it's impossible to analyze. Maybe I just have a much higher threshold of "solved" than you do, but anything non-deterministic (without some human judgment a la git) is not a good solve for document syncing. A "powerful enough LLM" is the equivalent of "and a wizard does it".
- antirez 2y ago
  LLMs at t=0 are predictable (will produce the same output starting from the same input) and do a much better merge work than any other non ML based algorithm. Cut & paste all the examples you can come up to Claude and tell me if LLMs are not able to do this kind of merging.
jakelazaroff 2y ago
If there are only two conflicting updates, sure, maybe that would work. But there might be more than two peers, and their updates might arrive in any order. That's why CRDT merges must be both commutative and associative. Can we guarantee that an LLM will get exactly the same result merging (A ∨ B) ∨ C as it would merging A ∨ (B ∨ C)? Even when the temperature is 0?
- mweidner 2y ago
  You could avoid the CRDT rules if you only use the LLM on the server. I.e., user comes online and sends their diff to the server, which LLM-merges it into the latest state and then sends that back to all clients. This doesn't help you do merges client-side during live collaboration (for showing your optimistic local updates), but there the low latency reduces conflicts anyway, so you can fall back on a semantically-imperfect CRDT.
  - satvikpendem 2y ago
    If you have a central server, you don't need CRDTs, which are designed to work even in pure peer-to-peer scenarios. Figma is one example of this [0]: > Figma isn't using true CRDTs though. CRDTs are designed for decentralized systems where there is no single central authority to decide what the final state should be. There is some unavoidable performance and memory overhead with doing this. Since Figma is centralized (our server is the central authority), we can simplify our system by removing this extra overhead and benefit from a faster and leaner implementation. > It’s also worth noting that Figma's data structure isn't a single CRDT. Instead it's inspired by multiple separate CRDTs and uses them in combination to create the final data structure that represents a Figma document (described below). [0] https://www.figma.com/blog/how-figmas-multiplayer-technology-works/ https://www.figma.com/blog/how-figmas-multiplayer-technology...
satvikpendem 2y ago
That's exactly how I see it too, true intelligent merging algorithms are, at the limit, going to be basically LLMs because only something that powerful can understand user intent in a way that non-ML algorithms cannot.
antics 2y ago
Author here! This comment is kind of getting dragged here and elsewhere but I actually think it's not completely ridiculous. You can (and we have) presented git-style merge conflicts to an LLM and it will mostly fix them in ways that no algorithm can. One example of this is if you and I both add a similar sentence in different spots in the document, asking an LLM to merge this will often result in only one of the sentences being accepted. It's not perfect but it's the kind of thing you can't get in any other technology! With that all said I don't think LLMs REPLACE merge algorithms. For one, to get sensible output from the LLMs you generally need a diff of some kind, either git style or as the trace ouput of something like eg-walker.

miltonlost 2y ago

> You just have to detect the conflicts and show a powerful enough LLM the two versions and tell it to do its best job at merging. This solves exactly the kind of issue described in the post. How does "LLM tries its best" solve the problem of exactly syncing documents offline? "Tries its best" implies it could fail which is what the problem already is. An LLM only adds a new layer of abstraction, but now the downside is it's impossible to analyze. Maybe I just have a much higher threshold of "solved" than you do, but anything non-deterministic (without some human judgment a la git) is not a good solve for document syncing. A "powerful enough LLM" is the equivalent of "and a wizard does it".

antirez 2y ago
LLMs at t=0 are predictable (will produce the same output starting from the same input) and do a much better merge work than any other non ML based algorithm. Cut & paste all the examples you can come up to Claude and tell me if LLMs are not able to do this kind of merging.

jakelazaroff 2y ago

If there are only two conflicting updates, sure, maybe that would work. But there might be more than two peers, and their updates might arrive in any order. That's why CRDT merges must be both commutative and associative. Can we guarantee that an LLM will get exactly the same result merging (A ∨ B) ∨ C as it would merging A ∨ (B ∨ C)? Even when the temperature is 0?

mweidner 2y ago
You could avoid the CRDT rules if you only use the LLM on the server. I.e., user comes online and sends their diff to the server, which LLM-merges it into the latest state and then sends that back to all clients. This doesn't help you do merges client-side during live collaboration (for showing your optimistic local updates), but there the low latency reduces conflicts anyway, so you can fall back on a semantically-imperfect CRDT.
- satvikpendem 2y ago
  If you have a central server, you don't need CRDTs, which are designed to work even in pure peer-to-peer scenarios. Figma is one example of this [0]: > Figma isn't using true CRDTs though. CRDTs are designed for decentralized systems where there is no single central authority to decide what the final state should be. There is some unavoidable performance and memory overhead with doing this. Since Figma is centralized (our server is the central authority), we can simplify our system by removing this extra overhead and benefit from a faster and leaner implementation. > It’s also worth noting that Figma's data structure isn't a single CRDT. Instead it's inspired by multiple separate CRDTs and uses them in combination to create the final data structure that represents a Figma document (described below). [0] https://www.figma.com/blog/how-figmas-multiplayer-technology-works/ https://www.figma.com/blog/how-figmas-multiplayer-technology...

satvikpendem 2y ago

That's exactly how I see it too, true intelligent merging algorithms are, at the limit, going to be basically LLMs because only something that powerful can understand user intent in a way that non-ML algorithms cannot.

antics 2y ago

Author here! This comment is kind of getting dragged here and elsewhere but I actually think it's not completely ridiculous. You can (and we have) presented git-style merge conflicts to an LLM and it will mostly fix them in ways that no algorithm can. One example of this is if you and I both add a similar sentence in different spots in the document, asking an LLM to merge this will often result in only one of the sentences being accepted. It's not perfect but it's the kind of thing you can't get in any other technology! With that all said I don't think LLMs REPLACE merge algorithms. For one, to get sensible output from the LLMs you generally need a diff of some kind, either git style or as the trace ouput of something like eg-walker.