3 ms·
I'll try to expose again my idea that somebody should try to use fast (and cheap) local LLMs to resolve conflicts. So you have a stateless CRDT, and when confli
by antirez 3y ago
I'll try to expose again my idea that somebody should try to use fast (and cheap) local LLMs to resolve conflicts. So you have a stateless CRDT, and when conflict updates reach the server, you ask the LLM how to merge them. You mark the merge with some attribute so that users will see it with a different background or alike to review to check if it's sane.
- yowlingcat 3y agoCouldn't this make merge behavior of the same conflict non-deterministic? If there was an LLM equivalent of a seed for an RNG, I think that would avoid this issue but I'm not sure if there is.
- antirez 3y agoSure, you can do it in a deterministic way just setting the temperature to zero and writing the prompt always in the same way. If T=0 (you always select top scoring token) and prompt is identical, the result is always the same.
- yowlingcat 3y agoAhh, that's a fantastic callout. I've experimented with low temperature so far but never zero temperature. Makes sense, and I think this is definitely a technique I'll use in the future. Thank you for the legendary work on Redis by the way! It's been an invaluable part of multiple large production systems I've built and is such a well-built tool. I'm hoping from your interest in this topic (and some of the blog posts you've written about it) that some part of the roadmap for Redis might help power LLM workloads in the future.
- antirez 3y agoThanks for the nice words! Appreciated, and good hacking!
- jsunderland323 3y agoThere is a way you could make it deterministic. You could basically let the LLM select the resolution edits and preserve that. Just as you could write a bash script to let an LLM handle your git merges. But then we go back to why? This seems like a really high risk thing to do, since at the point of conflict you really do want to know intent. While copilot is great on the code editing level I’m not sure I’d want 20% of my conflict resolutions to be the opposite of what I intended.
- immibis 3y agoThere is one. LLMs are only non-random because they use an RNG. If you use a seeded PRNG then you get deterministic behaviour.
- jitl 3y agoI want to try it out, but I imagine I’d only go through the LLM flow using a heuristic or explicit user action around branch/merge workflow from a long offline editor. I’m not sure I see a reason for more “online” realtime collaboration to prefer LLM intervention. For realtime it’s much more important things converge so we can just send little updates, and if you need things to converge deterministically I don’t think you can use an LLM. I wonder when VS Code Copilot will start suggesting git rebase solutions.