3 ms·
I think you may be confused about how LLMs work. Editing the history you send to the API works perfectly fine. You don't need the LLM to specifically "re-evalua
by jstanley 1y ago
I think you may be confused about how LLMs work. Editing the history you send to the API works perfectly fine. You don't need the LLM to specifically "re-evaluate" the history from the point you edited.
In a way it is "re-evaluating" the entire history for every token it generates.
- sureglymop 1y agoWhat I'm saying is: If, earlier in the history, I change something, that thing may also still be used further down in the history which I specifically don't want. So yes, I know that I can just do a simple text replacement somewhere in the history but that's not really useful. I want the conversation to be re-evaluated from that point because it might have gone differently with that changed message.
- haiku2077 1y agoI can do this with my assistants in Zed and Kagi just fine? Did a quick test and it works exactly how you describe.
- sureglymop 1y agoNo. When you edit one of your queries in Kagi Assistant, any queries you made in the conversation after that are discarded. Again, I don't just mean simple re-evaluation of a single query.
- haiku2077 1y agoMy previous queries aren't discarded. I can flip between them like branches in a conversation tree.
- justinrubek 1y agoPrecisely. They are discarded from the new query.
- quantadev 1y agoYeah, that's where the 'branching' comes in for sure. Ideally a chatbot would be 'tree-based' where you can just go back to a prior point where you wish you had said something different, and just pickup as if you were back at that point in time, while the other parallel branches are ignored. The way this is done, technically of course, is you build the "Context" by walking back up the tree parent by parent, until you reach the root, building up 'context' as you go along, in reverse order. That will then always be the 'right' context, based on branching.