4 ms·

3 Years in and we still dont have a useable chat fork in any of the major LLM chatbots providers. Seems like the only way to explore differnt outcomes is by ed

3 Years in and we still dont have a useable chat fork in any of the major LLM chatbots providers. Seems like the only way to explore differnt outcomes is by editing messages and losing whatever was there before the edit. Very annoying and I dont understand why they all refuse to implement such a simple feature.

__float 1y ago
Maybe this suggests it's not such a simple feature?
- LeoPanthera 1y ago
  LM Studio has this feature for local models and it works just fine.
- mccoyb 1y ago
  A perusal of the source code of, say, Ollama -- or the agentic harnesses of Crush / OpenCode -- will convince you that yes, this should be an extremely a simple feature (management of contexts are part and parcel). Also, these companies have the most advanced agentic coding systems on the planet. It should be able to fucking implement tree-like chat ...
- nomel 1y ago
  If the client supports chat history, that you can resume a conversation, it has everything required, and it's literally just a chat history organization problem, at that point.
amrrs 1y ago
Google AI Studio allows you to branch from a point in any conversation
- ZeroCool2u 1y ago
  Yeah, I think this is the best version of the branching interface I've seen.
- dwringer 1y ago
  This isn't quite the same as being able to edit an earlier post without discarding the subsequent ones, creating a context where the meaning of subsequent messages could be interpreted quite differently and leading to different responses later down the chain. Ideally I'd like to be able to edit both my replies and the responses at any point like a linear document in managing an ongoing context.
  - dist-epoch 1y ago
    Cherry Studio can do that, allows you to edit both your own and the model responses, but it requires API access.
  - CjHuber 1y ago
    But that's exactly what you can do with AI studio. You can edit any prior messages (then either just saving them at their place in the chat or rerunning them) and you can edit any response of the LLM. Also you can rerun queries within any part of the conversation without the following part of the conversation being deleted or branched
    - dwringer 1y ago
      Ah - I appreciate the clarification! Apologies for my misunderstanding. Guess that's something I need to check out.
deelowe 1y ago
Is it simple? Maintaining context seems extremely difficult with LLMs.
nomel 1y ago
This why I use a locally hosted LibreChat. It doesn't having merging though, which would be tricky, and probably require summarization. I would also really like to see a mode that colors by top-n "next best" ratio, or something similar.
scribu 1y ago
ChatGPT Plus has that (used to be in the free tier too). You can toggle between versions for each of your messages with little left-right arrows.
typpilol 1y ago
Copilot in vscode has checkpoints now which are similar They let you rollback to the previous conversation state
storus 1y ago
DeepSeek.com has it. You just edit a previous question and the old conversation is stored and can be resumed.
jatora 1y ago
Chatgpt has this baked in, as you can revert branches after editing, they just dont make it easy to traverse. This chrome extension used to work to allow you to traverse the tree: https://chromewebstore.google.com/detail/chatgpt-conversation-tree/dikfamiegodmdamfmhecejnlahclddom https://chromewebstore.google.com/detail/chatgpt-conversatio... I copied it a while ago and maintain my own version but it isnt on the store, just for personal use. I assume they dont implement it because it is such a niche user that wants this and so isnt worth the UI distraction
- ToValueFunfetti 1y ago
  >they just dont make it easy to traverse I needed to pull some detail from a large chat with many branches and regenerations the other day. I remembered enough context that I had no problem using search and finding the exact message I needed. And then I clicked on it and arrived at the bottom of the last message in final branch of the tree. From there, you scroll up one message, hover to check if there are variants, and recursively explore branches as they arise. I'd love to have a way to view the tree and I'd settle for a functional search.
- BriggyDwiggs42 1y ago
  Do you have your version up on github?
james2doyle 1y ago
I use https://chatwise.app/ https://chatwise.app/ and it has this in the form of "start new chat from here" on messages
martin-t 1y ago
> why they all refuse to implement such a simple feature Because it would let you peek behind the smoke and mirrors. Why do you think there's a randomized seed you can't touch?
trenchpilgrim 1y ago
Kagi Assistant and Claude Code both have chat forking that works how you want.
- CjHuber 1y ago
  I guess you mean normal Claude? What really annoys me with it is that when you attach a document you can't delete it in a branch, so you have to rerun the previous message so that its gone
  - trenchpilgrim 1y ago
    No, claude code. Double tap ESC.
    - CjHuber 1y ago
      But as far as I know that is reverting and the current state of the conversation is lost?
benreesman 1y ago
It is unfortunate that pretty basic "save/load" functionality is still spotty and underdocumented, seems pretty critical. I use gptel and a folder full of markdown with some light automation to get an adequate approximation of this, but it really should be built in (it would be more efficient for the vendors as well, tons of cache optimization opportunitirs).

__float 1y ago

Maybe this suggests it's not such a simple feature?

LeoPanthera 1y ago
LM Studio has this feature for local models and it works just fine.
mccoyb 1y ago
A perusal of the source code of, say, Ollama -- or the agentic harnesses of Crush / OpenCode -- will convince you that yes, this should be an extremely a simple feature (management of contexts are part and parcel). Also, these companies have the most advanced agentic coding systems on the planet. It should be able to fucking implement tree-like chat ...
nomel 1y ago
If the client supports chat history, that you can resume a conversation, it has everything required, and it's literally just a chat history organization problem, at that point.

amrrs 1y ago

Google AI Studio allows you to branch from a point in any conversation

ZeroCool2u 1y ago
Yeah, I think this is the best version of the branching interface I've seen.
dwringer 1y ago
This isn't quite the same as being able to edit an earlier post without discarding the subsequent ones, creating a context where the meaning of subsequent messages could be interpreted quite differently and leading to different responses later down the chain. Ideally I'd like to be able to edit both my replies and the responses at any point like a linear document in managing an ongoing context.
- dist-epoch 1y ago
  Cherry Studio can do that, allows you to edit both your own and the model responses, but it requires API access.
- CjHuber 1y ago
  But that's exactly what you can do with AI studio. You can edit any prior messages (then either just saving them at their place in the chat or rerunning them) and you can edit any response of the LLM. Also you can rerun queries within any part of the conversation without the following part of the conversation being deleted or branched
  - dwringer 1y ago
    Ah - I appreciate the clarification! Apologies for my misunderstanding. Guess that's something I need to check out.

deelowe 1y ago

Is it simple? Maintaining context seems extremely difficult with LLMs.

nomel 1y ago

This why I use a locally hosted LibreChat. It doesn't having merging though, which would be tricky, and probably require summarization. I would also really like to see a mode that colors by top-n "next best" ratio, or something similar.

scribu 1y ago

ChatGPT Plus has that (used to be in the free tier too). You can toggle between versions for each of your messages with little left-right arrows.

typpilol 1y ago

Copilot in vscode has checkpoints now which are similar They let you rollback to the previous conversation state

storus 1y ago

DeepSeek.com has it. You just edit a previous question and the old conversation is stored and can be resumed.

jatora 1y ago

Chatgpt has this baked in, as you can revert branches after editing, they just dont make it easy to traverse. This chrome extension used to work to allow you to traverse the tree: https://chromewebstore.google.com/detail/chatgpt-conversation-tree/dikfamiegodmdamfmhecejnlahclddom https://chromewebstore.google.com/detail/chatgpt-conversatio... I copied it a while ago and maintain my own version but it isnt on the store, just for personal use. I assume they dont implement it because it is such a niche user that wants this and so isnt worth the UI distraction

ToValueFunfetti 1y ago
>they just dont make it easy to traverse I needed to pull some detail from a large chat with many branches and regenerations the other day. I remembered enough context that I had no problem using search and finding the exact message I needed. And then I clicked on it and arrived at the bottom of the last message in final branch of the tree. From there, you scroll up one message, hover to check if there are variants, and recursively explore branches as they arise. I'd love to have a way to view the tree and I'd settle for a functional search.
BriggyDwiggs42 1y ago
Do you have your version up on github?

james2doyle 1y ago

I use https://chatwise.app/ https://chatwise.app/ and it has this in the form of "start new chat from here" on messages

martin-t 1y ago

> why they all refuse to implement such a simple feature Because it would let you peek behind the smoke and mirrors. Why do you think there's a randomized seed you can't touch?

trenchpilgrim 1y ago

Kagi Assistant and Claude Code both have chat forking that works how you want.

CjHuber 1y ago
I guess you mean normal Claude? What really annoys me with it is that when you attach a document you can't delete it in a branch, so you have to rerun the previous message so that its gone
- trenchpilgrim 1y ago
  No, claude code. Double tap ESC.
  - CjHuber 1y ago
    But as far as I know that is reverting and the current state of the conversation is lost?

benreesman 1y ago

It is unfortunate that pretty basic "save/load" functionality is still spotty and underdocumented, seems pretty critical. I use gptel and a folder full of markdown with some light automation to get an adequate approximation of this, but it really should be built in (it would be more efficient for the vendors as well, tons of cache optimization opportunitirs).