3 ms·
You mean it reacts to you writing something while it's thinking of that you can stop it while it's thinking?
by Gooblebrai 2y ago
You mean it reacts to you writing something while it's thinking of that you can stop it while it's thinking?
- hmottestad 2y agoYou can stop it at any time, then modify what it's written so far...then press continue and let it continue thinking and answering.
- thot_experiment 2y agoFundamentally the UI is up to you, I have a "typing-pauses-inference-and-starts-gaslighting" feature in my homebrew frontend, but in OpenWebUI/Sillytavern you can just pause it and edit the chain of thought and then have it continue from the edit.
- Gracana 2y agoThat's a great idea. In your frontend, do you write in the same text entry field as the bot? I use oobabooga/text-generation-webui and I findit's a little awkward to edit the bot responses.
- thot_experiment 2y agoNo, but the chat divs are all contenteditable.
- Gracana 2y agoOh! That is an excellent solution. I wish it was that easy in every UI.
- thot_experiment 2y agoThanks, for what it's worth unless you particularly need to use exl2 ollama works great for local inference and you can prompt together a half decent chat UI for yourself in a matter of minutes these days which gives you full control over everything. I also lean a lot on https://www.npmjs.com/package/amallo https://www.npmjs.com/package/amallo which is a api wrapper i wrote for ollama which makes this sort of hacking very very easy. (not that the default lib is bad, i just didn't like the ergonomics)