4 ms·
Pro tip: disable the title generation feature or set it to another model on another system. After every chat, open webui is sending everything to llamacpp agai
by idonotknowwhy 1y ago
Pro tip: disable the title generation feature or set it to another model on another system.
After every chat, open webui is sending everything to llamacpp again wrapped in a prompt to generate the summary, and this wipes out the KV cache, forcing you to reprocess the entire context.
This will get rid of the long prompt processing times id you're having long back and forth chats with it.