4 ms·
oobabooga is the king. As far as I know, ollama doesn’t support exllama, qlora fine tuning, multi-GPU, etc. Text-generation-webui might seem like a science pro
by oceanplexian 3y ago
oobabooga is the king.
As far as I know, ollama doesn’t support exllama, qlora fine tuning, multi-GPU, etc. Text-generation-webui might seem like a science project, but it’s leagues ahead (Like 2-4x faster inference with the right plugins) of everything else. Also has a nice openai mock API that works great.
- jstarfish 3y agooobabooga should be king, but it's unstable as hell. ooba's worth keeping an eye on, but koboldcpp is more stable, almost as versatile, and way less frustrating. It also still supports GGML.
- LoganDark 3y agoIn compatibility only. If you want keyboard shortcuts or a snappy interface, Python+gradio just isn't it. https://github.com/oobabooga/text-generation-webui/issues/4138 https://github.com/oobabooga/text-generation-webui/issues/41...