4 ms·
Best in what sense? Seems by description a whole lot less convenient (and more limited) if you want to use more than one model (and especially more than just mo
by dragonwriter 3y ago
Best in what sense? Seems by description a whole lot less convenient (and more limited) if you want to use more than one model (and especially more than just models available as GGUFs) than oobabooga/text-generation-webui or other similar tools that do things like bundle multiple backends for different LLM architectures, support downloading models from huggingface, present a common web UI for LLM configuration, managing prompts, and actually doing inference (in chat, notebook, and other styles), and also supports presenting an OpenAI-compatible API endpoint backed with a local LLM to support other frontends.