3 ms·
Looks like Ollama is focusing more and more on non-local offerings. Also their performance is worse than say vLLM. What's a good Ollama alternative (for keepin
by Tepix 1y ago
Looks like Ollama is focusing more and more on non-local offerings. Also their performance is worse than say vLLM.
What's a good Ollama alternative (for keeping 1-5x RTX 3090 busy) if you want to run things like open-webui (via an OpenAI compatible API) where your users can choose between a few LLMs?
- Ey7NFZ3P0nzAe 1y agoi heard about Llamaswap and vllm
- kgeist 1y agoAt work I've set up LibreChat + LlamaSwap + llama.cpp 200 weekly users :)
- Tepix 1y agoHow do you deal with different users wanting to use different LLMs at the same time?