3 ms·
> ollama's model index typically only distinguishes variants by their size Don't use ollama. The entire project is just a series of stupid decisions like this.
by zargon 2mo ago
> ollama's model index typically only distinguishes variants by their size
Don't use ollama. The entire project is just a series of stupid decisions like this.
- kristjansson 2mo agoHow they have any credibility after they told everyone they could run Deepseek R1 on their laptop (by giving the Qwen 7B distill the `deepseek` moniker)...
- Foobar8568 2mo agoAnd last time I have checked, you still can't run rerankers with it, yet you can download the models. See issue 3368, 2years old now.
- SirLordBoss 2mo agoWhat's the sota? The classic vLLM/llama.cpp? LM Studio? Unsloth studio any good?
- zargon 2mo agoAs a replacement for ollama, LM Studio is probably the most appropriate. Though I haven't tried Unsloth Studio yet. You get more control using llama.cpp directly, but there's more of a learning curve. vLLM has the best performance if you can fit your entire model into VRAM. llama.cpp is the usual go-to if you're partially loading into RAM. LM Studio is a sensible front-end to llama.cpp. ik_llama.cpp has more advanced CPU quantization strategies than llama.cpp. If you're running super large models mostly from RAM, ktransformers can sometimes be the highest performer, if it works for your model.