3 ms·
Everytime I try to mention how shit Ollama is, I get mass downvoted here by folks that don't want to hear the truth: There are 4 good inference engines (okay, 5
by Der_Einzige 4mo ago
Everytime I try to mention how shit Ollama is, I get mass downvoted here by folks that don't want to hear the truth: There are 4 good inference engines (okay, 5, but we don't count huggingface because it's slow):
1. vLLM
2. sglang
3. (nvidia only) TRT-LLM
4. llama.cpp (mac only, the above are better for non-mac)
If you're not using one of the above, you're doing it wrong