3 ms·
ok issue is with ollama as gpt-oss 20B runs much faster on 1x 5070 Ti with llama.cpp and LM Studio: llama-server = ~181 tok/s LM Studio = ~4
by mythz 1y ago
ok issue is with ollama as gpt-oss 20B runs much faster on 1x 5070 Ti with llama.cpp and LM Studio:
llama-server = ~181 tok/s
LM Studio = ~46 tok/s (default)
LM Studio Custom = ~158 tok/s (changed to offload to GPU and switch to CUDA llama.cpp engine)
and llama-server on my 3x A4000 GPU Server is getting 90 tok/s vs 46 tok/s on ollama