3 ms·
Getting great performance running gpt-oss on 3x A4000's: gpt-oss:20b = ~46 tok/s More than 2x faster than my previous leading OSS models: mistral-sma
by mythz 1y ago
Getting great performance running gpt-oss on 3x A4000's:
gpt-oss:20b = ~46 tok/s
More than 2x faster than my previous leading OSS models:
mistral-small3.2:24b = ~22 tok/s
gemma3:27b = ~19.5 tok/s
Strangely getting nearly the opposite performance running on 1x 5070 Ti:
mistral-small3.2:24b = ~39 tok/s
gpt-oss:20b = ~21 tok/s
Where gpt-oss is nearly 2x slow vs mistral-small 3.2.
- genpfault 1y agoSeeing ~70 tok/s on a 7900 XTX using Ollama.
- Matsta 1y agoI'm getting around 90 tok/s on a 3090 using Ollama. Pretty impressive
- mythz 1y agook issue is with ollama as gpt-oss 20B runs much faster on 1x 5070 Ti with llama.cpp and LM Studio: llama-server = ~181 tok/s LM Studio = ~46 tok/s (default) LM Studio Custom = ~158 tok/s (changed to offload to GPU and switch to CUDA llama.cpp engine) and llama-server on my 3x A4000 GPU Server is getting 90 tok/s vs 46 tok/s on ollama