3 ms·
Not sure how accurate my stats are. I used ollama with the --verbose flag. Using a 4090 and all default settings, I get 40TPS for Gemma 29B model `ollama run g
by trees101 1y ago
Not sure how accurate my stats are.
I used ollama with the --verbose flag.
Using a 4090 and all default settings, I get 40TPS for Gemma 29B model
`ollama run gemma3:27b --verbose` gives me 42.5 TPS +-0.3TPS
`ollama run gemma3:27b-it-qat --verbose` gives me 41.5 TPS +-0.3TPS
Strange results; the full model gives me slightly more TPS.
- orangecat 1y agoollama's `gemma3:27b` is also 4-bit quantized, you need `27b-it-q8_0` for 8 bit or `27b-it-fp16` for FP16. See https://ollama.com/library/gemma3/tags https://ollama.com/library/gemma3/tags.