3 ms·
That's quite slow I'm getting 8-12 t/s on a 13 year old CPU. (Speed varies by context size and other settings who knows) https://news.ycombinator.com/item?id=4
by throwaway2027 3mo ago
That's quite slow I'm getting 8-12 t/s on a 13 year old CPU. (Speed varies by context size and other settings who knows)
https://news.ycombinator.com/item?id=48354801 https://news.ycombinator.com/item?id=48354801
- neomindryan 3mo agoThank you for sharing / linking!
- tuwtuwtuwtuw 3mo agoBut OP is using Q8 and you're using Q4?
- kQq9oHeAz6wLLS 3mo agoYeah, I'm seeing 8-9 t/s on a Xeon CPU E3-1270 V2 @ 3.50GHz with an old Nvidia Quadro K2200 (4GB). I run gemma4:e2b and gemma4:12b-it-qat on Ollama.