4 ms·
I'm getting ~30 tok/s on the A3B model with my 3070 Ti and 32k context. > Do you feel you could replace the frontier models with it for everyday coding? Would/
by bigyabai 8mo ago
I'm getting ~30 tok/s on the A3B model with my 3070 Ti and 32k context.
> Do you feel you could replace the frontier models with it for everyday coding? Would/will you?
Probably not yet, but it's really good at composing shell commands. For scripting or one-liner generation, the A3B is really good. The web development skills are markedly better than Qwen's prior models in this parameter range, too.
- deleted 8mo ago[deleted]
- jasonjmcghee 8mo agoThat seems oddly low / slower by a fair amount than i get on my m4. (I believe it was ~45 tok/s?) What quant are you using? How much ram does it have?