3 ms·
It's better for VRAM poor people. I get 4-5 t/s with 27B and 20-30 t/s with 35B A3B.
by Fervicus 2mo ago
It's better for VRAM poor people. I get 4-5 t/s with 27B and 20-30 t/s with 35B A3B.
- dofm 2mo agoAlso radically better on an M1 Max. I get well up into the 60s t/s with the A3B, stuck at 9.5 or so with this new 27B, though perhaps an MLX build will help.