3 ms·
30-A3B model gives 13 t/s without GPU (I noticed that token/sec * # of params matches memory bandwidth).
by codedokode 8mo ago
30-A3B model gives 13 t/s without GPU (I noticed that token/sec * # of params matches memory bandwidth).
- yencabulator 8mo agoSomething like 21 t/s on pure CPU on a mini PC that's <2 years old.