3 ms·
The latter link says they get ~1.7 tok/s which is quite impressive for a near-SOTA local model running on ordinary hardware.
by zozbot234 7mo ago
The latter link says they get ~1.7 tok/s which is quite impressive for a near-SOTA local model running on ordinary hardware.