3 ms·
That seems oddly low / slower by a fair amount than i get on my m4. (I believe it was ~45 tok/s?) What quant are you using? How much ram does it have?
by jasonjmcghee 8mo ago
That seems oddly low / slower by a fair amount than i get on my m4. (I believe it was ~45 tok/s?)
What quant are you using? How much ram does it have?