2 ms·
You can, I was able to run 13B on my 16GB 8c8g M1 Air. The performance was 2-3 tokens/second. It felt on par with ChatGPT on a busy day.
by mrtksn 4y ago
You can, I was able to run 13B on my 16GB 8c8g M1 Air. The performance was 2-3 tokens/second. It felt on par with ChatGPT on a busy day.