3 ms·
A 7b quantised model is also about the biggest you can run on an M1 MacBook too. It's nowhere near that speed but it does work.
by regularfry 3y ago
A 7b quantised model is also about the biggest you can run on an M1 MacBook too. It's nowhere near that speed but it does work.