2 ms·
The recent update to llama.cpp to support Apple Silicon (at least for Q4_0 models) is really great. I can run pretty big models on my macbook now with decent p
by rootusrootus 3y ago
The recent update to llama.cpp to support Apple Silicon (at least for Q4_0 models) is really great. I can run pretty big models on my macbook now with decent performance, and smaller 13B models just fly. The ability to get a lot of GPU-accessible RAM is perfect.