3 ms·I've been occasionally using a 7b Q4 quant on llama.cpp on an 8GB M1. It's usable, if not amazing.by regularfry 3y agoI've been occasionally using a 7b Q4 quant on llama.cpp on an 8GB M1. It's usable, if not amazing.