4 ms·

Try a quantized model on llama.cpp — I have an M1 Max 32Gb and it’s running fairly fast in 20Gb of memory.