5 ms·Llama.cpp inference on fast cpus is perfectly usable for 70B parameter models.by nullc 3y agoLlama.cpp inference on fast cpus is perfectly usable for 70B parameter models.