3 ms·
Probably quantizing or using base weights and this project https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp on a CPU machine with A
by jerrygenser 3y ago
Probably quantizing or using base weights and this project https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp on a CPU machine with AVX512 instructions.