2 ms·
What is the cheapest way to run it? I'm looking to build a product over it.
by jerrygoyal 3y ago
What is the cheapest way to run it? I'm looking to build a product over it.
- jerrygenser 3y agoProbably quantizing or using base weights and this project https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp on a CPU machine with AVX512 instructions.