4 ms·
What is the best way in terms of price/convenience ratio to run the 70B model on the cloud? Are there any providers offering out-of-the box setups?
by wklm 3y ago
What is the best way in terms of price/convenience ratio to run the 70B model on the cloud? Are there any providers offering out-of-the box setups?
- jerrygenser 3y agoI think using this project https://github.com/ggerganov/llama.cppav https://github.com/ggerganov/llama.cppav on a CPU machine with AVX instructions would be a better bang for your buck than GPU. Depends on if your use case can tolerate the latency