4 ms·
> and $10,000+ of compute hardware per inference session. What hardware would you need to run it at home?
by Sosh101 3y ago
> and $10,000+ of compute hardware per inference session.
What hardware would you need to run it at home?
- ramesh31 3y ago>What hardware would you need to run it at home? Step 1: https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/blob/main/llama-2-7b-chat.ggmlv3.q4_0.bin https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/blob/ma... Step 2: https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp Step 3: you're welcome
- Sosh101 3y agoThat's very helpful, thank you.
- avion23 3y ago> and $10,000+ of compute hardware per inference session. That is not true. A common macbook with lots of RAM (>32GB) is enough. Or any x86 computer with lots of RAM. llama.cpp is CPU only and quite fast