3 ms·
You don't need a GPU, alpaca.cpp runs purely on the CPU if you have 32+ GB of RAM for the 30B Llama model. It's a bit slow, but you can toy with it..
by l33tman 4y ago
You don't need a GPU, alpaca.cpp runs purely on the CPU if you have 32+ GB of RAM for the 30B Llama model. It's a bit slow, but you can toy with it..