4 ms·
Make sure that you're telling it to use the GPU. How are you launching llama_cpp?
by MPSimmons 3y ago
Make sure that you're telling it to use the GPU. How are you launching llama_cpp?
- m463 3y agoI was using the command line. llama-7b I did some investigating, and I found it doesn't start using the GPU unless you have a lot of input (such as a long prompt).