4 ms·
I'm running it on Serge with 51 GB of RAM. I don't think it requires any GPU memory (llama.cpp runs on the CPU), I have a RTX 2060 in the system I'm running it
by netsec_burn 3y ago
I'm running it on Serge with 51 GB of RAM. I don't think it requires any GPU memory (llama.cpp runs on the CPU), I have a RTX 2060 in the system I'm running it on. Correct me if I'm wrong.
- forgingahead 3y agoHow many tokens per second are you getting with your setup?
- netsec_burn 3y agoAbout .5 tokens per second if I had to guess.