3 ms·
I've noticed that llama 2 + llama.cpp doesn't seem to even use the GPU much. I tried a better gpu (more speed, more memory) and my inference speed didn't incre
by m463 3y ago
I've noticed that llama 2 + llama.cpp doesn't seem to even use the GPU much. I tried a better gpu (more speed, more memory) and my inference speed didn't increase.