10 ms·
Llama.cpp running on the GPU is pretty slow. Better try with something else. The speedup going from CPU to RTX 3090 is usually around 10x or 15x.
by Tepix 3y ago
Llama.cpp running on the GPU is pretty slow. Better try with something else.
The speedup going from CPU to RTX 3090 is usually around 10x or 15x.