Would adding a single GPU help with prompt processing here? When I run a llama.cpp GPU build with no layers offloaded prompt processing is still way faster as a

Would adding a single GPU help with prompt processing here? When I run a llama.cpp GPU build with no layers offloaded prompt processing is still way faster as all the matrix multiplies are done on the accelerator. The actual memory bound inference continues to run on the CPU. Since tensor cores are so fast you still come out ahead when you send the weights over PCIE to the GPU and return the completed products back to main memory.