3 ms·
That's absolutely false, unless you mean running llamacpp with half the layers on the CPU and half the layers on the GPU. And that was like 2t/s.
by redox99 3y ago
That's absolutely false, unless you mean running llamacpp with half the layers on the CPU and half the layers on the GPU. And that was like 2t/s.
- yumraj 3y ago> running llamacpp with half the layers on the CPU and half the layers on the GPU How do you do that? Could you please provide a link if you have it handy.