2 ms·
Unless I'm missing something, the full 1TB bandwidth isn't used because the memory layout is wrong. But that's fixable. Since it's memory bound, it might be p
by erichocean 2y ago
Unless I'm missing something, the full 1TB bandwidth isn't used because the memory layout is wrong.
But that's fixable.
Since it's memory bound, it might be possible to reach 15 tok/sec with this build.
- r14c 2y agoI'm curious, what's wrong with the memory layout? You mean the ollama settings?