3 ms·
You’ve gotta deal with amd’s AI software stack though. How is that these days? I assume cuda is still king?
by josephg 2y ago
You’ve gotta deal with amd’s AI software stack though. How is that these days? I assume cuda is still king?
- rglullis 2y agoI've been running ollama on an XTX7900 (AMD GPU with 24GB of RAM) with any model that fits in it, and absolutely no issues there.
- ac29 2y ago> You’ve gotta deal with amd’s AI software stack though Not if you are using the CPU. I am under the impression most inference use cases are memory bandwidth limited, not compute limited, so running on the GPU would gain you little to nothing unless the GPU has faster access to the shared memory.
- Havoc 2y agoInference fine, training less so