3 ms·

Just watch out. Sometimes AMD's ROCm support for their consumer GPU can be only 4 years (like the RX 580). So if you're not buying cutting edge on release day t

Just watch out. Sometimes AMD's ROCm support for their consumer GPU can be only 4 years (like the RX 580). So if you're not buying cutting edge on release day that means you have ~2-3 years max of ROCm support. And then you have to use opencl and let me assure you: it's not fun or widespread.

kkielhofner 3y ago
It also took AMD a year to officially support ROCm with the 7900xtx. IME it's still rough because ROCm is fundamentally rough but non-officially supported cards are even rougher.
3abiton 3y ago
I have wasted so many nights debugging and trying to figure out how to get LLMs to run on my AMD iGPU. Despite pytorch and xformers (unofficially) being supported, I was still unsuccessful. Cuda on the other hand, it just works. I don't see the value proposition of AMD over Nvidia for now
- superkuh 3y ago
  For that I'd suggest using llama.cpp and the clBLAST (opencl) config to compile it.
- sam_goody 3y ago
  You could also run CUDA on AMD cards using ZLUDA.
  - 3abiton 3y ago
    ZLUDA has its own baggage for now. Still lacking many support, but that changing fast, maybe in a month or so.