4 ms·

I have wasted so many nights debugging and trying to figure out how to get LLMs to run on my AMD iGPU. Despite pytorch and xformers (unofficially) being suppor

I have wasted so many nights debugging and trying to figure out how to get LLMs to run on my AMD iGPU. Despite pytorch and xformers (unofficially) being supported, I was still unsuccessful. Cuda on the other hand, it just works. I don't see the value proposition of AMD over Nvidia for now

superkuh 3y ago
For that I'd suggest using llama.cpp and the clBLAST (opencl) config to compile it.
sam_goody 3y ago
You could also run CUDA on AMD cards using ZLUDA.
- 3abiton 3y ago
  ZLUDA has its own baggage for now. Still lacking many support, but that changing fast, maybe in a month or so.