3 ms·

Hi, we’ve been working to support AMD GPUs directly via ROCm. It’s still under development but if you build from source it does work: https://github.com/ollama

Hi, we’ve been working to support AMD GPUs directly via ROCm. It’s still under development but if you build from source it does work: https://github.com/ollama/ollama/blob/main/docs/development.md#linux-rocm-amd https://github.com/ollama/ollama/blob/main/docs/development....

Filligree 3y ago
Every time I try to run anything through ROCm, my machine kernel-panics. I’m not blaming you for this, but I’m also sticking with nvidia.
- mchiang 3y ago
  Really sorry about this. Do you happen to have logs for us to look into? This is definitely not the way we want
  - Filligree 3y ago
    To be clearer, it isn't Ollama-specific. I first encountered the issue with Stable Diffusion, and it's remained since, but the GPU that causes it isn't currently inside any machine; I replaced it with a 3090 a few days ago.
  - weebull 3y ago
    I'd recommend trying stuff that exhausts the VRAM. That seems to be where thinks get flakey for me (RX 7600 - 8GB), especially if running a desktop too.
- superkuh 3y ago
  And you're the lucky one getting the chance to kernel panic with ROCm. AMD drops ROCm support for their consumer GPUs so fast it'll make your head spin. I bought my GPU for $230 in 2020 and by 2021 AMD had dropped support for it. Just a bit under 4 years after the card's release on market.
agartner 3y ago
Working well for me on a 7900XT with ROCm 6 and Linux 6.7.5 thanks!
- antman 3y ago
  What is the speedup vs cpu?