4 ms·

Same. I really want AMD to succeed because as a long time Linux user I have strong distaste for Nvidia and the hell they put me through. I paid a lot for a be

Same. I really want AMD to succeed because as a long time Linux user I have strong distaste for Nvidia and the hell they put me through. I paid a lot for a beastly AMD card in the hopes that it would be shortly behind Nvidia and that has most definitely not been the case, and I blame AMD for not putting the resources behind it. AMD, you can change, but you need to start NOW.

mchiang 3y ago
Hi, we’ve been working to support AMD GPUs directly via ROCm. It’s still under development but if you build from source it does work: https://github.com/ollama/ollama/blob/main/docs/development.md#linux-rocm-amd https://github.com/ollama/ollama/blob/main/docs/development....
- Filligree 3y ago
  Every time I try to run anything through ROCm, my machine kernel-panics. I’m not blaming you for this, but I’m also sticking with nvidia.
  - mchiang 3y ago
    Really sorry about this. Do you happen to have logs for us to look into? This is definitely not the way we want
    - Filligree 3y ago
      To be clearer, it isn't Ollama-specific. I first encountered the issue with Stable Diffusion, and it's remained since, but the GPU that causes it isn't currently inside any machine; I replaced it with a 3090 a few days ago.
    - weebull 3y ago
      I'd recommend trying stuff that exhausts the VRAM. That seems to be where thinks get flakey for me (RX 7600 - 8GB), especially if running a desktop too.
  - superkuh 3y ago
    And you're the lucky one getting the chance to kernel panic with ROCm. AMD drops ROCm support for their consumer GPUs so fast it'll make your head spin. I bought my GPU for $230 in 2020 and by 2021 AMD had dropped support for it. Just a bit under 4 years after the card's release on market.
- agartner 3y ago
  Working well for me on a 7900XT with ROCm 6 and Linux 6.7.5 thanks!
  - antman 3y ago
    What is the speedup vs cpu?
visarga 3y ago
Ollama is a model-management app that runs on top of llama.cpp so you should ask there about AMD support.
- progman32 3y ago
  I've been running llama.cpp with full GPU acceleration on my AMD card, using the text-generation-webui install script on kubuntu. Same with stable diffusion using a1111. AMD's compute stack is indeed quite broken and is more fragile, but it does work using most modern cards. The kernel panics though... Yeah, I had those on my Radeon vii before I upgraded.
- 65a 3y ago
  llama.cpp has had ROCm support for a long time
michaelmrose 3y ago
What problems have you had with AMD and in what fashion do they fall short of Nvidia?
- freedomben 3y ago
  I've had no end of difficulty installing the Pro drivers and/or ROCm. The "solution" that was recommended was to install a different distro (I use Fedora and installing CentOS or Ubuntu was recommended). When I finally could get it installed, I got kernel panics and my system frequently became unbootable. Then once it was installed, getting user space programs to recognize it was the next major pain point.
  - michaelmrose 3y ago
    I've been using Nvidia and it stopped being challenging in about 2006. I hear perpetually that Nvidia is horrible and I should try AMD. The 2 times I did admitted a long time ago it was... not great.
    - freedomben 3y ago
      Do you use Ubuntu LTS? If so, then indeed Nvidia is not a problem. But if you run a distro that has anywhere near new kernels such as Fedora and Arch, you'll be constantly in fear of receiving new kernel updates. And every so often the packages will be broken and you'll have to use Nvidia's horrible installer. Oh and every once in a while they'll subtly drop support for older cards and you'll need to move to the legacy package, but the way you'll find out is that your system suddenly doesn't boot and you just happen to think about it being the old Nvidia card so you Kagi that and discover the change.
      - 65a 3y ago
        I found it much easier to make ROCm/AMD work for AI (including on an laptop) than getting nvidia work with Xorg on an optimus laptop with an intel iGPU/nvidia dGPU. I swore off nvidia at that point.
karolist 3y ago
What hell specifically, do you mean loading binary blob drivers in the past?
zare_st 3y ago
Curious how different a long time FreeBSD user feels. I have a strong distaste for anything not nvidia. Official nvidia drivers have been added to FreeBSD repository 21 years ago. I can't count the number of different types of drivers used for ATi/AMD in these two decades. And none had the performance or stability.

mchiang 3y ago

Hi, we’ve been working to support AMD GPUs directly via ROCm. It’s still under development but if you build from source it does work: https://github.com/ollama/ollama/blob/main/docs/development.md#linux-rocm-amd https://github.com/ollama/ollama/blob/main/docs/development....

Filligree 3y ago
Every time I try to run anything through ROCm, my machine kernel-panics. I’m not blaming you for this, but I’m also sticking with nvidia.
- mchiang 3y ago
  Really sorry about this. Do you happen to have logs for us to look into? This is definitely not the way we want
  - Filligree 3y ago
    To be clearer, it isn't Ollama-specific. I first encountered the issue with Stable Diffusion, and it's remained since, but the GPU that causes it isn't currently inside any machine; I replaced it with a 3090 a few days ago.
  - weebull 3y ago
    I'd recommend trying stuff that exhausts the VRAM. That seems to be where thinks get flakey for me (RX 7600 - 8GB), especially if running a desktop too.
- superkuh 3y ago
  And you're the lucky one getting the chance to kernel panic with ROCm. AMD drops ROCm support for their consumer GPUs so fast it'll make your head spin. I bought my GPU for $230 in 2020 and by 2021 AMD had dropped support for it. Just a bit under 4 years after the card's release on market.
agartner 3y ago
Working well for me on a 7900XT with ROCm 6 and Linux 6.7.5 thanks!
- antman 3y ago
  What is the speedup vs cpu?

visarga 3y ago

Ollama is a model-management app that runs on top of llama.cpp so you should ask there about AMD support.

progman32 3y ago
I've been running llama.cpp with full GPU acceleration on my AMD card, using the text-generation-webui install script on kubuntu. Same with stable diffusion using a1111. AMD's compute stack is indeed quite broken and is more fragile, but it does work using most modern cards. The kernel panics though... Yeah, I had those on my Radeon vii before I upgraded.
65a 3y ago
llama.cpp has had ROCm support for a long time

michaelmrose 3y ago

What problems have you had with AMD and in what fashion do they fall short of Nvidia?

freedomben 3y ago
I've had no end of difficulty installing the Pro drivers and/or ROCm. The "solution" that was recommended was to install a different distro (I use Fedora and installing CentOS or Ubuntu was recommended). When I finally could get it installed, I got kernel panics and my system frequently became unbootable. Then once it was installed, getting user space programs to recognize it was the next major pain point.
- michaelmrose 3y ago
  I've been using Nvidia and it stopped being challenging in about 2006. I hear perpetually that Nvidia is horrible and I should try AMD. The 2 times I did admitted a long time ago it was... not great.
  - freedomben 3y ago
    Do you use Ubuntu LTS? If so, then indeed Nvidia is not a problem. But if you run a distro that has anywhere near new kernels such as Fedora and Arch, you'll be constantly in fear of receiving new kernel updates. And every so often the packages will be broken and you'll have to use Nvidia's horrible installer. Oh and every once in a while they'll subtly drop support for older cards and you'll need to move to the legacy package, but the way you'll find out is that your system suddenly doesn't boot and you just happen to think about it being the old Nvidia card so you Kagi that and discover the change.
    - 65a 3y ago
      I found it much easier to make ROCm/AMD work for AI (including on an laptop) than getting nvidia work with Xorg on an optimus laptop with an intel iGPU/nvidia dGPU. I swore off nvidia at that point.

karolist 3y ago

What hell specifically, do you mean loading binary blob drivers in the past?

zare_st 3y ago

Curious how different a long time FreeBSD user feels. I have a strong distaste for anything not nvidia. Official nvidia drivers have been added to FreeBSD repository 21 years ago. I can't count the number of different types of drivers used for ATi/AMD in these two decades. And none had the performance or stability.