7 ms·

It's absolutely mindboggling to me that AMD is still struggling so badly on this. There is an absolutely enormous market for AMD GPUs for this, but they seem t

It's absolutely mindboggling to me that AMD is still struggling so badly on this. There is an absolutely enormous market for AMD GPUs for this, but they seem to be completely stuck on how to build a developer ecosystem. Why aren't AMD throwing as many developers as possible submitting PRs for the open source LLM effort adding ROCm support, for example? It would give AMD real world insights to the problems with their drivers and SDKs as well, which are incredibly numerous. People would be willing to overlook a huge amount of jank for cheap(er) cards with large VRAM configurations. I don't think they when need to be particularly fast, just have the VRAM needed, which I'm sure AMD could put specialist cards together for.

hedgehog 3y ago
Historically they believed that "the community" would address broader ML software support. I think the idea was they could assign dedicated engineers for bigger customers and together that was a sort of Pareto-goodish solution given their constraints as a company. Even in retrospect I'm not sure if that was a good call or not.
- Almondsetat 3y ago
  I mean, they would be right if all their cards, both consumer and enterprises, supported the same programming interface. You cannot trust the community to do the work for you but then only make the software available for $Xk dollar cards
  - hedgehog 3y ago
    That's not necessary or sufficient. Going back to 2017 or so when I was working in the area their OpenCL support was good enough, the missing parts were an equivalent to cuDNN and upstreamed support in TensorFlow etc. That work does not subdivide in a way amenable to being a community effort and it's way too big for a hobby project. Today the technical landscape is different but from what I can tell the basic problems are the same.
    - deleted 3y ago
      [deleted]
    - vetinari 3y ago
      In 2017, when I got Vega, OpenCL didn't work yet. Today, in 2023, Vega is already not supported. Meanwhile, during this period, ROCm was unbuildable by mere mortal distribution maintainers; you either used the binaries thrown over the wall by AMD (for specific versions RHEL/Centos, SuSE and Ubuntu only), or didn't run anything at all. Also in 2023, Fedora managed to package basic ROCm packages for Fedora 38; I can finally run darktable and blender (but it is crashing!) on Vega. Woohoo!
      - a-french-anon 3y ago
        Support for Gentoo existed for a long time in https://github.com/justxi/rocm https://github.com/justxi/rocm before being merged in the main Portage tree.
        AstralStorm 3y ago
        By existed you mean maybe built and sometimes worked. That's not support. That's throwing things over the wall. More importantly, even with Vega it had numerous crashes. Anything else, like Polaris or RDNA? Forget about it. Even the AMD docker setups weren't quite good enough at times.
        a-french-anon 3y ago
        No I know, but pretending it was so hard to build it could be considered as good as closed sourced isn't needed to shit on it.
        vetinari 3y ago
        It is hard to build. AMD uses cpack deb/rpm generators; and the build process requires random things in path (some built from git checkouts of other projects). If you want to create standard deb or rpm build script for building in distro build infra, or inside mock, for rpm-based distributions, the cmake build actively makes steps to make it as difficult as possible. There used to be a talk titled something like "How to make distribution maintainers hate you" (I cannot find a link to in now); it seems that ROCm developers have seen it, took it to their hearts and then wrote several new chapters themselves. That's building. Do not even think about testing. There's a reason why it took distributions years to package it (still not done completely). The upstream project was like student projects, thrown over the wall once students stopped working at it, including the build "system".
dev_throw 3y ago
Intel has managed to get their drivers on 23.04 Ubuntu with no additional packages needed to be installed for their Arc dGPU offerings.
- brucethemoose2 3y ago
  Now if only they would offer some bigger Arc GPUs... I would have picked up a 32GB+ Arc over my 3090 in a heartbeat. Maybe even a 24GB card.
  - dev_throw 3y ago
    16 GB is a really nice offering at that price point for AI workloads. I'm keeping my fingers crossed for a higher end Battlemage offering and some real competition for Nvidia.
musha68k 3y ago
They also lag on the dataplane side do they not? AFAIR nvidia bought the main (remaining?) infiniband supplier and seamlessly integrated it with all their data center offerings? Cue Jensen Huang "the data center is the computer"?
Dalewyn 3y ago
>People would be willing to overlook a huge amount of jank for cheap(er) cards with large VRAM configurations. The older I get, the more intolerable I find jank to be because my time only keeps becoming ever more valuable.
imtringued 3y ago
They only care about selling data center cards for GPGPU. The thing is, why would anyone buy them if CUDA just works?

hedgehog 3y ago

Historically they believed that "the community" would address broader ML software support. I think the idea was they could assign dedicated engineers for bigger customers and together that was a sort of Pareto-goodish solution given their constraints as a company. Even in retrospect I'm not sure if that was a good call or not.

Almondsetat 3y ago
I mean, they would be right if all their cards, both consumer and enterprises, supported the same programming interface. You cannot trust the community to do the work for you but then only make the software available for $Xk dollar cards
- hedgehog 3y ago
  That's not necessary or sufficient. Going back to 2017 or so when I was working in the area their OpenCL support was good enough, the missing parts were an equivalent to cuDNN and upstreamed support in TensorFlow etc. That work does not subdivide in a way amenable to being a community effort and it's way too big for a hobby project. Today the technical landscape is different but from what I can tell the basic problems are the same.
  - deleted 3y ago
    [deleted]
  - vetinari 3y ago
    In 2017, when I got Vega, OpenCL didn't work yet. Today, in 2023, Vega is already not supported. Meanwhile, during this period, ROCm was unbuildable by mere mortal distribution maintainers; you either used the binaries thrown over the wall by AMD (for specific versions RHEL/Centos, SuSE and Ubuntu only), or didn't run anything at all. Also in 2023, Fedora managed to package basic ROCm packages for Fedora 38; I can finally run darktable and blender (but it is crashing!) on Vega. Woohoo!
    - a-french-anon 3y ago
      Support for Gentoo existed for a long time in https://github.com/justxi/rocm https://github.com/justxi/rocm before being merged in the main Portage tree.
      - AstralStorm 3y ago
        By existed you mean maybe built and sometimes worked. That's not support. That's throwing things over the wall. More importantly, even with Vega it had numerous crashes. Anything else, like Polaris or RDNA? Forget about it. Even the AMD docker setups weren't quite good enough at times.
        a-french-anon 3y ago
        No I know, but pretending it was so hard to build it could be considered as good as closed sourced isn't needed to shit on it.
        vetinari 3y ago
        It is hard to build. AMD uses cpack deb/rpm generators; and the build process requires random things in path (some built from git checkouts of other projects). If you want to create standard deb or rpm build script for building in distro build infra, or inside mock, for rpm-based distributions, the cmake build actively makes steps to make it as difficult as possible. There used to be a talk titled something like "How to make distribution maintainers hate you" (I cannot find a link to in now); it seems that ROCm developers have seen it, took it to their hearts and then wrote several new chapters themselves. That's building. Do not even think about testing. There's a reason why it took distributions years to package it (still not done completely). The upstream project was like student projects, thrown over the wall once students stopped working at it, including the build "system".

dev_throw 3y ago

Intel has managed to get their drivers on 23.04 Ubuntu with no additional packages needed to be installed for their Arc dGPU offerings.

brucethemoose2 3y ago
Now if only they would offer some bigger Arc GPUs... I would have picked up a 32GB+ Arc over my 3090 in a heartbeat. Maybe even a 24GB card.
- dev_throw 3y ago
  16 GB is a really nice offering at that price point for AI workloads. I'm keeping my fingers crossed for a higher end Battlemage offering and some real competition for Nvidia.

musha68k 3y ago

They also lag on the dataplane side do they not? AFAIR nvidia bought the main (remaining?) infiniband supplier and seamlessly integrated it with all their data center offerings? Cue Jensen Huang "the data center is the computer"?

Dalewyn 3y ago

>People would be willing to overlook a huge amount of jank for cheap(er) cards with large VRAM configurations. The older I get, the more intolerable I find jank to be because my time only keeps becoming ever more valuable.

imtringued 3y ago

They only care about selling data center cards for GPGPU. The thing is, why would anyone buy them if CUDA just works?