24 ms·

AMD ROCm: A Wasted Opportunity

144 points by mariuz 4y ago

montebicyclelo 4y ago
At one point I set up rocm with an rx 580, but.. it only supported older versions of tensorflow, I didn't manage to get pytorch working, and the installation/set up was not fun.
juliangoldsmith 4y ago
What about Mesa for OpenCL? Its support for AMD cards is pretty good in my (limited) experience. The codegen stuff is all shared (formerly Gallium, now llvmpipe?), so Mesa's implementation of OCL should be great.
- imtringued 4y ago
  In principle if you really wanted to stick with AMD GPUs, it is probably better than ROCm. Rusticl is slightly faster (maybe 3%) than ROCm and supports SPIR-V but the problem boils down to the fact that if OpenCL doesn't work well on NVIDIA GPUs then nobody is going to invest the time and effort into porting pytorch to take full advantage of OpenCL. There is a pytorch dlprim fork but so far it is just a random GitHub repository. That is better than nothing but you would expect AMD to just cut some checks so these people can work on the software full time.
  - hedgehog 4y ago
    I spent some time trying to get them to do this for a previous startup back around 2017 but their view at the time was essentially that a couple interns and/or "the community" would solve the problem. At a higher level I think they just didn't see it as an important market to serve. We were getting decent performance on their cards even without ROCm but it just wasn't something they cared about.
  - tormeh 4y ago
    It’s absolutely wild to me that AMD and Intel doesn’t collaborate to port PyTorch and Tensorflow to OpenCL/Vulkan. I just don’t get it. Seems like the obvious thing to do.
- blihp 4y ago
  The legacy OpenCL implementation doesn't support modern AMD (at least since RDNA) cards. RustICL seems to be the path forward but it's still a work-in-progress for AMD.
dotnet00 4y ago
The way ROCm has turned out is such a shame. It really is effectively an advertisement for CUDA and served exactly that purpose for me. Even Intel's oneAPI stuff seems to be heading in a more viable direction. I feel like AMD for some reason doesn't understand that support for GPGPU on the consumer stack is important to eventually getting adoption in the enterprise space. CUDA is so convenient in part because as long as you have an NVIDIA card it'll probably be usable for testing, even if not necessarily optimal or fast. Then, once confidence has been built about its usability, a more serious investment can be made into getting the more powerful cards.
- Tostino 4y ago
  Exactly, having your programs able to run on a developer's machine is really important for productivity.
- mastax 4y ago
  It's funny because AMD was at least rhetorically invested in consumer GPGPU a long time ago with the APU thing.
  - pjmlp 4y ago
    Which were dropped when the open source driver came to be, hence why my Asus 1215B netbook can only do OpenGL 3.3, when it originally was capable of doing OpenGL 4.1 with fglrx.
  - dotnet00 4y ago
    On top of that, GCN was a beautiful design for compute. Truly ahead of its time and a beast of a chip, as evident by how when running something they actually supported as well as NVIDIA chips, GCN cards would punch well above their gaming weight. It's just a huge shame that they never really got the software sorted out well enough to take advantage of the hardware.
- buffet_overflow 4y ago
  Exactly. I wish I could consider a 7900 XTX or two for "enthusiast" level ML work at home (thinking open sourced GPT models, Stable Diffusion, etc), but the state of library support has me pricing out 3090s instead. Sure, I could probably get ROCm going after a lot of tinkering and headaches but...that's not where I want to spend that effort. Would happily be proven wrong on this by the way.
paulmd 4y ago
> And worse, different binary slices are used between different dies of the same product line. As an example, for RDNA2, ROCm math libraries are compiled only for Navi21. This means that on a (smaller) Navi22 die (notably present in the 6700 XT), those components aren’t functional. The workaround is manually recompiling ROCm with support for more targets. Such a roadblock is very discouraging for adopters – and does complicate application distribution too. holy shit I didn't realize the portability story was that bad, it can't even target a whole uarch/family at once (eg target all RDNA2) and it has to know about every specific die it will ever run on? yeah I mean that's the deep deep problem with ROCm is there's no equivalent to PTX, AMD just wants you to distribute source and recompile everything at runtime, everything about ROCm pushes you towards source distribution rather than any kind of bytecode/IL or gosh even an executable file you could actually just run. On NVIDIA the support story is simple: the driver does the final translation from PTX to assembly at runtime, so, as long as there is at least one overlap in the PTX versions packaged in the app and the PTX versions supported by the driver, it runs. In practice this means you can take a program that was compiled for a CUDA 1.0 GPU (tesla uarch) and it'll run today on an ada, no questions asked. You might be leaving performance on the table by not fully exploiting the newer uarchs, but it'll run, just like x86. And you can take a program today and compile it against CUDA 1.0 capability targets (as long as it doesn't use any features that don't exist in those older versions) and the full 2023-era software toolchain will all just magically work on your 8800GTX even though driver support for that hardware has been dead for 10+ years. Because the driver knows how to run PTX 1.0 and the toolchain knows how to build PTX 1.0, and that's all that matters. ROCm has always really struck me as being aimed at the HPC market - only supporting pro-tier GPUs, focusing on users who are already running custom software, openly disdaining amateur support, etc. And frankly the distribute-as-source model makes ROCm just a complete nonstarter for any sort of commercial or other model where the vendor would absolutely be distributing libraries or a compiled application, let alone trying to wrangle dumb customers through getting a working ROCm environment. Which right now is the authentic "linux in 1995" level experience/ordeal. Getting serious would be restarting the GPU Ocelot project and going after PTX compatibility. But a corporation is never going to accept being a client on someone else's platform, just like they won't support the open-source Streamline framework because something something 'pluggable frameworks and library code are anti-user-freedom'. Same thing there too, AMD will only support FSR in statically-compiled code and they expect everyone to recompile all their shit and validate and push updates for every single game, every time AMD releases an update, because it's inconvenient for AMD's corporate strategy. https://youtu.be/8ve5dDQ6TQE?t=974 https://youtu.be/8ve5dDQ6TQE?t=974 (oh and now that they finally have the ML hardware, the rumor is they're working on their own ML-based upscaler, quelle surprise that the "gosh we would never do anything that legacy users couldn't run" was just a bit too.) AMD doesn't want user freedom, they want to be the one with the leash. That's why they do the source-distribution-only model... you'll have to work with your code in the ROCm ecosystem and not NVIDIA's. You'll be compiling against HIP and not NVIDIA's stuff. Etc etc. It's not a zero-effort thing to leap the gap, and they want to keep you once you do it, you'll have the same leap to get back out.
t4h4 4y ago
Power of CUDA comes from NVPTX ISA that CUDA code is compiled against. AMD needs an equivalent or they will be behind forever.
- sorenjan 4y ago
  SYCL uses SPIR-V. Hopefully AMD doesn't introduce a proprietary format but uses an open standard. It would be amazing to be able to run the same code on AMD, Nvidia, and Intel, both current and future hardware, without recompiling.
  - JonChesterfield 4y ago
    It's still with-compiling. Shipping SPIR-V just means it gets compiled on the end user machine, which thus needs to have a compiler toolchain running. So it's more convenient, and popular with closed source libraries, but not magic. Intel compiles LLVM IR to SPIR-V to LLVM IR. There was a thing on HSA/AMDGPU called HSAIL which I think was meant to work broadly like PTX but didn't work out. I think we should adapt LLVM IR for use as a serialisation format which gets specialised to the hardware at the last moment, instead of bothering with the SPIR-V indirection, but that's somewhat in tension with LLVM changing their IR representation. I'd be pretty happy with running the same code on those architectures _with recompiling_ as a first step, there's way too much #ifdef noise needed to make that hold together today.
justinclift 4y ago
(2022)
Loic 4y ago
I have no ideas about the complexity of the underlying CUDA/ROCm GPGPU, I am just using my card with CUDA a bit stupidly. But why AMD do not setup a team of 5 good coders to have a good support of the well deployed/used frameworks in the happy path case? Later they can cover more cases but at least it would drive the adoption of their cards.
ajb 4y ago
Question for those with experience in this area: is vulkan compute or webgpu going to rescue AMD here? Are ML frameworks going to support either of those as a target, making it easier for AMD to catch up again?
- figomore 4y ago
  I know this framework https://github.com/webonnx/wonnx https://github.com/webonnx/wonnx, but I’ve never used that.
formerly_proven 4y ago
AMD [software]: A Wasted Opportunity AMD often makes good and even great hardware, but they're just inept at software and always have been. That's obviously a problem, because software is what makes the hardware useful.
- pjmlp 4y ago
  RenderMonkey was nice, but helas also gone.
0xd1r 4y ago
It's really shocking that AMD fails to extend support natively. Workarounds such as DirectML claim to be the answer in unifying people with NVIDIA or AMD GPUs, but thus far it hasn't, with issues such as [this](https://github.com/microsoft/DirectML/issues/58 https://github.com/microsoft/DirectML/issues/58) constantly popping up. As nicolaslem points out, Arch does have community packages for ROCm, but that, unsurprisingly fails to lend support to many consumer GPUs. The best community support I have come across are [rocm-opencl](https://copr.fedorainfracloud.org/coprs/mystro256/rocm-opencl/)and https://copr.fedorainfracloud.org/coprs/mystro256/rocm-openc... [rocm-hip](https://copr.fedorainfracloud.org/coprs/mystro256/rocm-hip/ https://copr.fedorainfracloud.org/coprs/mystro256/rocm-hip/) for Fedora maintained by [mystro256](https://github.com/Mystro256 https://github.com/Mystro256), who is a single AMD employee.Thanks to him, my AMD GPU (Radeon 6800XT) hasn't completely gone to waste, and I was able to tinker with some things (Gaming isn't really up my alley). Lately however, after beginning to work on DGX V100s and A100s, and using my older laptop with a GTX 1650, it was apparent how simple setting up CUDA was, and how easily I could experiment with it on my consumer card. Many have spoken about similar stories, and here's mine. Really hope AMD does a whole lot more, and doesn't exclusively keep their powerful GPUs for gaming.
sorenjan 4y ago
There's a lot of talk about ML in general, and training in particular, when talking about GPGPU. But it's a very performant resource for various other productivity work as well. Meshroom (photogrammetry) needs Cuda for high quality models. Photoshop uses OpenCL for some filters. Lots of consumers and amateurs wants to use the new ML models for super resolution, generative art, transcriptions, etc. Continuing to sell basically super computers in a PCI express card form factor that's half the value of the entire computer that's only usable for games is a huge waste, and it's a strange strategy for an underdog.
amrb 4y ago
We're in a much better place with the 7900 cards now having a hardware accelerator for FP16 and BF16 precision. Looking at benchmarks for Stable Diffusion they can match the rtx3090. https://www.tomshardware.com/news/stable-diffusion-gpu-benchmarks https://www.tomshardware.com/news/stable-diffusion-gpu-bench... The strategy should be to break the chicken and egg situation we are in, fixing the impression that NVIDIA is the go too. First make the packages easy to install and get started, plus show-off things like SD performance being competitive or models on huggingface.co Secondly if AMD can't allocate the development budget, they should open it up to the smart people in the community, give them all the hardware documentation required to build out support.
- JonChesterfield 4y ago
  > Secondly if AMD can't allocate the development budget, they should open it up to the smart people in the community, give them all the hardware documentation required to build out support. The Linux driver is open source and a rather comprehensive description of how the hardware is believed to work. I'd like source code for the firmware but the HSA programming model is reasonably straightforward and does appear to be implemented by the firmware. The compiler stack is LLVM and should work out of the box. As a minor existence proof, I'm one of the people responsible for having openmp on amdgpu work, and I haven't read any internal documentation whatsoever. Openmp has bugs but also has some happy users. The public ISA pdfs have been sufficient so far.
  - amrb 4y ago
    That's great news and thanks for the work on openmp!
JonChesterfield 4y ago
ROCm is doing excellently at pursuing the opportunity AMD is interested in. It's running on seriously fast supercomputers doing useful stuff for customers. I think it's also running production machine learning workloads though I don't pay much attention to that. AMDGPU in general is doing brilliantly in games consoles and, as far as I can tell from the outside, roughly holding its own in PC gaming. ROCm as a Linux first, open source lets-do-maths-on-GPUs is not being pursued to the same extent. That's evident in the supported hardware list (which is essentially the cards in supercomputers) and that the ROCm stack that comes in binary form hasn't been compiled for various gaming cards. Arch and I think Debian have packaging efforts in place for it, and with determination it's possible to assemble working toolchains from source code. It's not easy to build a working system out of exclusively open source code and amdgpu gaming hardware but it can be done. The challenge is that the linux kernel, ROCm libraries and LLVM toolchain all need to broadly agree on ABI at a given point in time, and that interface changes more often than one would like, and not simultaneously across the projects. That's the sort of thing which is easy to manage on HPC and a mess on miscellaneous user systems, and it's expensive to fix properly which brings us back to which opportunities are of highest commercial value. The positive side to this is that the foundational layers - the driver in the linux kernel and the llvm toolchain - are both developed primarily upstream. Building either from source is therefore likely to give you binaries that work. If they don't work, you can raise bugs against those projects directly. So it's quite DIY but at least you've got the pieces to work from. When cuda falls over, you've got no hope.
- incrudible 4y ago
  Ah yes, the old it-is-FOSS-so-you-can-fix-it-yourself argument. Well, I am not going to. I will be taking my chances with CUDA, because in my experience, it does not fall over all the time, unlike AMDs stack.
stuaxo 4y ago
The whole situation is pretty annoying. Have been through a couple of generations of Ryzen laptops and for seemingly arbitrary reasons all the good stuff is locked in ROCM, and now they are not even supported. Given how shonky some aspects are - (no proper wake up from sleep 5 years in), there's no way I'd risk a whole other graphics stack that seems locked to a particular distro Wasted opportunity sums up the whole AMD experience of the last few years (same laptops on windows havent been the best either the few times I've used them - just as likely to shut down possibly due to overheating during games there as much as on Linux).
vkaku 4y ago
I was waiting for things to stabilize a bit on the software and kernel side, for the last 3 years. First was to ensure the Kernel driver would work, then ROCM would work, then something like Tensorflow would work. If ROCM wasn't going to be there, it was PlaidML for me. A lot of that died in time. Quite recently I decided to say f--- it and moved to Pytorch, Dockerized ROCM and things seemed to work great on most distros today. I guess Pytorch did a better job of it than Google did in getting it to work on AMD cards. Ditto with Blender who used this stack on Windows a long time before the rest of us could get to. Very recently I tried to HIPify most CUDA code and I was able to get it to work. AMD seem to be open to patches too. So that may be the best things that could happen to this platform moving forward. What sucks with ROCM is that AMD are purposely removing older cards from ROCM and that is such a wasted opportunity for AMD. Kudos to that one guy who still ports the Polaris code to work with ROCM 5.4.x. If they extended their attitude of graphics driver improvements for older cards to ROCM, it would Rock em. Finally, I want to add this, if AMD would rather the community do a better job - it should let the community ship the next version. We'd all benefit - like Mesa. A lot of game consoles would highly benefit from this work too.
jacooper 4y ago
ROCm is the reason why if you want to even bother with AI, you just need to buy an nvidia GPU. Amd's crappy support and absolutely ridiculous support list makes it absolutely useless.
- saboot 4y ago
  Depends on the scale of the AI. Many current gen supercomputers use AMD and ROCm exclusively.
  - synergy20 4y ago
    really? epyc used a lot for host cpu to work with nvidia gpus, which is totally cuda, nobody uses rocm for ai.
    - unnah 4y ago
      Check out https://top500.org/lists/top500/2022/11/ https://top500.org/lists/top500/2022/11/ - of the top 10 supercomputers, 2 are based on AMD GPUs, 5 on NVIDIA, and 3 are more specialized solutions. All AMD has to do is offer lots of petaflops on the cheap, and software that technically works. Once the hardware is there, and research groups have won grants to work on it, it's up to grad students (who actually do the research work) to figure out how to get anything done on ROCm.
      - _yo2u 4y ago
        AFAIK AMD based supercomputers were chosen for their traditional HPC perf (eg. FP64) perf not AI perf (eg. BF16, FP16, FP8, INT8)
        synergy20 4y ago
        That's true, MI300 might be the first real device geared towards AI indeed.
    - beebeepka 4y ago
nicolaslem 4y ago
Since very recently Arch community packages ROCm. At least it makes it a bit easier to try for yourself without having to jump through too many hoops. https://archlinux.org/packages/community/x86_64/rocm-opencl-runtime/ https://archlinux.org/packages/community/x86_64/rocm-opencl-...
- matxip 4y ago
  I built this and the other rocm packages from AUR (some removed now) along with pytorch build-flagged for rocm. It did eventually work, but It was a real pain in the ass and cost a whole lot of time. After that though, something broke (who knows) and I ended up rebuilding everything again just to get it back to a working state. I'm currently too afraid to update to these community packages, partially because I don't know if the build flags I used are the duct tape keeping my builds functional (I have an rx590-8gb which is maybe at the edge of supported gpus). Instead, I just added every rocm package to my IgnorePkgs, which isn't a great sign.
- MayeulC 4y ago
  Unfortunately, ROCm is broken for gfx803 (and possibly others?). I don't think they perform regression testing with their older cards. I think this is due to custom assembly kernels that are now outdated, I recall seeing somewhere that removing the optimized kernels and allowing to fall back to the generic implementation allowed it to work. ROCm is in such a sad state. If I can't make it work at home, why would I ever want to investigate it at work?

144 points by mariuz 4y ago

At one point I set up rocm with an rx 580, but.. it only supported older versions of tensorflow, I didn't manage to get pytorch working, and the installation/set up was not fun.

juliangoldsmith 4y ago

What about Mesa for OpenCL? Its support for AMD cards is pretty good in my (limited) experience. The codegen stuff is all shared (formerly Gallium, now llvmpipe?), so Mesa's implementation of OCL should be great.

imtringued 4y ago
In principle if you really wanted to stick with AMD GPUs, it is probably better than ROCm. Rusticl is slightly faster (maybe 3%) than ROCm and supports SPIR-V but the problem boils down to the fact that if OpenCL doesn't work well on NVIDIA GPUs then nobody is going to invest the time and effort into porting pytorch to take full advantage of OpenCL. There is a pytorch dlprim fork but so far it is just a random GitHub repository. That is better than nothing but you would expect AMD to just cut some checks so these people can work on the software full time.
- hedgehog 4y ago
  I spent some time trying to get them to do this for a previous startup back around 2017 but their view at the time was essentially that a couple interns and/or "the community" would solve the problem. At a higher level I think they just didn't see it as an important market to serve. We were getting decent performance on their cards even without ROCm but it just wasn't something they cared about.
- tormeh 4y ago
  It’s absolutely wild to me that AMD and Intel doesn’t collaborate to port PyTorch and Tensorflow to OpenCL/Vulkan. I just don’t get it. Seems like the obvious thing to do.
blihp 4y ago
The legacy OpenCL implementation doesn't support modern AMD (at least since RDNA) cards. RustICL seems to be the path forward but it's still a work-in-progress for AMD.

dotnet00 4y ago

The way ROCm has turned out is such a shame. It really is effectively an advertisement for CUDA and served exactly that purpose for me. Even Intel's oneAPI stuff seems to be heading in a more viable direction. I feel like AMD for some reason doesn't understand that support for GPGPU on the consumer stack is important to eventually getting adoption in the enterprise space. CUDA is so convenient in part because as long as you have an NVIDIA card it'll probably be usable for testing, even if not necessarily optimal or fast. Then, once confidence has been built about its usability, a more serious investment can be made into getting the more powerful cards.

Tostino 4y ago
Exactly, having your programs able to run on a developer's machine is really important for productivity.
mastax 4y ago
It's funny because AMD was at least rhetorically invested in consumer GPGPU a long time ago with the APU thing.
- pjmlp 4y ago
  Which were dropped when the open source driver came to be, hence why my Asus 1215B netbook can only do OpenGL 3.3, when it originally was capable of doing OpenGL 4.1 with fglrx.
- dotnet00 4y ago
  On top of that, GCN was a beautiful design for compute. Truly ahead of its time and a beast of a chip, as evident by how when running something they actually supported as well as NVIDIA chips, GCN cards would punch well above their gaming weight. It's just a huge shame that they never really got the software sorted out well enough to take advantage of the hardware.
buffet_overflow 4y ago
Exactly. I wish I could consider a 7900 XTX or two for "enthusiast" level ML work at home (thinking open sourced GPT models, Stable Diffusion, etc), but the state of library support has me pricing out 3090s instead. Sure, I could probably get ROCm going after a lot of tinkering and headaches but...that's not where I want to spend that effort. Would happily be proven wrong on this by the way.

paulmd 4y ago

> And worse, different binary slices are used between different dies of the same product line. As an example, for RDNA2, ROCm math libraries are compiled only for Navi21. This means that on a (smaller) Navi22 die (notably present in the 6700 XT), those components aren’t functional. The workaround is manually recompiling ROCm with support for more targets. Such a roadblock is very discouraging for adopters – and does complicate application distribution too. holy shit I didn't realize the portability story was that bad, it can't even target a whole uarch/family at once (eg target all RDNA2) and it has to know about every specific die it will ever run on? yeah I mean that's the deep deep problem with ROCm is there's no equivalent to PTX, AMD just wants you to distribute source and recompile everything at runtime, everything about ROCm pushes you towards source distribution rather than any kind of bytecode/IL or gosh even an executable file you could actually just run. On NVIDIA the support story is simple: the driver does the final translation from PTX to assembly at runtime, so, as long as there is at least one overlap in the PTX versions packaged in the app and the PTX versions supported by the driver, it runs. In practice this means you can take a program that was compiled for a CUDA 1.0 GPU (tesla uarch) and it'll run today on an ada, no questions asked. You might be leaving performance on the table by not fully exploiting the newer uarchs, but it'll run, just like x86. And you can take a program today and compile it against CUDA 1.0 capability targets (as long as it doesn't use any features that don't exist in those older versions) and the full 2023-era software toolchain will all just magically work on your 8800GTX even though driver support for that hardware has been dead for 10+ years. Because the driver knows how to run PTX 1.0 and the toolchain knows how to build PTX 1.0, and that's all that matters. ROCm has always really struck me as being aimed at the HPC market - only supporting pro-tier GPUs, focusing on users who are already running custom software, openly disdaining amateur support, etc. And frankly the distribute-as-source model makes ROCm just a complete nonstarter for any sort of commercial or other model where the vendor would absolutely be distributing libraries or a compiled application, let alone trying to wrangle dumb customers through getting a working ROCm environment. Which right now is the authentic "linux in 1995" level experience/ordeal. Getting serious would be restarting the GPU Ocelot project and going after PTX compatibility. But a corporation is never going to accept being a client on someone else's platform, just like they won't support the open-source Streamline framework because something something 'pluggable frameworks and library code are anti-user-freedom'. Same thing there too, AMD will only support FSR in statically-compiled code and they expect everyone to recompile all their shit and validate and push updates for every single game, every time AMD releases an update, because it's inconvenient for AMD's corporate strategy. https://youtu.be/8ve5dDQ6TQE?t=974 https://youtu.be/8ve5dDQ6TQE?t=974 (oh and now that they finally have the ML hardware, the rumor is they're working on their own ML-based upscaler, quelle surprise that the "gosh we would never do anything that legacy users couldn't run" was just a bit too.) AMD doesn't want user freedom, they want to be the one with the leash. That's why they do the source-distribution-only model... you'll have to work with your code in the ROCm ecosystem and not NVIDIA's. You'll be compiling against HIP and not NVIDIA's stuff. Etc etc. It's not a zero-effort thing to leap the gap, and they want to keep you once you do it, you'll have the same leap to get back out.

t4h4 4y ago

Power of CUDA comes from NVPTX ISA that CUDA code is compiled against. AMD needs an equivalent or they will be behind forever.

sorenjan 4y ago
SYCL uses SPIR-V. Hopefully AMD doesn't introduce a proprietary format but uses an open standard. It would be amazing to be able to run the same code on AMD, Nvidia, and Intel, both current and future hardware, without recompiling.
- JonChesterfield 4y ago
  It's still with-compiling. Shipping SPIR-V just means it gets compiled on the end user machine, which thus needs to have a compiler toolchain running. So it's more convenient, and popular with closed source libraries, but not magic. Intel compiles LLVM IR to SPIR-V to LLVM IR. There was a thing on HSA/AMDGPU called HSAIL which I think was meant to work broadly like PTX but didn't work out. I think we should adapt LLVM IR for use as a serialisation format which gets specialised to the hardware at the last moment, instead of bothering with the SPIR-V indirection, but that's somewhat in tension with LLVM changing their IR representation. I'd be pretty happy with running the same code on those architectures _with recompiling_ as a first step, there's way too much #ifdef noise needed to make that hold together today.

justinclift 4y ago

(2022)

Loic 4y ago

I have no ideas about the complexity of the underlying CUDA/ROCm GPGPU, I am just using my card with CUDA a bit stupidly. But why AMD do not setup a team of 5 good coders to have a good support of the well deployed/used frameworks in the happy path case? Later they can cover more cases but at least it would drive the adoption of their cards.

ajb 4y ago

Question for those with experience in this area: is vulkan compute or webgpu going to rescue AMD here? Are ML frameworks going to support either of those as a target, making it easier for AMD to catch up again?

figomore 4y ago
I know this framework https://github.com/webonnx/wonnx https://github.com/webonnx/wonnx, but I’ve never used that.

formerly_proven 4y ago

AMD [software]: A Wasted Opportunity AMD often makes good and even great hardware, but they're just inept at software and always have been. That's obviously a problem, because software is what makes the hardware useful.

pjmlp 4y ago
RenderMonkey was nice, but helas also gone.

0xd1r 4y ago

It's really shocking that AMD fails to extend support natively. Workarounds such as DirectML claim to be the answer in unifying people with NVIDIA or AMD GPUs, but thus far it hasn't, with issues such as [this](https://github.com/microsoft/DirectML/issues/58 https://github.com/microsoft/DirectML/issues/58) constantly popping up. As nicolaslem points out, Arch does have community packages for ROCm, but that, unsurprisingly fails to lend support to many consumer GPUs. The best community support I have come across are [rocm-opencl](https://copr.fedorainfracloud.org/coprs/mystro256/rocm-opencl/)and https://copr.fedorainfracloud.org/coprs/mystro256/rocm-openc... [rocm-hip](https://copr.fedorainfracloud.org/coprs/mystro256/rocm-hip/ https://copr.fedorainfracloud.org/coprs/mystro256/rocm-hip/) for Fedora maintained by [mystro256](https://github.com/Mystro256 https://github.com/Mystro256), who is a single AMD employee.Thanks to him, my AMD GPU (Radeon 6800XT) hasn't completely gone to waste, and I was able to tinker with some things (Gaming isn't really up my alley). Lately however, after beginning to work on DGX V100s and A100s, and using my older laptop with a GTX 1650, it was apparent how simple setting up CUDA was, and how easily I could experiment with it on my consumer card. Many have spoken about similar stories, and here's mine. Really hope AMD does a whole lot more, and doesn't exclusively keep their powerful GPUs for gaming.

sorenjan 4y ago

There's a lot of talk about ML in general, and training in particular, when talking about GPGPU. But it's a very performant resource for various other productivity work as well. Meshroom (photogrammetry) needs Cuda for high quality models. Photoshop uses OpenCL for some filters. Lots of consumers and amateurs wants to use the new ML models for super resolution, generative art, transcriptions, etc. Continuing to sell basically super computers in a PCI express card form factor that's half the value of the entire computer that's only usable for games is a huge waste, and it's a strange strategy for an underdog.

amrb 4y ago

We're in a much better place with the 7900 cards now having a hardware accelerator for FP16 and BF16 precision. Looking at benchmarks for Stable Diffusion they can match the rtx3090. https://www.tomshardware.com/news/stable-diffusion-gpu-benchmarks https://www.tomshardware.com/news/stable-diffusion-gpu-bench... The strategy should be to break the chicken and egg situation we are in, fixing the impression that NVIDIA is the go too. First make the packages easy to install and get started, plus show-off things like SD performance being competitive or models on huggingface.co Secondly if AMD can't allocate the development budget, they should open it up to the smart people in the community, give them all the hardware documentation required to build out support.

JonChesterfield 4y ago
> Secondly if AMD can't allocate the development budget, they should open it up to the smart people in the community, give them all the hardware documentation required to build out support. The Linux driver is open source and a rather comprehensive description of how the hardware is believed to work. I'd like source code for the firmware but the HSA programming model is reasonably straightforward and does appear to be implemented by the firmware. The compiler stack is LLVM and should work out of the box. As a minor existence proof, I'm one of the people responsible for having openmp on amdgpu work, and I haven't read any internal documentation whatsoever. Openmp has bugs but also has some happy users. The public ISA pdfs have been sufficient so far.
- amrb 4y ago
  That's great news and thanks for the work on openmp!

JonChesterfield 4y ago

ROCm is doing excellently at pursuing the opportunity AMD is interested in. It's running on seriously fast supercomputers doing useful stuff for customers. I think it's also running production machine learning workloads though I don't pay much attention to that. AMDGPU in general is doing brilliantly in games consoles and, as far as I can tell from the outside, roughly holding its own in PC gaming. ROCm as a Linux first, open source lets-do-maths-on-GPUs is not being pursued to the same extent. That's evident in the supported hardware list (which is essentially the cards in supercomputers) and that the ROCm stack that comes in binary form hasn't been compiled for various gaming cards. Arch and I think Debian have packaging efforts in place for it, and with determination it's possible to assemble working toolchains from source code. It's not easy to build a working system out of exclusively open source code and amdgpu gaming hardware but it can be done. The challenge is that the linux kernel, ROCm libraries and LLVM toolchain all need to broadly agree on ABI at a given point in time, and that interface changes more often than one would like, and not simultaneously across the projects. That's the sort of thing which is easy to manage on HPC and a mess on miscellaneous user systems, and it's expensive to fix properly which brings us back to which opportunities are of highest commercial value. The positive side to this is that the foundational layers - the driver in the linux kernel and the llvm toolchain - are both developed primarily upstream. Building either from source is therefore likely to give you binaries that work. If they don't work, you can raise bugs against those projects directly. So it's quite DIY but at least you've got the pieces to work from. When cuda falls over, you've got no hope.

incrudible 4y ago
Ah yes, the old it-is-FOSS-so-you-can-fix-it-yourself argument. Well, I am not going to. I will be taking my chances with CUDA, because in my experience, it does not fall over all the time, unlike AMDs stack.

stuaxo 4y ago

The whole situation is pretty annoying. Have been through a couple of generations of Ryzen laptops and for seemingly arbitrary reasons all the good stuff is locked in ROCM, and now they are not even supported. Given how shonky some aspects are - (no proper wake up from sleep 5 years in), there's no way I'd risk a whole other graphics stack that seems locked to a particular distro Wasted opportunity sums up the whole AMD experience of the last few years (same laptops on windows havent been the best either the few times I've used them - just as likely to shut down possibly due to overheating during games there as much as on Linux).

vkaku 4y ago

I was waiting for things to stabilize a bit on the software and kernel side, for the last 3 years. First was to ensure the Kernel driver would work, then ROCM would work, then something like Tensorflow would work. If ROCM wasn't going to be there, it was PlaidML for me. A lot of that died in time. Quite recently I decided to say f--- it and moved to Pytorch, Dockerized ROCM and things seemed to work great on most distros today. I guess Pytorch did a better job of it than Google did in getting it to work on AMD cards. Ditto with Blender who used this stack on Windows a long time before the rest of us could get to. Very recently I tried to HIPify most CUDA code and I was able to get it to work. AMD seem to be open to patches too. So that may be the best things that could happen to this platform moving forward. What sucks with ROCM is that AMD are purposely removing older cards from ROCM and that is such a wasted opportunity for AMD. Kudos to that one guy who still ports the Polaris code to work with ROCM 5.4.x. If they extended their attitude of graphics driver improvements for older cards to ROCM, it would Rock em. Finally, I want to add this, if AMD would rather the community do a better job - it should let the community ship the next version. We'd all benefit - like Mesa. A lot of game consoles would highly benefit from this work too.

jacooper 4y ago

ROCm is the reason why if you want to even bother with AI, you just need to buy an nvidia GPU. Amd's crappy support and absolutely ridiculous support list makes it absolutely useless.

saboot 4y ago
Depends on the scale of the AI. Many current gen supercomputers use AMD and ROCm exclusively.
- synergy20 4y ago
  really? epyc used a lot for host cpu to work with nvidia gpus, which is totally cuda, nobody uses rocm for ai.
  - unnah 4y ago
    Check out https://top500.org/lists/top500/2022/11/ https://top500.org/lists/top500/2022/11/ - of the top 10 supercomputers, 2 are based on AMD GPUs, 5 on NVIDIA, and 3 are more specialized solutions. All AMD has to do is offer lots of petaflops on the cheap, and software that technically works. Once the hardware is there, and research groups have won grants to work on it, it's up to grad students (who actually do the research work) to figure out how to get anything done on ROCm.
    - _yo2u 4y ago
      AFAIK AMD based supercomputers were chosen for their traditional HPC perf (eg. FP64) perf not AI perf (eg. BF16, FP16, FP8, INT8)
      - synergy20 4y ago
        That's true, MI300 might be the first real device geared towards AI indeed.
  - beebeepka 4y ago

nicolaslem 4y ago

Since very recently Arch community packages ROCm. At least it makes it a bit easier to try for yourself without having to jump through too many hoops. https://archlinux.org/packages/community/x86_64/rocm-opencl-runtime/ https://archlinux.org/packages/community/x86_64/rocm-opencl-...

matxip 4y ago
I built this and the other rocm packages from AUR (some removed now) along with pytorch build-flagged for rocm. It did eventually work, but It was a real pain in the ass and cost a whole lot of time. After that though, something broke (who knows) and I ended up rebuilding everything again just to get it back to a working state. I'm currently too afraid to update to these community packages, partially because I don't know if the build flags I used are the duct tape keeping my builds functional (I have an rx590-8gb which is maybe at the edge of supported gpus). Instead, I just added every rocm package to my IgnorePkgs, which isn't a great sign.
MayeulC 4y ago
Unfortunately, ROCm is broken for gfx803 (and possibly others?). I don't think they perform regression testing with their older cards. I think this is due to custom assembly kernels that are now outdated, I recall seeing somewhere that removing the optimized kernels and allowing to fall back to the generic implementation allowed it to work. ROCm is in such a sad state. If I can't make it work at home, why would I ever want to investigate it at work?