6 ms·

The way ROCm has turned out is such a shame. It really is effectively an advertisement for CUDA and served exactly that purpose for me. Even Intel's oneAPI stuf

The way ROCm has turned out is such a shame. It really is effectively an advertisement for CUDA and served exactly that purpose for me. Even Intel's oneAPI stuff seems to be heading in a more viable direction. I feel like AMD for some reason doesn't understand that support for GPGPU on the consumer stack is important to eventually getting adoption in the enterprise space. CUDA is so convenient in part because as long as you have an NVIDIA card it'll probably be usable for testing, even if not necessarily optimal or fast. Then, once confidence has been built about its usability, a more serious investment can be made into getting the more powerful cards.

Tostino 4y ago
Exactly, having your programs able to run on a developer's machine is really important for productivity.
mastax 4y ago
It's funny because AMD was at least rhetorically invested in consumer GPGPU a long time ago with the APU thing.
- pjmlp 4y ago
  Which were dropped when the open source driver came to be, hence why my Asus 1215B netbook can only do OpenGL 3.3, when it originally was capable of doing OpenGL 4.1 with fglrx.
- dotnet00 4y ago
  On top of that, GCN was a beautiful design for compute. Truly ahead of its time and a beast of a chip, as evident by how when running something they actually supported as well as NVIDIA chips, GCN cards would punch well above their gaming weight. It's just a huge shame that they never really got the software sorted out well enough to take advantage of the hardware.
buffet_overflow 4y ago
Exactly. I wish I could consider a 7900 XTX or two for "enthusiast" level ML work at home (thinking open sourced GPT models, Stable Diffusion, etc), but the state of library support has me pricing out 3090s instead. Sure, I could probably get ROCm going after a lot of tinkering and headaches but...that's not where I want to spend that effort. Would happily be proven wrong on this by the way.
- dotnet00 4y ago
  Yeah, I faced the same conundrum back in the Radeon VII/5700XT days. I had gone with a Radeon VII and a 5700XT, believing AMD to eventually support the latter in ML. Using the Radeon VII for ML was still pain, but I had more free time back then and was willing to tolerate the tinkering. The support for the 5700XT never came, was just constantly led on by AMD devs on their GitHub. I didn't really have that kind of free time when I wanted to upgrade, so buying 3090s ended up being the only realistic option. Even on Linux, as long as I'm not trying to stick to the latest bleeding edge kernel, the closed source drivers work very well. Thus far I feel the 3090s have been well worth the extra cost simply in terms of tinkering time saved.
  - buffet_overflow 4y ago
    I'm in the middle point of a similar journey. I got a 5700xt for a steal, and ran GPT2 on it to see if I could. It worked, but just barely. It now currently drives my primary monitors, and I keep a second GPU, a 1080ti, plugged in and passed through to a VM. I can then operate the 1080ti as a "program" from my desktop, which is fun, but somewhat limited given the space constraints in my case. Extra benefit is it can also be used for gaming. Next steps might be to toss 3090s in a dedicated chassis under Proxmox, but then I lose the part time gaming use case. The path to >40GB VRAM as a hobbyist is tricky, but I have to remind myself that even having it as an option is wild. What are you doing with your 3090s?
Dalewyn 4y ago
>support for GPGPU on the consumer stack is important to eventually getting adoption in the enterprise space. I've noticed as time goes on that more and more of the enterprise/mainstream tech that eventually become household names have their roots in ye olde vidja gaemz. All this technology powering all the algorithms today? Yeah so, it was originally so we could have better and better Unreal Tournaments and Dooms and Call of Dutys and Runescapes.
- beebeepka 4y ago
  Sort of. It's true a lot of this stuff was paid for by us gamers so that we could play Quake and stuff. UE is a lot more than a game engine these days. I've always been an id guy but I do consider UE a vital technology at this point. It really is because it can, and does, power so much more than games. That said, these GPU and AI things started as US military project which trickled down to us
  - paulmd 4y ago
    > That said, these GPU and AI things started as US military project which trickled down to us no, GPGPU started with opengl compute shaders, then CUDA packaged this into something not built around the graphics framework starting with the Tesla microarchitecture (6800/8800/200 series) and fully-programmable shaders. AI started really taking off during the maxwell era due to those gaming GPUs offering a fairly incredible amount of FP32 performance which was enough for ML (in contrast to prior HPC GPGPU which often focused on FP64). really at no point has GPU, GPGPU, or AI been a military thing that flowed to gaming, but completely the opposite. Or otherwise please explain what you mean because that's a very contrary perspective to the common view.
    - iforgotpassword 4y ago
      GPUs came from the enterprise, and to an extend, military space. Just three letters, SGI. They built mighty expensive 3D accelerators that were used in prohibitively expensive setups for training pilots for example. They never saw the potential of a consumer grade version of their tech. It should be well known that 3dfx was founded by ex SGI employees.
roenxi 4y ago
> I feel like AMD for some reason doesn't understand that support for GPGPU on the consumer stack is important to eventually getting adoption in the enterprise space. I made an attempt a few years ago at getting in to the machine learning world and discovered very quickly that I was facing a choice - either I could go AMD and open source drivers or Nvidia and cutting edge software. To learn the algorithms on my own PC just didn't seem feasible with an AMD GPU - running them on the CPU was too slow and I never found a way to make things work on the GPU with the resources I located. I gave up on AI since open source software is more important to me. Maybe I just lacked the right mindset, but the difficulties were so extreme it seems plausible that most people with an AMD GPU never learned to use ROCm at the amateur level and there wasn't a sufficiently good strategy to get it adopted at the professional level without an amateur community. Low level GPU programming is really hard. OpenGL, CUDA, OpenCL, whatever. I'm still on the lookout for an introductory resource on how I'm meant to use an AMD gpu to multiple 2 matrices together. I literally can't find one with a 5 minute google search, I get a blank article [0]. Maybe it is just blank for me? For using OpenGL I can just go play around [1]. There needs to be a strong community of advanced people explaining what to do which means big amateur support. So, in short, I strongly agree. I think AMDs (poor) support of consumer stack for GPGPU hurt them a lot more than they realise. OpenCL doesn't seem to cut it. The lack of libraries and community indicates that the feeder pipeline of amateur -> professional growth is broken and nobody is learning to use their platform even for fun. [0] https://developer.amd.com/article/a-heterogeneous-accelerated-matrix-multiplication-opencl-apu-gpu-fast-matrix-multiply/ https://developer.amd.com/article/a-heterogeneous-accelerate... [1] https://www.shadertoy.com/ https://www.shadertoy.com/
- heavyset_go 4y ago
  ROCm is a mess and I am similarly disappointed in AMD for the same reasons. Using AMD GPUs for accelerated compute on Linux is buggy and disappointing even if you get close to making it work.
- touisteur 4y ago
  Moreover they screwed over a big chunk of the people willing to use their hardware and their OpenCL stack (who might have migrated to ROCm if there was a not too painful path) when they EoLed their embedded GPU offer out of the blue. Now your options are TigerLake (and who knows whether Intel is going to keep the sycl/oneAPI support up, and I don't feel Intel really serious about GPUs in the long term... Or even in the 'open standard' shtick they're selling on oneAPI/SYCL) or NVIDIA and the CUDA collar.
okamiueru 4y ago
I'm not sure I agree with this. In my view the good alternative would have been OpenCL based machine learning libraries, no? Nvidia deliberately undermined that at every turn, and pushed a proprietary solution, hard. Almost all gpgpu based machine learning used it, and, what is AMD left with? HIP, the "CUDA wrapper to make it work on ROCm". What's the alternative here? Say that Nvidia shouldn't do anticompetitive things? That's been an exceedingly successful market strategy for them for quite some time.
- pjmlp 4y ago
  Don't blame NVidia for Intel and AMD's failure to do anything meaningful with OpenCL, or Google for completely ignoring it on Android and push Renderscript instead.
  - okamiueru 4y ago
    What on earth is your point? Let me make one thing clear, I could not care one bit about this team based mentality. Companies can be bad at each their thing. NVIDIA delayed driver support for OpenCL deliberately, because they wanted the only viable alternative for their users to be CUDA. Pushing ML universities to invest resources into that. Your point about AMD nok making something out of OpenCL isn't particularly fair... Maybe what you mean is that they should sponsor their own HPC labs at various universities? Is that "making something out of it"?
    - nightski 4y ago
      What's the advantage of OpenCL if it still only works good on Nvidia? Because it has "Open" in the name? You are acting like it was Nvidia holding OpenCL back, but I'm not convinced that was the case.
    - onphonenow 4y ago
      Nvidia delivered a good solution with cuda. You are whining about them not investing as hard in opencl. What if focusing on cuda was what they thought would let them deliver a good result for users? Lack of focus and sustained investment is part of issue w amd and others. If openCL is the hot thing that nvidia supposedly purposely delaying was a great opportunity for amd and intel to eat their lunch. Where were they?
microtonal 4y ago
It really is effectively an advertisement for CUDA 100%. At a previous employer, we had some budget left after buying a new NVIDIA GPU machine, but not enough to buy another bunch of workstation/server-class cards. So we decided to buy an additional machine with two Radeon VII, which were quite affordable for the compute it provided. So, we wanted to try if that is a viable path for the future. Unfortunately, ROCm was death by a thousand paper cuts. You could barely train any model without ROCm crashing with some memory order, weird performance regressions, or garbage output. The only thing we ever really got working was transformer training, with various ROCm libraries and PyTorch pinned to specific revisions and a bunch of patches. Luckily I had the foresight to encode all the dependencies with Nix and put all the Nix files in git, so we could reproduce this exact working version. We attempted to update the stack at various occasions, but it would always lead to more breakage. Two colleagues meticulously reported bugs for issues that were encountered. But AMD did not really seem interested to try to fix these things (or were just severely understaffed). After this experience I vowed to never buy AMD again for ML for at least a decade.
ogurechny 4y ago
The cycle hasn't really changed since 2007 and ATi Stream. Release some tool to program the GPU, expect someone else to do the rest, watch it fade into obscurity, rename it or release a new tool. Motivated users with simple tasks (hashing) would adapt to anything anyway, and so would industry grunts paid to deal with existing hardware and software. Still, each time they have announced the same magical “offloading of tasks to the GPU”, and the same magical “unified memory” to hold those tasks. Based on those sources, by now we are expected to have smart compilers that can take any Notepad and turn it into some CPU+GPU hybrid.
marcyb5st 4y ago
On top of what you said, I hoped that stable diffusion and others would have been a wake up call for AMD. It doesn't seem to be the case. I can imagine a not too distant feature where even "normal" users would run a plethora of ML models on their GPUs and if things stay this way third parties developers would have a tough time supporting AMD cards in a reliable way.

Tostino 4y ago

Exactly, having your programs able to run on a developer's machine is really important for productivity.

mastax 4y ago

It's funny because AMD was at least rhetorically invested in consumer GPGPU a long time ago with the APU thing.

pjmlp 4y ago
Which were dropped when the open source driver came to be, hence why my Asus 1215B netbook can only do OpenGL 3.3, when it originally was capable of doing OpenGL 4.1 with fglrx.
dotnet00 4y ago
On top of that, GCN was a beautiful design for compute. Truly ahead of its time and a beast of a chip, as evident by how when running something they actually supported as well as NVIDIA chips, GCN cards would punch well above their gaming weight. It's just a huge shame that they never really got the software sorted out well enough to take advantage of the hardware.

buffet_overflow 4y ago

Exactly. I wish I could consider a 7900 XTX or two for "enthusiast" level ML work at home (thinking open sourced GPT models, Stable Diffusion, etc), but the state of library support has me pricing out 3090s instead. Sure, I could probably get ROCm going after a lot of tinkering and headaches but...that's not where I want to spend that effort. Would happily be proven wrong on this by the way.

dotnet00 4y ago
Yeah, I faced the same conundrum back in the Radeon VII/5700XT days. I had gone with a Radeon VII and a 5700XT, believing AMD to eventually support the latter in ML. Using the Radeon VII for ML was still pain, but I had more free time back then and was willing to tolerate the tinkering. The support for the 5700XT never came, was just constantly led on by AMD devs on their GitHub. I didn't really have that kind of free time when I wanted to upgrade, so buying 3090s ended up being the only realistic option. Even on Linux, as long as I'm not trying to stick to the latest bleeding edge kernel, the closed source drivers work very well. Thus far I feel the 3090s have been well worth the extra cost simply in terms of tinkering time saved.
- buffet_overflow 4y ago
  I'm in the middle point of a similar journey. I got a 5700xt for a steal, and ran GPT2 on it to see if I could. It worked, but just barely. It now currently drives my primary monitors, and I keep a second GPU, a 1080ti, plugged in and passed through to a VM. I can then operate the 1080ti as a "program" from my desktop, which is fun, but somewhat limited given the space constraints in my case. Extra benefit is it can also be used for gaming. Next steps might be to toss 3090s in a dedicated chassis under Proxmox, but then I lose the part time gaming use case. The path to >40GB VRAM as a hobbyist is tricky, but I have to remind myself that even having it as an option is wild. What are you doing with your 3090s?

Dalewyn 4y ago

>support for GPGPU on the consumer stack is important to eventually getting adoption in the enterprise space. I've noticed as time goes on that more and more of the enterprise/mainstream tech that eventually become household names have their roots in ye olde vidja gaemz. All this technology powering all the algorithms today? Yeah so, it was originally so we could have better and better Unreal Tournaments and Dooms and Call of Dutys and Runescapes.

beebeepka 4y ago
Sort of. It's true a lot of this stuff was paid for by us gamers so that we could play Quake and stuff. UE is a lot more than a game engine these days. I've always been an id guy but I do consider UE a vital technology at this point. It really is because it can, and does, power so much more than games. That said, these GPU and AI things started as US military project which trickled down to us
- paulmd 4y ago
  > That said, these GPU and AI things started as US military project which trickled down to us no, GPGPU started with opengl compute shaders, then CUDA packaged this into something not built around the graphics framework starting with the Tesla microarchitecture (6800/8800/200 series) and fully-programmable shaders. AI started really taking off during the maxwell era due to those gaming GPUs offering a fairly incredible amount of FP32 performance which was enough for ML (in contrast to prior HPC GPGPU which often focused on FP64). really at no point has GPU, GPGPU, or AI been a military thing that flowed to gaming, but completely the opposite. Or otherwise please explain what you mean because that's a very contrary perspective to the common view.
  - iforgotpassword 4y ago
    GPUs came from the enterprise, and to an extend, military space. Just three letters, SGI. They built mighty expensive 3D accelerators that were used in prohibitively expensive setups for training pilots for example. They never saw the potential of a consumer grade version of their tech. It should be well known that 3dfx was founded by ex SGI employees.

roenxi 4y ago

> I feel like AMD for some reason doesn't understand that support for GPGPU on the consumer stack is important to eventually getting adoption in the enterprise space. I made an attempt a few years ago at getting in to the machine learning world and discovered very quickly that I was facing a choice - either I could go AMD and open source drivers or Nvidia and cutting edge software. To learn the algorithms on my own PC just didn't seem feasible with an AMD GPU - running them on the CPU was too slow and I never found a way to make things work on the GPU with the resources I located. I gave up on AI since open source software is more important to me. Maybe I just lacked the right mindset, but the difficulties were so extreme it seems plausible that most people with an AMD GPU never learned to use ROCm at the amateur level and there wasn't a sufficiently good strategy to get it adopted at the professional level without an amateur community. Low level GPU programming is really hard. OpenGL, CUDA, OpenCL, whatever. I'm still on the lookout for an introductory resource on how I'm meant to use an AMD gpu to multiple 2 matrices together. I literally can't find one with a 5 minute google search, I get a blank article [0]. Maybe it is just blank for me? For using OpenGL I can just go play around [1]. There needs to be a strong community of advanced people explaining what to do which means big amateur support. So, in short, I strongly agree. I think AMDs (poor) support of consumer stack for GPGPU hurt them a lot more than they realise. OpenCL doesn't seem to cut it. The lack of libraries and community indicates that the feeder pipeline of amateur -> professional growth is broken and nobody is learning to use their platform even for fun. [0] https://developer.amd.com/article/a-heterogeneous-accelerated-matrix-multiplication-opencl-apu-gpu-fast-matrix-multiply/ https://developer.amd.com/article/a-heterogeneous-accelerate... [1] https://www.shadertoy.com/ https://www.shadertoy.com/

heavyset_go 4y ago
ROCm is a mess and I am similarly disappointed in AMD for the same reasons. Using AMD GPUs for accelerated compute on Linux is buggy and disappointing even if you get close to making it work.
touisteur 4y ago
Moreover they screwed over a big chunk of the people willing to use their hardware and their OpenCL stack (who might have migrated to ROCm if there was a not too painful path) when they EoLed their embedded GPU offer out of the blue. Now your options are TigerLake (and who knows whether Intel is going to keep the sycl/oneAPI support up, and I don't feel Intel really serious about GPUs in the long term... Or even in the 'open standard' shtick they're selling on oneAPI/SYCL) or NVIDIA and the CUDA collar.

okamiueru 4y ago

I'm not sure I agree with this. In my view the good alternative would have been OpenCL based machine learning libraries, no? Nvidia deliberately undermined that at every turn, and pushed a proprietary solution, hard. Almost all gpgpu based machine learning used it, and, what is AMD left with? HIP, the "CUDA wrapper to make it work on ROCm". What's the alternative here? Say that Nvidia shouldn't do anticompetitive things? That's been an exceedingly successful market strategy for them for quite some time.

pjmlp 4y ago
Don't blame NVidia for Intel and AMD's failure to do anything meaningful with OpenCL, or Google for completely ignoring it on Android and push Renderscript instead.
- okamiueru 4y ago
  What on earth is your point? Let me make one thing clear, I could not care one bit about this team based mentality. Companies can be bad at each their thing. NVIDIA delayed driver support for OpenCL deliberately, because they wanted the only viable alternative for their users to be CUDA. Pushing ML universities to invest resources into that. Your point about AMD nok making something out of OpenCL isn't particularly fair... Maybe what you mean is that they should sponsor their own HPC labs at various universities? Is that "making something out of it"?
  - nightski 4y ago
    What's the advantage of OpenCL if it still only works good on Nvidia? Because it has "Open" in the name? You are acting like it was Nvidia holding OpenCL back, but I'm not convinced that was the case.
  - onphonenow 4y ago
    Nvidia delivered a good solution with cuda. You are whining about them not investing as hard in opencl. What if focusing on cuda was what they thought would let them deliver a good result for users? Lack of focus and sustained investment is part of issue w amd and others. If openCL is the hot thing that nvidia supposedly purposely delaying was a great opportunity for amd and intel to eat their lunch. Where were they?

microtonal 4y ago

It really is effectively an advertisement for CUDA 100%. At a previous employer, we had some budget left after buying a new NVIDIA GPU machine, but not enough to buy another bunch of workstation/server-class cards. So we decided to buy an additional machine with two Radeon VII, which were quite affordable for the compute it provided. So, we wanted to try if that is a viable path for the future. Unfortunately, ROCm was death by a thousand paper cuts. You could barely train any model without ROCm crashing with some memory order, weird performance regressions, or garbage output. The only thing we ever really got working was transformer training, with various ROCm libraries and PyTorch pinned to specific revisions and a bunch of patches. Luckily I had the foresight to encode all the dependencies with Nix and put all the Nix files in git, so we could reproduce this exact working version. We attempted to update the stack at various occasions, but it would always lead to more breakage. Two colleagues meticulously reported bugs for issues that were encountered. But AMD did not really seem interested to try to fix these things (or were just severely understaffed). After this experience I vowed to never buy AMD again for ML for at least a decade.

ogurechny 4y ago

The cycle hasn't really changed since 2007 and ATi Stream. Release some tool to program the GPU, expect someone else to do the rest, watch it fade into obscurity, rename it or release a new tool. Motivated users with simple tasks (hashing) would adapt to anything anyway, and so would industry grunts paid to deal with existing hardware and software. Still, each time they have announced the same magical “offloading of tasks to the GPU”, and the same magical “unified memory” to hold those tasks. Based on those sources, by now we are expected to have smart compilers that can take any Notepad and turn it into some CPU+GPU hybrid.

marcyb5st 4y ago

On top of what you said, I hoped that stable diffusion and others would have been a wake up call for AMD. It doesn't seem to be the case. I can imagine a not too distant feature where even "normal" users would run a plethora of ML models on their GPUs and if things stay this way third parties developers would have a tough time supporting AMD cards in a reliable way.