17 ms·

The trouble is that AMD just didn't take AI seriously. For a long time, their equivalent of CUDA was not only Linux only and a pain to use but was outright brok

by makomk 3y ago

The trouble is that AMD just didn't take AI seriously. For a long time, their equivalent of CUDA was not only Linux only and a pain to use but was outright broken on all consumer cards - as in, they dropped official support for the only consumer cards it officially ran on, promptly broke it so that machine learning runs failed, and dismissed bug reports from the users who were left high and dry because their cards were no longer officially supported. The only way to use AMD for machine learning was to pay out much more than the price of NVidia's consumer cards for server-focused AMD cards that worked worse, were harder to use, and that AMD didn't support for long either. They just never had the small-scale desktop usage that lead to NVidia's cards being the choice for bigger machine learning once scaled up because it simply didn't work.

ChuckNorris89 3y ago
>their equivalent of CUDA Nvidia didn't pull their AI leadership out of thin air overnight, but they shipped the first CUDA capable consumer cards with the GeForce 8000 series way back in 2007 and committed to this ecosystem over the years, consistently investing in the HW, SW. By the time AMD woke up and shipped ROCm in 2016, Nvidia already had nearly 10 years head start and a cemented moat in this field. AMD now has a huge mountain to climb to catch up to Nvidia.
- mahkeiro 3y ago
  I always had hope that ROCm will be able to compete with CUDA but it’s nowhere here despite the time. Seems funny to see that Intel is doing a better job at that with OneAPI.
- bryanlarsen 3y ago
  AMD invested significantly into OpenCL prior to 2016. It seemed like a safe bet -- the open industry standard usually ends up beating the proprietary standard in the long run. Especially for something like this, with massive open source / open standard companies like Google as heavy users. It seems surprising to me that Google didn't ensure that open standards won in an area that they are so heavily dependent on.
  - ChuckNorris89 3y ago
    Because all the researchers that used GPUs for CV and ML used what they had at their disposal, which was Nvidia GPUs and CUDA. OpenCL brought no advantage here considering it only worked on AMD GPUs which were lackluster in performance and switching from CUDA to OpenCL meant extra work that researches already iterating on CUDA weren't willing to do.
    - cibyr 3y ago
      OpenCL works fine on Nvidia GPUs. It's more annoying to code for, and harder to get good performance out of than CUDA, but it works.
      - dragontamer 3y ago
        Poorly, in my experience. CUDA is compiled into PTX, an intermediate language. PTX is then compiled into a specific NVidia assembly language (often called SASS, though each SASS for each generation of cards is different). This way, NVidia can make huge changes to the underlying assembly code from generation-to-generation, but still have portability. OpenCL, especially OpenCL 1.2, (which is the version of OpenCL that works on the widest set of cards), does not have an intermediate language. SPIR is an OpenCL2.+ concept. This means that OpenCL 1.2 code is distributed in source and recompiled in practice. But that means that compiler errors can kill your code before it even runs. This is especially annoying because the OpenCL 1.2 compiler is part of a device-driver. Meaning if the end-user updates the device driver, the compiler may have a new bug (or old bug), that changes the behavior of your code. ------------- This doesn't matter for DirectX, because like CUDA, Microsoft compiles DirectX into DXIR / DirectX intermediate language. And then has device drivers compile the intermediate-language into the final assembly code on a per-device basis. ------------- It is this intermediate layer that AMD is missing, and IMO is the key to their problems in practice. SPIR (OpenCL's standard intermediate layer) has spotty support across cards. I'm guessing NVidia knows that PTX intermediate language is their golden goose and doesn't want to offer good SPIR support. Microsoft probably prefers people to use DirectX / DXIR as well. So that leaves AMD and Intel as the only groups who could possibly push SPIR and align together. SPIR is a good idea, but I'm not sure if the politics will allow it to happen.
- Keyframe 3y ago
  Nvidia took a big gamble with CUDA and it took years and ton of investment to get there. Jensen Huang talks about it on commencement speech he did in Taiwan recently here https://www.youtube.com/watch?v=oi89u6q0_AY https://www.youtube.com/watch?v=oi89u6q0_AY It's a big moat to cross.
starkd 3y ago
That would explain my Radeon Graphics card I never managed to get working properly. It arbitrarily froze. I was told that it did that for Linux and that it was guaranteed to work on Windows. But when I tried it on Windows, it did the exact same thing. They were unresponsive.
emmender 3y ago
the range of technology that needs to come together for ai training is underestimated. there is cuda of course, but there is also nccl, infiniband, gpudirect, each of which requires years of sw and hw maturity. unlike the cpu which has a clean interface (instruction set) the gpu has no such thing - it is more like an octopus with tentacles into networking, compute, storage etc.
deleted 3y ago
[deleted]
amunicio 3y ago
> The trouble is that AMD just didn't take AI seriously. Until a couple of years ago, AMD was in survival mode, fighting Intel on one side and Nvidia on the other. Two rivals that were making money hand over fist while AMD was bleeding money. AMD picked open standards and made investments on open source frameworks and libraries commensurate with their financials, the hope being that the community could help pick up some of the slack. The community, understandably, went with the proprietary solution that worked well at the time and had resources behind. The net results is that the Nvidia ecosystem has gained a dominant position in the industry and benefits from being perceived as a quasi-standard. On the other hand, open source efforts by AMD or others get viewed as "not serious". The financial situation of AMD has improved somewhat over the last couple years. So AMD is "taking AI more seriously now". But it might be too late and the proprietary ecosystem has probably won.
- parker_mountain 3y ago
  For what it's worth, AMD is also incredibly proprietary. The drivers being open source really helps with compatibility and your kernel, but you're still interacting with a massive computer running it's own OS with its own trusted code solution. And that computer also has DMA to your computer. I would consider their open efforts to be "not serious" for anyone but the consumer space - games, desktop users, maybe even professional text editors. If you're using the GPUs for "professional" applications in a one-off scenario, even AMD falls short. I'm honestly not sure what the moral of this story is.
  - wmf 3y ago
    The moral is that PSP FUD has nothing to do with AMD's lack of success in AI.
  - beebeepka 3y ago
    Are they "incredibly proprietary" compared to the competition? Clearly they aren't. Nvidia offers blobs in both consumer and professional markets. Even going to the extent of gimping performance hardware through drivers on more than one occasion. That said, I think AMD isn't really competing with Nvidia. Sure, their R&D budget is smallish but it feels like they're somewhat fine with the current status quo.
superkuh 3y ago
The AMD RX 580 was released in April 2018. AMD had already dropped ROCm/HIP support for it by 2021. They only supported the card for 3 years. 3 years. It's so lame it's bordering on fraudulent, even if not legally fraud. I know CUDA via their HIP is a moving target they don't control making it hard and expensive for them to prevent bit rot but this is still an AMD caused problem due to opencl not getting any love by anyone anymore. AMD included.
- paulmd 3y ago
  Also, while AMD's OpenCL implementation has more features on paper, the runtime is frequently broken where NVIDIA's claimed features actually all work. Everything I've heard from people who've used it is that they ended up with so much vendor-specific code to patch around AMD's bugs and deficiencies that they might as well have just written CUDA in the first place. This is an old article but the old "vendor B" stuff still rings incredibly true with at least AMD's OpenCL stack as well. https://richg42.blogspot.com/2014/05/the-truth-on-opengl-driver-quality.html https://richg42.blogspot.com/2014/05/the-truth-on-opengl-dri... Thus NVIDIA actually has even less of a lock-in than people think. If you want to write a better OneAPI ecosystem and run it on OpenCL runtime... go hog wild! NVIDIA is best at that too! You just don't get the benefit of NVIDIA's engineers writing libraries for you.
  - deleted 3y ago
    [deleted]
  - paulmd 3y ago
    btw as far as anyone insisting "ROCm is good now"... see a practical example in action. https://github.com/RadeonOpenCompute/ROCm/issues/2198#issuecomment-1574383483 https://github.com/RadeonOpenCompute/ROCm/issues/2198#issuec...
- JonChesterfield 3y ago
  You dont have to change to a new toolchain. If the 2021 version was working with your card, keep using it.
osti 3y ago
This is my thought as well. Their devs who work on the graphics drivers are heavily underpaid in Canada. As an ex-AMD, I took AMD's offer for 20k less than another software company because I like low level stuff and have always been an AMD fan since the Athlon days. But when Amazon offered me double, I easily took Amazon's offer and left AMD after less than a year. I think a lot of companies are overpaying their software people, but if there is any one that should pay their devs much more it would be AMD, because they are in a position to compete against Nvidia if their software integrated well with the AI training stuff.
- jrockway 3y ago
  Why do you think that people are being overpaid if you literally left a job you liked to make more money? Being underpaid is just being underpaid; just because the numbers are high at the big tech companies doesn't mean that those companies are overpaying. They're probably underpaying! The dollar just isn't worth what it used to be. (Canada does chronically underpay its software engineers, though.)
  - osti 3y ago
    Not going to argue whether we are being overpaid or not (I certainly hope we aren't because now I'm getting even crazier compensation than my Amazon days lol). But I think the current layoffs which are putting downward pressures on salaries will prove that we were getting overpaid. My main point was still that AMD should really pay a lot more than what they currently are paying, they actually already increased it quite a bit compared to 3 years ago, but not nearly enough! In a way, I think this reflects poorly on Lisa Su because she didn't invest enough into AI while it should have been obvious from the start.
croes 3y ago
AI is only one use case of CUDA and at first it wasn't the main use case.
amelius 3y ago
> The trouble is that AMD just didn't take AI seriously. No worries, AI is not very complicated tech. It's just a core that can do arithmetic (something AMD already knows how to do very well) copied a very large number of times on a chip, plus some interconnect. CPUs with all their speculative execution and random memory access patterns are much more complicated.
- dotnet00 3y ago
  AI is more than just the underlying math. The software ecosystem is very important, which is what NVIDIA's lead is built on. AMD has a very hard time providing an "it just works" type experience in the way that NVIDIA offers these days. Machine learning engineers (or most people writing GPU code) do not typically have the time, knowledge or interest to diagnose driver issues and beg AMD engineers to address them in a reasonable time frame.

by makomk 3y ago

ChuckNorris89 3y ago

>their equivalent of CUDA Nvidia didn't pull their AI leadership out of thin air overnight, but they shipped the first CUDA capable consumer cards with the GeForce 8000 series way back in 2007 and committed to this ecosystem over the years, consistently investing in the HW, SW. By the time AMD woke up and shipped ROCm in 2016, Nvidia already had nearly 10 years head start and a cemented moat in this field. AMD now has a huge mountain to climb to catch up to Nvidia.

mahkeiro 3y ago
I always had hope that ROCm will be able to compete with CUDA but it’s nowhere here despite the time. Seems funny to see that Intel is doing a better job at that with OneAPI.
bryanlarsen 3y ago
AMD invested significantly into OpenCL prior to 2016. It seemed like a safe bet -- the open industry standard usually ends up beating the proprietary standard in the long run. Especially for something like this, with massive open source / open standard companies like Google as heavy users. It seems surprising to me that Google didn't ensure that open standards won in an area that they are so heavily dependent on.
- ChuckNorris89 3y ago
  Because all the researchers that used GPUs for CV and ML used what they had at their disposal, which was Nvidia GPUs and CUDA. OpenCL brought no advantage here considering it only worked on AMD GPUs which were lackluster in performance and switching from CUDA to OpenCL meant extra work that researches already iterating on CUDA weren't willing to do.
  - cibyr 3y ago
    OpenCL works fine on Nvidia GPUs. It's more annoying to code for, and harder to get good performance out of than CUDA, but it works.
    - dragontamer 3y ago
      Poorly, in my experience. CUDA is compiled into PTX, an intermediate language. PTX is then compiled into a specific NVidia assembly language (often called SASS, though each SASS for each generation of cards is different). This way, NVidia can make huge changes to the underlying assembly code from generation-to-generation, but still have portability. OpenCL, especially OpenCL 1.2, (which is the version of OpenCL that works on the widest set of cards), does not have an intermediate language. SPIR is an OpenCL2.+ concept. This means that OpenCL 1.2 code is distributed in source and recompiled in practice. But that means that compiler errors can kill your code before it even runs. This is especially annoying because the OpenCL 1.2 compiler is part of a device-driver. Meaning if the end-user updates the device driver, the compiler may have a new bug (or old bug), that changes the behavior of your code. ------------- This doesn't matter for DirectX, because like CUDA, Microsoft compiles DirectX into DXIR / DirectX intermediate language. And then has device drivers compile the intermediate-language into the final assembly code on a per-device basis. ------------- It is this intermediate layer that AMD is missing, and IMO is the key to their problems in practice. SPIR (OpenCL's standard intermediate layer) has spotty support across cards. I'm guessing NVidia knows that PTX intermediate language is their golden goose and doesn't want to offer good SPIR support. Microsoft probably prefers people to use DirectX / DXIR as well. So that leaves AMD and Intel as the only groups who could possibly push SPIR and align together. SPIR is a good idea, but I'm not sure if the politics will allow it to happen.
Keyframe 3y ago
Nvidia took a big gamble with CUDA and it took years and ton of investment to get there. Jensen Huang talks about it on commencement speech he did in Taiwan recently here https://www.youtube.com/watch?v=oi89u6q0_AY https://www.youtube.com/watch?v=oi89u6q0_AY It's a big moat to cross.

starkd 3y ago

That would explain my Radeon Graphics card I never managed to get working properly. It arbitrarily froze. I was told that it did that for Linux and that it was guaranteed to work on Windows. But when I tried it on Windows, it did the exact same thing. They were unresponsive.

emmender 3y ago

the range of technology that needs to come together for ai training is underestimated. there is cuda of course, but there is also nccl, infiniband, gpudirect, each of which requires years of sw and hw maturity. unlike the cpu which has a clean interface (instruction set) the gpu has no such thing - it is more like an octopus with tentacles into networking, compute, storage etc.

deleted 3y ago

[deleted]

amunicio 3y ago

> The trouble is that AMD just didn't take AI seriously. Until a couple of years ago, AMD was in survival mode, fighting Intel on one side and Nvidia on the other. Two rivals that were making money hand over fist while AMD was bleeding money. AMD picked open standards and made investments on open source frameworks and libraries commensurate with their financials, the hope being that the community could help pick up some of the slack. The community, understandably, went with the proprietary solution that worked well at the time and had resources behind. The net results is that the Nvidia ecosystem has gained a dominant position in the industry and benefits from being perceived as a quasi-standard. On the other hand, open source efforts by AMD or others get viewed as "not serious". The financial situation of AMD has improved somewhat over the last couple years. So AMD is "taking AI more seriously now". But it might be too late and the proprietary ecosystem has probably won.

parker_mountain 3y ago
For what it's worth, AMD is also incredibly proprietary. The drivers being open source really helps with compatibility and your kernel, but you're still interacting with a massive computer running it's own OS with its own trusted code solution. And that computer also has DMA to your computer. I would consider their open efforts to be "not serious" for anyone but the consumer space - games, desktop users, maybe even professional text editors. If you're using the GPUs for "professional" applications in a one-off scenario, even AMD falls short. I'm honestly not sure what the moral of this story is.
- wmf 3y ago
  The moral is that PSP FUD has nothing to do with AMD's lack of success in AI.
- beebeepka 3y ago
  Are they "incredibly proprietary" compared to the competition? Clearly they aren't. Nvidia offers blobs in both consumer and professional markets. Even going to the extent of gimping performance hardware through drivers on more than one occasion. That said, I think AMD isn't really competing with Nvidia. Sure, their R&D budget is smallish but it feels like they're somewhat fine with the current status quo.

superkuh 3y ago

The AMD RX 580 was released in April 2018. AMD had already dropped ROCm/HIP support for it by 2021. They only supported the card for 3 years. 3 years. It's so lame it's bordering on fraudulent, even if not legally fraud. I know CUDA via their HIP is a moving target they don't control making it hard and expensive for them to prevent bit rot but this is still an AMD caused problem due to opencl not getting any love by anyone anymore. AMD included.

paulmd 3y ago
Also, while AMD's OpenCL implementation has more features on paper, the runtime is frequently broken where NVIDIA's claimed features actually all work. Everything I've heard from people who've used it is that they ended up with so much vendor-specific code to patch around AMD's bugs and deficiencies that they might as well have just written CUDA in the first place. This is an old article but the old "vendor B" stuff still rings incredibly true with at least AMD's OpenCL stack as well. https://richg42.blogspot.com/2014/05/the-truth-on-opengl-driver-quality.html https://richg42.blogspot.com/2014/05/the-truth-on-opengl-dri... Thus NVIDIA actually has even less of a lock-in than people think. If you want to write a better OneAPI ecosystem and run it on OpenCL runtime... go hog wild! NVIDIA is best at that too! You just don't get the benefit of NVIDIA's engineers writing libraries for you.
- deleted 3y ago
  [deleted]
- paulmd 3y ago
  btw as far as anyone insisting "ROCm is good now"... see a practical example in action. https://github.com/RadeonOpenCompute/ROCm/issues/2198#issuecomment-1574383483 https://github.com/RadeonOpenCompute/ROCm/issues/2198#issuec...
JonChesterfield 3y ago
You dont have to change to a new toolchain. If the 2021 version was working with your card, keep using it.

osti 3y ago

This is my thought as well. Their devs who work on the graphics drivers are heavily underpaid in Canada. As an ex-AMD, I took AMD's offer for 20k less than another software company because I like low level stuff and have always been an AMD fan since the Athlon days. But when Amazon offered me double, I easily took Amazon's offer and left AMD after less than a year. I think a lot of companies are overpaying their software people, but if there is any one that should pay their devs much more it would be AMD, because they are in a position to compete against Nvidia if their software integrated well with the AI training stuff.

jrockway 3y ago
Why do you think that people are being overpaid if you literally left a job you liked to make more money? Being underpaid is just being underpaid; just because the numbers are high at the big tech companies doesn't mean that those companies are overpaying. They're probably underpaying! The dollar just isn't worth what it used to be. (Canada does chronically underpay its software engineers, though.)
- osti 3y ago
  Not going to argue whether we are being overpaid or not (I certainly hope we aren't because now I'm getting even crazier compensation than my Amazon days lol). But I think the current layoffs which are putting downward pressures on salaries will prove that we were getting overpaid. My main point was still that AMD should really pay a lot more than what they currently are paying, they actually already increased it quite a bit compared to 3 years ago, but not nearly enough! In a way, I think this reflects poorly on Lisa Su because she didn't invest enough into AI while it should have been obvious from the start.

croes 3y ago

AI is only one use case of CUDA and at first it wasn't the main use case.

amelius 3y ago

> The trouble is that AMD just didn't take AI seriously. No worries, AI is not very complicated tech. It's just a core that can do arithmetic (something AMD already knows how to do very well) copied a very large number of times on a chip, plus some interconnect. CPUs with all their speculative execution and random memory access patterns are much more complicated.

dotnet00 3y ago
AI is more than just the underlying math. The software ecosystem is very important, which is what NVIDIA's lead is built on. AMD has a very hard time providing an "it just works" type experience in the way that NVIDIA offers these days. Machine learning engineers (or most people writing GPU code) do not typically have the time, knowledge or interest to diagnose driver issues and beg AMD engineers to address them in a reasonable time frame.