5 ms·

I really hope this breaks Nvidia's stranglehold on deep learning. Some competition would hopefully bring down prices at the compute high-end. AMD don't seem to

I really hope this breaks Nvidia's stranglehold on deep learning. Some competition would hopefully bring down prices at the compute high-end. AMD don't seem to even be trying on the software-side at the moment. ROCm is a mess.

jjcon 5y ago
I wholeheartedly agree. PyTorch did recently release AMD support which I was happy to see (though I have not tested it), I’m hoping there is more to come. https://pytorch.org/blog/pytorch-for-amd-rocm-platform-now-available-as-python-package/ https://pytorch.org/blog/pytorch-for-amd-rocm-platform-now-a...
- byefruit 5y ago
  Unfortunately that support is via ROCm, which doesn't support the last three generations (!) of AMD hardware: https://github.com/ROCm/ROCm.github.io/blob/master/hardware.md https://github.com/ROCm/ROCm.github.io/blob/master/hardware....
  - dragontamer 5y ago
    ROCm supports Vega, Vega 7nm, and CDNA just fine. The issue is that AMD has split their compute into two categories: * RDNA -- consumer cards. A new ISA with new compilers / everything. I don't think its reasonable to expect AMD's compilers to work on RDNA, when such large changes have been made to the architecture. (32-wide instead of 64-wide. 1024 registers. Etc. etc.) * CDNA -- based off of Vega's ISA. Despite being "legacy ISA", its pretty modern in terms of capabilities. MI100 is competitive against the A100. CDNA is likely going to run Frontier and El Capitan supercomputers. ------------ ROCm focused on CDNA. They've had compilers emit RDNA code, but its not "official" and still buggy. But if you went for CDNA, that HIP / ROCm stuff works enough for the Oak Ridge National Labs. Yeah, CDNA is expensive ($5k for MI50 / Radeon VII, and $9k for MI100). But that's the price of full-speed scientific-oriented double-precision floating point GPUs these days.
    - byefruit 5y ago
      That makes a lot more sense, thanks. They could do with making that a lot clearer on the project. Still handicaps them compared to Nvidia where you can just buy anything recent and expect it to work. Suspect it also means they get virtually no open source contributions from the community because nobody can run or test it on personal hardware.
      - dragontamer 5y ago
        NVidia can support anything because they have a PTX-translation layer between cards, and invest heavily on PTX. Each assembly language from each generation of cards changes. PTX recompiles the "pseudo-assembly" instructions into the new assembly code each generation. --------- AMD has no such technology. When AMD's assembly language changes (ex: from Vega into RDNA), its a big compiler change. AMD managed to keep the ISA mostly compatible from 7xxx GCN 1.0 series in the late 00s all the way to Vega 7nm in the late 10s... but RDNA's ISA change was pretty massive. I think its only natural that RDNA was going to have compiler issues. --------- AMD focused on Vulkan / DirectX support for its RDNA cards, while its compute team focused on continuing "CDNA" (which won large supercomputer contracts). So that's just how the business ended up.
      - blagie 5y ago
        I bought an ATI card for deep learning. I'm a big fan of open source. Less than 12 months later, ROCm dropped support. I bought an NVidia, and I'm not looking back. This makes absolutely no sense to me, and I have a Ph.D: "* RDNA -- consumer cards. A new ISA with new compilers / everything. I don't think its reasonable to expect AMD's compilers to work on RDNA, when such large changes have been made to the architecture. (32-wide instead of 64-wide. 1024 registers. Etc. etc.) * CDNA -- based off of Vega's ISA. Despite being "legacy ISA", its pretty modern in terms of capabilities. MI100 is competitive against the A100. CDNA is likely going to run Frontier and El Capitan supercomputers. ROCm focused on CDNA. They've had compilers emit RDNA code, but its not "official" and still buggy. But if you went for CDNA, that HIP / ROCm stuff works enough for the Oak Ridge National Labs. Yeah, CDNA is expensive ($5k for MI50 / Radeon VII, and $9k for MI100). But that's the price of full-speed scientific-oriented double-precision floating point GPUs these days. I neither know nor care what RDNA, CDNA, A100, MI50, Radeon VII, MI100, or all the other AMD acronyms are. Yes, I could figure it out, but I want plug-and-play, stability, and backwards-compatibility. I ran into a whole different minefield with AMD. I'd need to run old ROCm, downgrade my kernel, and use a different card to drive monitors than for ROCm. It was a mess. NVidia gave me plug-and-play. I bought a random NVidia card with the highest "compute level," and was confident everything would work. It does. I'm happy. Intel has historically had great open source drivers, and if it give better plug-and-play and open source, I'll buy Intel next time. I'm skeptical, though. The past few year, Intel has a hard time tying their own shoelaces. I can't imagine this will be different.
rowanG077 5y ago
I think this situation can only be fixed by moving up into languages that compile to vendor specific GPU languages. Just treat CUDA, openCL, vulkan compute, metal compute(??) etc. as the assembly of graphics cards.
- T-A 5y ago
  https://www.oneapi.io/ https://www.oneapi.io/
  - snicker7 5y ago
    Currently only supported by Intel.
    - T-A 5y ago
      https://www.codeplay.com/solutions/oneapi/for-cuda/ https://www.codeplay.com/solutions/oneapi/for-cuda/ https://www.urz.uni-heidelberg.de/en/newsroom/oneapi-academic-center-of-excellence-established-at-the-heidelberg-university-computing-centre-urz https://www.urz.uni-heidelberg.de/en/newsroom/oneapi-academi...
- hobofan 5y ago
  Barely anyone is writing CUDA directly these days. Just add support in PyTorch and Tensorflow and you've covered probably 90% of the deep learning market.
  - hprotagonist 5y ago
    and ONNX.
- pjmlp 5y ago
  That is just part of the story. CUDA wiped out OpenCL, because it went polyglot as of version 3.0, while insisting that everyone should write in a C dialect. They also provide great graphical debugging tools and libraries. Khronos waited too long to introduce SPIR, and in traditional Khronos fashion, waited for the partners to provide the tooling. One could blame NVidia, but it isn't as the competition has done a better job.
rektide 5y ago
ROCm seems to be tolerably decent, if you are willing to spend a couple hours, and, big if, if HIP supports all the various libraries you were relying on. CUDA has a huge support library, and ROCm has been missing not just the small fry stuff but a lot of the core stuff in the that library. Long term, AI (& a lot of other interests) need to serve themselves. CUDA is excellently convenient, but long term I have a hard time imagining there being a worthwhile future for anything but Vulkan. There don't seem to be a lot of forays into writing good all-encompassing libraries in Vulkan yet, nor many more specialized AI/ML Vulkan libraries, so it feels largely like we more or less haven't started really trying.
- dnautics 5y ago
  is there any indication that ROCm has solved its stability issues? I wasn't doing the testing myself, but the reason why we rejected ROCm a while back (2 years?) was because you could get segfaults hours into a ML training run, which is... frustrating, to say the least, and not easily identifiable in quickie test runs (or CI, if ML did more CI).
- rektide 5y ago
  Lot of downvotes. Anyone have any opinion? Is CUDA fine forever? Is there something other than Vulkan we should also try? Do you think AMD should solve every problem CUDA solves for their customers too? What gives here? I see a lot a lot a lot of resistance to the idea that we should start trying to align to Vulkan. Here & elsewhere. I don't get it, it makes no sense, & everyone else using GPU's is running fast as they can towards Vulkan. Is it just too soon too early in the adoption curve, or do ya'll think there are more serious obstructions long term to building a more Vulkan centric AI/ML toolkit? It still feels inevitable to me. What we are doing now feels like a waste of time. I wish ya'll wouldn't downvote so casually, wouldn't just try to brush this viewpoint away.
  - BadInformatics 5y ago
    > Do you think AMD should solve every problem CUDA solves for their customers too? They had no choice. Getting a bunch of HPC people to completely rewrite their code for a different API is a tough pill to swallow when you're trying to win supercomputer contracts. Would they have preferred to spend development resources elsewhere? Probably, they've even got their own standards and SDKs from days past. > everyone else using GPU's is running fast as they can towards Vulkan I'm not qualified to comment on the entirety of it, but I can say that basically no claim in this statement is true: 1. Not everyone doing compute is using GPUs. Companies are increasingly designing and releasing their own custom hardware (TPUs, IPUs, NPUs, etc.) 2. Not everyone using GPUs is cares about Vulkan. Certainly many folks doing graphics stuff don't, and DirectX is as healthy as ever. There have been bits and pieces of work around Vulkan compute for mobile ML model deployment, but it's a tiny niche and doesn't involve discrete GPUs at all. > Is it just too soon too early in the adoption curve Yes. Vulkan compute is still missing many of the niceties of more developed compute APIs. Tooling is one big part of that: writing shaders using GLSL is a pretty big step down from using whatever language you were using before (C++, Fortran, Python, etc). > do ya'll think there are more serious obstructions long term to building a more Vulkan centric AI/ML toolkit You could probably write a whole page about this, but TL;DR yes. It would take at least as much effort as AMD and Intel put into their respective compute stacks to get Vulkan ML anywhere near ready for prime time. You need to have inference, training, cross-device communication, headless GPU usage, reasonably wide compatibility, not garbage performance, framework integration, passable tooling and more. Sure these are all feasible, but who has the incentive to put in the time to do it? The big 3 vendors have their supercomputer contracts already, so all they need to do is keep maintaining their 1st-party compute stacks. Interop also requires going through Khronos, which is its own political quagmire when it comes to standardization. Nvidia already managed to obstruct OpenCL into obscurity, why would they do anything different here? Downstream libraries have also poured untold millions into existing compute stacks, OR rely on the vendors to implement that functionality for them. This is before we even get into custom hardware like TPUs that don't behave like a GPU at all. So in short, there is little inevitable about this at all. The reason people may have been frustrated by your comment is because Vulkan compute comes up all the time as some silver bullet that will save us from the walled gardens of CUDA and co (especially for ML, arguably the most complex and expensive subdomain of them all). We'd all like it to come true, but until all of the aforementioned points are addressed this will remain primarily in pipe dream territory.
lvl100 5y ago
I agree 100% and if Nvidia’s recent showing and puzzling focus on “Omniverse” is any indication, they’re operating in a fantasy world a bit.
pjmlp 5y ago
You know how to break it? With modern tooling. Instead of forcing devs to live in the pre-historic days of C dialects and printf debugging, provide polyglot IDEs with graphical debugging tools capable of single step GPU shaders and a rich libraries ecosystem. Khronos got the message too late and now no one cares.

jjcon 5y ago

I wholeheartedly agree. PyTorch did recently release AMD support which I was happy to see (though I have not tested it), I’m hoping there is more to come. https://pytorch.org/blog/pytorch-for-amd-rocm-platform-now-available-as-python-package/ https://pytorch.org/blog/pytorch-for-amd-rocm-platform-now-a...

byefruit 5y ago
Unfortunately that support is via ROCm, which doesn't support the last three generations (!) of AMD hardware: https://github.com/ROCm/ROCm.github.io/blob/master/hardware.md https://github.com/ROCm/ROCm.github.io/blob/master/hardware....
- dragontamer 5y ago
  ROCm supports Vega, Vega 7nm, and CDNA just fine. The issue is that AMD has split their compute into two categories: * RDNA -- consumer cards. A new ISA with new compilers / everything. I don't think its reasonable to expect AMD's compilers to work on RDNA, when such large changes have been made to the architecture. (32-wide instead of 64-wide. 1024 registers. Etc. etc.) * CDNA -- based off of Vega's ISA. Despite being "legacy ISA", its pretty modern in terms of capabilities. MI100 is competitive against the A100. CDNA is likely going to run Frontier and El Capitan supercomputers. ------------ ROCm focused on CDNA. They've had compilers emit RDNA code, but its not "official" and still buggy. But if you went for CDNA, that HIP / ROCm stuff works enough for the Oak Ridge National Labs. Yeah, CDNA is expensive ($5k for MI50 / Radeon VII, and $9k for MI100). But that's the price of full-speed scientific-oriented double-precision floating point GPUs these days.
  - byefruit 5y ago
    That makes a lot more sense, thanks. They could do with making that a lot clearer on the project. Still handicaps them compared to Nvidia where you can just buy anything recent and expect it to work. Suspect it also means they get virtually no open source contributions from the community because nobody can run or test it on personal hardware.
    - dragontamer 5y ago
      NVidia can support anything because they have a PTX-translation layer between cards, and invest heavily on PTX. Each assembly language from each generation of cards changes. PTX recompiles the "pseudo-assembly" instructions into the new assembly code each generation. --------- AMD has no such technology. When AMD's assembly language changes (ex: from Vega into RDNA), its a big compiler change. AMD managed to keep the ISA mostly compatible from 7xxx GCN 1.0 series in the late 00s all the way to Vega 7nm in the late 10s... but RDNA's ISA change was pretty massive. I think its only natural that RDNA was going to have compiler issues. --------- AMD focused on Vulkan / DirectX support for its RDNA cards, while its compute team focused on continuing "CDNA" (which won large supercomputer contracts). So that's just how the business ended up.
    - blagie 5y ago
      I bought an ATI card for deep learning. I'm a big fan of open source. Less than 12 months later, ROCm dropped support. I bought an NVidia, and I'm not looking back. This makes absolutely no sense to me, and I have a Ph.D: "* RDNA -- consumer cards. A new ISA with new compilers / everything. I don't think its reasonable to expect AMD's compilers to work on RDNA, when such large changes have been made to the architecture. (32-wide instead of 64-wide. 1024 registers. Etc. etc.) * CDNA -- based off of Vega's ISA. Despite being "legacy ISA", its pretty modern in terms of capabilities. MI100 is competitive against the A100. CDNA is likely going to run Frontier and El Capitan supercomputers. ROCm focused on CDNA. They've had compilers emit RDNA code, but its not "official" and still buggy. But if you went for CDNA, that HIP / ROCm stuff works enough for the Oak Ridge National Labs. Yeah, CDNA is expensive ($5k for MI50 / Radeon VII, and $9k for MI100). But that's the price of full-speed scientific-oriented double-precision floating point GPUs these days. I neither know nor care what RDNA, CDNA, A100, MI50, Radeon VII, MI100, or all the other AMD acronyms are. Yes, I could figure it out, but I want plug-and-play, stability, and backwards-compatibility. I ran into a whole different minefield with AMD. I'd need to run old ROCm, downgrade my kernel, and use a different card to drive monitors than for ROCm. It was a mess. NVidia gave me plug-and-play. I bought a random NVidia card with the highest "compute level," and was confident everything would work. It does. I'm happy. Intel has historically had great open source drivers, and if it give better plug-and-play and open source, I'll buy Intel next time. I'm skeptical, though. The past few year, Intel has a hard time tying their own shoelaces. I can't imagine this will be different.

rowanG077 5y ago

I think this situation can only be fixed by moving up into languages that compile to vendor specific GPU languages. Just treat CUDA, openCL, vulkan compute, metal compute(??) etc. as the assembly of graphics cards.

T-A 5y ago
https://www.oneapi.io/ https://www.oneapi.io/
- snicker7 5y ago
  Currently only supported by Intel.
  - T-A 5y ago
    https://www.codeplay.com/solutions/oneapi/for-cuda/ https://www.codeplay.com/solutions/oneapi/for-cuda/ https://www.urz.uni-heidelberg.de/en/newsroom/oneapi-academic-center-of-excellence-established-at-the-heidelberg-university-computing-centre-urz https://www.urz.uni-heidelberg.de/en/newsroom/oneapi-academi...
hobofan 5y ago
Barely anyone is writing CUDA directly these days. Just add support in PyTorch and Tensorflow and you've covered probably 90% of the deep learning market.
- hprotagonist 5y ago
  and ONNX.
pjmlp 5y ago
That is just part of the story. CUDA wiped out OpenCL, because it went polyglot as of version 3.0, while insisting that everyone should write in a C dialect. They also provide great graphical debugging tools and libraries. Khronos waited too long to introduce SPIR, and in traditional Khronos fashion, waited for the partners to provide the tooling. One could blame NVidia, but it isn't as the competition has done a better job.

rektide 5y ago

ROCm seems to be tolerably decent, if you are willing to spend a couple hours, and, big if, if HIP supports all the various libraries you were relying on. CUDA has a huge support library, and ROCm has been missing not just the small fry stuff but a lot of the core stuff in the that library. Long term, AI (& a lot of other interests) need to serve themselves. CUDA is excellently convenient, but long term I have a hard time imagining there being a worthwhile future for anything but Vulkan. There don't seem to be a lot of forays into writing good all-encompassing libraries in Vulkan yet, nor many more specialized AI/ML Vulkan libraries, so it feels largely like we more or less haven't started really trying.

dnautics 5y ago
is there any indication that ROCm has solved its stability issues? I wasn't doing the testing myself, but the reason why we rejected ROCm a while back (2 years?) was because you could get segfaults hours into a ML training run, which is... frustrating, to say the least, and not easily identifiable in quickie test runs (or CI, if ML did more CI).
rektide 5y ago
Lot of downvotes. Anyone have any opinion? Is CUDA fine forever? Is there something other than Vulkan we should also try? Do you think AMD should solve every problem CUDA solves for their customers too? What gives here? I see a lot a lot a lot of resistance to the idea that we should start trying to align to Vulkan. Here & elsewhere. I don't get it, it makes no sense, & everyone else using GPU's is running fast as they can towards Vulkan. Is it just too soon too early in the adoption curve, or do ya'll think there are more serious obstructions long term to building a more Vulkan centric AI/ML toolkit? It still feels inevitable to me. What we are doing now feels like a waste of time. I wish ya'll wouldn't downvote so casually, wouldn't just try to brush this viewpoint away.
- BadInformatics 5y ago
  > Do you think AMD should solve every problem CUDA solves for their customers too? They had no choice. Getting a bunch of HPC people to completely rewrite their code for a different API is a tough pill to swallow when you're trying to win supercomputer contracts. Would they have preferred to spend development resources elsewhere? Probably, they've even got their own standards and SDKs from days past. > everyone else using GPU's is running fast as they can towards Vulkan I'm not qualified to comment on the entirety of it, but I can say that basically no claim in this statement is true: 1. Not everyone doing compute is using GPUs. Companies are increasingly designing and releasing their own custom hardware (TPUs, IPUs, NPUs, etc.) 2. Not everyone using GPUs is cares about Vulkan. Certainly many folks doing graphics stuff don't, and DirectX is as healthy as ever. There have been bits and pieces of work around Vulkan compute for mobile ML model deployment, but it's a tiny niche and doesn't involve discrete GPUs at all. > Is it just too soon too early in the adoption curve Yes. Vulkan compute is still missing many of the niceties of more developed compute APIs. Tooling is one big part of that: writing shaders using GLSL is a pretty big step down from using whatever language you were using before (C++, Fortran, Python, etc). > do ya'll think there are more serious obstructions long term to building a more Vulkan centric AI/ML toolkit You could probably write a whole page about this, but TL;DR yes. It would take at least as much effort as AMD and Intel put into their respective compute stacks to get Vulkan ML anywhere near ready for prime time. You need to have inference, training, cross-device communication, headless GPU usage, reasonably wide compatibility, not garbage performance, framework integration, passable tooling and more. Sure these are all feasible, but who has the incentive to put in the time to do it? The big 3 vendors have their supercomputer contracts already, so all they need to do is keep maintaining their 1st-party compute stacks. Interop also requires going through Khronos, which is its own political quagmire when it comes to standardization. Nvidia already managed to obstruct OpenCL into obscurity, why would they do anything different here? Downstream libraries have also poured untold millions into existing compute stacks, OR rely on the vendors to implement that functionality for them. This is before we even get into custom hardware like TPUs that don't behave like a GPU at all. So in short, there is little inevitable about this at all. The reason people may have been frustrated by your comment is because Vulkan compute comes up all the time as some silver bullet that will save us from the walled gardens of CUDA and co (especially for ML, arguably the most complex and expensive subdomain of them all). We'd all like it to come true, but until all of the aforementioned points are addressed this will remain primarily in pipe dream territory.

lvl100 5y ago

I agree 100% and if Nvidia’s recent showing and puzzling focus on “Omniverse” is any indication, they’re operating in a fantasy world a bit.

pjmlp 5y ago

You know how to break it? With modern tooling. Instead of forcing devs to live in the pre-historic days of C dialects and printf debugging, provide polyglot IDEs with graphical debugging tools capable of single step GPU shaders and a rich libraries ecosystem. Khronos got the message too late and now no one cares.