4 ms·

1. Driver stability 2. Works on more consumer grade cards 3. Ecosystem advantage (lots of software developed against an existing and well supported ecosystem)

1. Driver stability 2. Works on more consumer grade cards 3. Ecosystem advantage (lots of software developed against an existing and well supported ecosystem) I have a laptop with a mobile 2060 and a desktop with a top-of-the-line consumer 7900XTX. As of yet, the 7900XTX isn't officially supported (and I haven't bothered to go down the obnoxious rabbit hole to figure out how to compute on it). Meanwhile, I can load up CUDA.jl on my laptop in mere minutes with absolutely no fuss. Edit: if there are any GPU gurus out there who are capable of working on AMDGPU.jl to make it work on cards like the 7900XTX out of the box and writing documentation/tutorials for it... start a Patreon. I bet you could fund some significant effort getting that up and running!

officialchicken 3y ago
As of today, there is zero consumer card support from AMD. It is an option only if you have a PRO card. "Formal support for RDNA 3-based GPUs on Linux is planned to begin rolling out this fall, starting with the 48GB Radeon PRO W7900 and the 24GB Radeon RX 7900 XTX, with additional cards and expanded capabilities to be released over time." [0] [0] https://community.amd.com/t5/rocm/new-rocm-5-6-release-brings-enhancements-and-optimizations-for/ba-p/614745 https://community.amd.com/t5/rocm/new-rocm-5-6-release-bring...
- empyrrhicist 3y ago
  Right, which SUCKS. Everyone who wants to prototype on their existing gear before jumping into a big pro card purchase is stuck with Nvidia, and the availability/performance of the software stack shows it.
  - mrguyorama 3y ago
    Or even just have some hands on time to get familiar with the flow, to dick around, to build skills, to teach etc, but you CANNOT DO THAT with AMD
    - empyrrhicist 3y ago
      Exactly
andy_ppp 3y ago
I’m asking at a lower level than this, CUDA presumably has a list of functionality for GPGPU stuff like tensors, loading data, splitting up training, and building pipelines of networks /attention stuff that can efficiently fit neural networks to many sorts of data. Why is it so difficult for other manufacturers to provide a compatible layer? If Apple can make Direct X 12 work on Apple Silicon surely AMD should be able to make CUDA (which has to be much simpler that DX12) work on their graphics cards? Is there some fundamental architectural differences that stop this from working?
- empyrrhicist 3y ago
  There might be intellectual property concerns with "directly" implementing CUDA, and the architectures are (as I understand it) a bit different. That doesn't explain why they don't support something with similar broad compatibility though, as the actual card capabilities are very similar.
- hedgehog 3y ago
  There's nothing conceptually hard but it's really a lot of work. In addition to the items you listed there's the actual compute kernels or compiler to generate those, and then porting frameworks over (PyTorch etc), and then doing the level of testing, documentation, and ongoing maintenance to make an alternative platform a reasonable idea for end users. The pitch for buying NVIDIA hardware is that existing tools, example code, and third party research will more or less work and perform well out of the box. Edit: Going back to your original question, the main thing that makes CUDA so special is NVIDIA has already poured billions of dollars into all of this infrastructure and credibly will keep doing so.
- mrguyorama 3y ago
  Sure, AMD could write a CUDA emulator (if it was legal) for AMD GPUs, but if it's one tenth the performance, whats the point?
  - empyrrhicist 3y ago
    There's no real reason it would need to be 1/10 the performance though, depending on the kernel.
adfiognquio3 3y ago
Nvidia's software is also pretty atrocious, in my opinion. The output of various tools is cryptic, updates regularly result in a totally broken system, and things often stop working for no discernible reason. Nvidia GPUs are always the most finnicky part of a system. A modern Linux system should have uptime measured in years with minimal effort. A modern Linux system with Nvidia GPUs will have uptime of weeks with a lot of fuss. (I'm no expert, just someone who's managed a number of PCs and a few servers.)
- empyrrhicist 3y ago
  Right, but they can get away with that because they have essentially no competition. With that said, Pop!OS does a really nice job of handling the Nvidia software stack - I've been running it on the laptop mentioned above for several years with no issues (though I don't leave my machines on 24/7).

officialchicken 3y ago

As of today, there is zero consumer card support from AMD. It is an option only if you have a PRO card. "Formal support for RDNA 3-based GPUs on Linux is planned to begin rolling out this fall, starting with the 48GB Radeon PRO W7900 and the 24GB Radeon RX 7900 XTX, with additional cards and expanded capabilities to be released over time." [0] [0] https://community.amd.com/t5/rocm/new-rocm-5-6-release-brings-enhancements-and-optimizations-for/ba-p/614745 https://community.amd.com/t5/rocm/new-rocm-5-6-release-bring...

empyrrhicist 3y ago
Right, which SUCKS. Everyone who wants to prototype on their existing gear before jumping into a big pro card purchase is stuck with Nvidia, and the availability/performance of the software stack shows it.
- mrguyorama 3y ago
  Or even just have some hands on time to get familiar with the flow, to dick around, to build skills, to teach etc, but you CANNOT DO THAT with AMD
  - empyrrhicist 3y ago
    Exactly

andy_ppp 3y ago

I’m asking at a lower level than this, CUDA presumably has a list of functionality for GPGPU stuff like tensors, loading data, splitting up training, and building pipelines of networks /attention stuff that can efficiently fit neural networks to many sorts of data. Why is it so difficult for other manufacturers to provide a compatible layer? If Apple can make Direct X 12 work on Apple Silicon surely AMD should be able to make CUDA (which has to be much simpler that DX12) work on their graphics cards? Is there some fundamental architectural differences that stop this from working?

empyrrhicist 3y ago
There might be intellectual property concerns with "directly" implementing CUDA, and the architectures are (as I understand it) a bit different. That doesn't explain why they don't support something with similar broad compatibility though, as the actual card capabilities are very similar.
hedgehog 3y ago
There's nothing conceptually hard but it's really a lot of work. In addition to the items you listed there's the actual compute kernels or compiler to generate those, and then porting frameworks over (PyTorch etc), and then doing the level of testing, documentation, and ongoing maintenance to make an alternative platform a reasonable idea for end users. The pitch for buying NVIDIA hardware is that existing tools, example code, and third party research will more or less work and perform well out of the box. Edit: Going back to your original question, the main thing that makes CUDA so special is NVIDIA has already poured billions of dollars into all of this infrastructure and credibly will keep doing so.
mrguyorama 3y ago
Sure, AMD could write a CUDA emulator (if it was legal) for AMD GPUs, but if it's one tenth the performance, whats the point?
- empyrrhicist 3y ago
  There's no real reason it would need to be 1/10 the performance though, depending on the kernel.

adfiognquio3 3y ago

Nvidia's software is also pretty atrocious, in my opinion. The output of various tools is cryptic, updates regularly result in a totally broken system, and things often stop working for no discernible reason. Nvidia GPUs are always the most finnicky part of a system. A modern Linux system should have uptime measured in years with minimal effort. A modern Linux system with Nvidia GPUs will have uptime of weeks with a lot of fuss. (I'm no expert, just someone who's managed a number of PCs and a few servers.)

empyrrhicist 3y ago
Right, but they can get away with that because they have essentially no competition. With that said, Pop!OS does a really nice job of handling the Nvidia software stack - I've been running it on the laptop mentioned above for several years with no issues (though I don't leave my machines on 24/7).