8 ms·

They have a massive lead due to vendor locking any software written to depend on CUDA instead of an open standard... and that is a good thing? Okeydokey.

snovv_crash 4y ago
CUDA is easy to use and the open standard has an extremely high barrier to entry. This is what enabled Nvidia to even lock people in in the first place - their tech was just so much better.
- sennight 4y ago
  That "barrier to entry" line works for things that saturate broad markets... and that is definitely not the case with GPGPU. So when you try to use that line of thinking, given the incredibly well funded and hyper niche use cases it sees, it sounds as if you're saying that opencl is too hard for those dummies writing nuclear weapon simulators at Oak Ridge National Laboratory. And before anybody swings in on a chandelier with "but the scientists are science-ing, they can't be expected to learn to code - something something python!": take a look at the software and documentation those labs make publicly available - they are definitely expected to be comfortable with things weirder than opencl.
  - oytis 4y ago
    Maybe they can work with complex libraries, but if there is a better one available I would totally understand that they prefer it. You need to be in a certain software bubble not to understand how to work with OpenCL, but to care enough about whether something is an open standard, whether something is open source etc.
    - sennight 4y ago
      > You need to be in a certain software bubble... Or have a fundamental understanding of the way mainframes have been built since forever... massive job schedulers feeding various ASICs purpose built for different tasks. IBM is really something else when it comes to this sort of thing, the left hand really doesn't know what the right hand is doing over there. Summit at ORNL... a homogeneous supercomputer cluster made of power9 machines effectively acting as pci backplanes for GPGPUs. You'd think they'd know better... the choice for the ISA made sense given the absolute gulf between them and x86 I/O at the time, but to then not take full advantage of their CPUs by going with CUDA... wow. Oh well, this is the same company that fully opened their CPU - and then immediately announce how their next CPU was going to depend on binary blobbed memory controllers... aaand they also sold the IP for the controllers to some scumbag softcore silicon IP outfit. So despite all their talk with regard to open source, no - they don't seem to actually understand how to fully take advantage of it.
      - snovv_crash 4y ago
        Mainframes died for a reason, and that reason wasn't the capabilities of their specialized hardware blocks, but a. the difficulty of using them in unintended ways, such as wiring together a bunch of them to get better performance b. the specialized knowledge required to get them to do anything at all c. the limited use cases of the hardware (see a.) which made it expensive Applauding OpenCL because it is more like mainframes than CUDA... seems nonsensical to me.
  - snovv_crash 4y ago
    If you have a hard task to accomplish, and there's a way of making it substantially easier, the smart engineer is, frankly, going to take the easier option. They're only going to go with the hard one if that's the only hardware they have access to. Back in university we had to specify to the IT department that we needed Nvidia GPUs because people had done all sorts of cool things with CUDA that we could build on, and if we'd had to write it on AMD GPUs back in 2013 we would have burnt through all of our time just getting the frameworks compiling.
    - jjoonathan 4y ago
      Yep. Source: burnt through all my time just getting frameworks compiling on AMD GPUs back in 2014.
dotnet00 4y ago
Developers weren't forced into using CUDA, that was entirely because of their ecosystem being much better than anyone else's. Facebook and Google obviously wouldn't want to lock themselves into CUDA for PyTorch and Tensorflow, but there genuinely wasn't any other realistic option. OpenCL existed but the implementation on AMD was just as bad as the one on NVIDIA. Consider that Blender's Cycles render engine had only gotten OpenCL support when AMD assigned some devs specifically to help work through driver bugs and even then they had to resort to a 'split kernel' hack, which recently led to OpenCL support once again being entirely dropped as the situation hadn't really improved over the decade. Instead the CUDA version was ported to HIP and a HIP supporting Windows driver was released. Even now, if you need to do GPGPU on PCs, CUDA is essentially the easiest option. Every NVIDIA card supports it pretty much right from launch on both Linux and Windows, while with AMD you currently only get support for a few distros (Windows support is probably not too far off now), slow support for new hardware and a system of phasing out support for older cards even if they're still very common. On top of that, NVIDIA offers amazing profiling and debugging tools that the competition hasn't caught up to.
- sennight 4y ago
  > Developers weren't forced into using CUDA... ... no, just as any other consumer isn't necessarily "forced" by companies employing anticompetitive practices. > Facebook and Google... lol, they have such a high churn rate on hardware that I seriously doubt they'd give it much thought at all. Their use case is unique to a tiny number of companies - high churn, low capital constraint, no tolerance for supplier delay. In such a scenario CUDA vendor lock in wouldn't even register as a potential point of pain. > OpenCL existed but the implementation on AMD was just as bad as the one on NVIDIA. For those unaware of how opencl works: an API is provided by the standard, to which software can be written by people - even those without signed NDAs. The API can interface to a hardware endpoint that has further open code and generous documentation... like an open source DSP, CPU, etc - or it can hit an opaque pointer. If your hardware vendor is absurdly secretive and insists on treating microcode and binary blobs as competitive advantages, then your opencl experience is wholly dependent on that vendor implementation. Unfortunately for GPUs that means either NVIDIA or AMD (maybe Intel, we'll see)... so yeah - not good. AMD has improved things open sourcing a great deal of their code, but that is a relatively recent development. While I'm familiar with some aspects of their codebase (had to fix an endian bug, guess what ISA I use), I dunno how much GPGPU functionality they're still hiding behind their encrypted firmware binary blobs. Also, to the point on NVIDIA's opencl sucking: anybody else remember that time that Intel intentionally crippled performance for non-Intel hardware running code generated by their compiler or linked to their high performance scientific libraries? Surely NVIDIA would never sandbag opencl... Anyway, this is kind of a goofy thing to even discuss given two facts: * There are basically two GPU vendors - so vendor lock is practically assured already. * CUDA is designed to run parallel code on NVIDIA GPUs - full stop. Opencl is designed for heterogeneous computing, and GPUs are just one of many computing units possible. So not apples to apples.
jacquesm 4y ago
One such developer: I love CUDA, even if I don't like Nvidia. CUDA is the most direct and transparent way to work with the GPU for the stuff I do and it is on account of it not being an open standard: it doesn't have four vendors trying to pull it their way ending somewhere in the middle and it has been very stable for a long time so I don't need to update my code every time I get new hardware, though sometimes some tweaks are required to get it close to theoretical maximum speed. That alone stops me from going with an 'open standard' even though I'm a huge fan of those. But in this case the hardware is so tightly coupled to the software I see no point: there isn't anything out there that would tempt me. So, locked in? Yes. But voluntarily so, I could switch if I wanted to but I see absolutely no incentive, performance wise or software architecture wise. And the way things are today that is unlikely to change unless some part is willing to invest a massive amount of money into incentivizing people to switch. And I'm not talking about miners here but people that do useful computations and modeling on their hardware, and all this on Linux to boot, a platform that most vendors could not care less about.
- einpoklum 4y ago
  > CUDA is the most direct and transparent way to work with the GPU Yes, but it's still not direct and transparent enough. The libraries and drivers are closed. > it doesn't have four vendors trying to pull it their way ending somewhere in the middle Well, no, but it does have "marketing-ware", i.e. features introduced mostly to be able to say: "Oh, we have feature X" - even if the feature does not help performance.
  - TomVDB 4y ago
    I don’t know which features you’re talking about, but over the years, CUDA has received quite a bit of features where Nvidia was quite explicit that they were not for performance, but for ease of use. “If you want code to work with 90% performance, use this, if you want 100%, use the old way, but with significantly more developer pain.” Seems fair to me.
  - dahart 4y ago
    > The libraries and drivers are closed. Are you including the new open source kernel modules? https://developer.nvidia.com/blog/nvidia-releases-open-source-gpu-kernel-modules/ https://developer.nvidia.com/blog/nvidia-releases-open-sourc... > it’s still not direct and transparent enough. Out of curiosity, not direct enough for what? What do you need access to that you don’t have at the moment? > features introduced mostly to be able to o say: “Oh we have feature X” - even if the feature does not help performance. Which features are you referring to? Are you suggesting that features that make programming easier and features that users request must not be added? Does your opinion extend to all computing platforms and all vendors equally? Do you have any examples of a widely used platform/language/compiler/hardware that has no features outside of performance?
pjmlp 4y ago
Competition has had 15 years to provide competitive graphical development experience, GPGPU libraries and polygon development experience. They don't have anyone to blame but themselves.
- sennight 4y ago
  Uh, and everyone else who isn't NVIDIA or the "Competition" aka AMD? Do we get the good old "Don't like the way Private Company does X, build your own X then!"? Everyone had years to provide a competitive petroleum company - apparently Standard Oil did nothing wrong after all.
  - pjmlp 4y ago
    Definitely, they are the ones to blame by not upping their game, this includes Intel as well. They have had 15 years to improve OpenCL tooling to outmatch CUDA, they haven't done so.
    - sennight 4y ago
      Lazy end users, not writing drivers to hardware they don't have engineering specs to. Oh well, NVIDIA might get more than they bargained for with their scheme to dominate GPGPU with their proprietary API. It would be a real shame if people started using GPUs for rendering graphics, and just computed the prior work loads on DSPs and FPGAs sitting behind APIs already wired into LLVM, and silicon fab at process levels suited to the task only gets cheaper as new facilities are created to meet the demands of the next node level. That would be just awful, CUDA being so great and all - so beloved due to "Investments ... made 15-17 years ago" and no other reason. Huh, I wonder if that is why they unsuccessfully tried to buy Arm - because they knew the GPU carriage is at risk of turning back into a pumpkin, and they want to continue dominating the market of uncanny cat image generators.
      - pjmlp 4y ago
        Playing victim usually doesn't work. Again, AMD and Intel could have done their job to appeal to developers, they fully failed at it . ARM you say? Ironically Google never supported OpenCL on mobile, rather pushed their RenderScript dialect that no one cares, and now is pushing for Vulkan Compute, even worse than OpenCL in regards to tooling.
physicsguy 4y ago
It's not vendor locking when the functionality doesn't exist on the other platforms. Just take as an example cuFFT. That's a core library that's been there pretty much since the beginning with CUDA. It has a compatibility interface compatible with FFTW, which everybody knows how to use. So porting from a CPU code that used FFTW was trivial. rocFFT is not as mature, the documentation is poor, and the performance is worse. And that's where there is an equivalent library that exists. In other cases, there isn't one.

snovv_crash 4y ago

CUDA is easy to use and the open standard has an extremely high barrier to entry. This is what enabled Nvidia to even lock people in in the first place - their tech was just so much better.

sennight 4y ago
That "barrier to entry" line works for things that saturate broad markets... and that is definitely not the case with GPGPU. So when you try to use that line of thinking, given the incredibly well funded and hyper niche use cases it sees, it sounds as if you're saying that opencl is too hard for those dummies writing nuclear weapon simulators at Oak Ridge National Laboratory. And before anybody swings in on a chandelier with "but the scientists are science-ing, they can't be expected to learn to code - something something python!": take a look at the software and documentation those labs make publicly available - they are definitely expected to be comfortable with things weirder than opencl.
- oytis 4y ago
  Maybe they can work with complex libraries, but if there is a better one available I would totally understand that they prefer it. You need to be in a certain software bubble not to understand how to work with OpenCL, but to care enough about whether something is an open standard, whether something is open source etc.
  - sennight 4y ago
    > You need to be in a certain software bubble... Or have a fundamental understanding of the way mainframes have been built since forever... massive job schedulers feeding various ASICs purpose built for different tasks. IBM is really something else when it comes to this sort of thing, the left hand really doesn't know what the right hand is doing over there. Summit at ORNL... a homogeneous supercomputer cluster made of power9 machines effectively acting as pci backplanes for GPGPUs. You'd think they'd know better... the choice for the ISA made sense given the absolute gulf between them and x86 I/O at the time, but to then not take full advantage of their CPUs by going with CUDA... wow. Oh well, this is the same company that fully opened their CPU - and then immediately announce how their next CPU was going to depend on binary blobbed memory controllers... aaand they also sold the IP for the controllers to some scumbag softcore silicon IP outfit. So despite all their talk with regard to open source, no - they don't seem to actually understand how to fully take advantage of it.
    - snovv_crash 4y ago
      Mainframes died for a reason, and that reason wasn't the capabilities of their specialized hardware blocks, but a. the difficulty of using them in unintended ways, such as wiring together a bunch of them to get better performance b. the specialized knowledge required to get them to do anything at all c. the limited use cases of the hardware (see a.) which made it expensive Applauding OpenCL because it is more like mainframes than CUDA... seems nonsensical to me.
- snovv_crash 4y ago
  If you have a hard task to accomplish, and there's a way of making it substantially easier, the smart engineer is, frankly, going to take the easier option. They're only going to go with the hard one if that's the only hardware they have access to. Back in university we had to specify to the IT department that we needed Nvidia GPUs because people had done all sorts of cool things with CUDA that we could build on, and if we'd had to write it on AMD GPUs back in 2013 we would have burnt through all of our time just getting the frameworks compiling.
  - jjoonathan 4y ago
    Yep. Source: burnt through all my time just getting frameworks compiling on AMD GPUs back in 2014.

dotnet00 4y ago

Developers weren't forced into using CUDA, that was entirely because of their ecosystem being much better than anyone else's. Facebook and Google obviously wouldn't want to lock themselves into CUDA for PyTorch and Tensorflow, but there genuinely wasn't any other realistic option. OpenCL existed but the implementation on AMD was just as bad as the one on NVIDIA. Consider that Blender's Cycles render engine had only gotten OpenCL support when AMD assigned some devs specifically to help work through driver bugs and even then they had to resort to a 'split kernel' hack, which recently led to OpenCL support once again being entirely dropped as the situation hadn't really improved over the decade. Instead the CUDA version was ported to HIP and a HIP supporting Windows driver was released. Even now, if you need to do GPGPU on PCs, CUDA is essentially the easiest option. Every NVIDIA card supports it pretty much right from launch on both Linux and Windows, while with AMD you currently only get support for a few distros (Windows support is probably not too far off now), slow support for new hardware and a system of phasing out support for older cards even if they're still very common. On top of that, NVIDIA offers amazing profiling and debugging tools that the competition hasn't caught up to.

sennight 4y ago
> Developers weren't forced into using CUDA... ... no, just as any other consumer isn't necessarily "forced" by companies employing anticompetitive practices. > Facebook and Google... lol, they have such a high churn rate on hardware that I seriously doubt they'd give it much thought at all. Their use case is unique to a tiny number of companies - high churn, low capital constraint, no tolerance for supplier delay. In such a scenario CUDA vendor lock in wouldn't even register as a potential point of pain. > OpenCL existed but the implementation on AMD was just as bad as the one on NVIDIA. For those unaware of how opencl works: an API is provided by the standard, to which software can be written by people - even those without signed NDAs. The API can interface to a hardware endpoint that has further open code and generous documentation... like an open source DSP, CPU, etc - or it can hit an opaque pointer. If your hardware vendor is absurdly secretive and insists on treating microcode and binary blobs as competitive advantages, then your opencl experience is wholly dependent on that vendor implementation. Unfortunately for GPUs that means either NVIDIA or AMD (maybe Intel, we'll see)... so yeah - not good. AMD has improved things open sourcing a great deal of their code, but that is a relatively recent development. While I'm familiar with some aspects of their codebase (had to fix an endian bug, guess what ISA I use), I dunno how much GPGPU functionality they're still hiding behind their encrypted firmware binary blobs. Also, to the point on NVIDIA's opencl sucking: anybody else remember that time that Intel intentionally crippled performance for non-Intel hardware running code generated by their compiler or linked to their high performance scientific libraries? Surely NVIDIA would never sandbag opencl... Anyway, this is kind of a goofy thing to even discuss given two facts: * There are basically two GPU vendors - so vendor lock is practically assured already. * CUDA is designed to run parallel code on NVIDIA GPUs - full stop. Opencl is designed for heterogeneous computing, and GPUs are just one of many computing units possible. So not apples to apples.

jacquesm 4y ago

One such developer: I love CUDA, even if I don't like Nvidia. CUDA is the most direct and transparent way to work with the GPU for the stuff I do and it is on account of it not being an open standard: it doesn't have four vendors trying to pull it their way ending somewhere in the middle and it has been very stable for a long time so I don't need to update my code every time I get new hardware, though sometimes some tweaks are required to get it close to theoretical maximum speed. That alone stops me from going with an 'open standard' even though I'm a huge fan of those. But in this case the hardware is so tightly coupled to the software I see no point: there isn't anything out there that would tempt me. So, locked in? Yes. But voluntarily so, I could switch if I wanted to but I see absolutely no incentive, performance wise or software architecture wise. And the way things are today that is unlikely to change unless some part is willing to invest a massive amount of money into incentivizing people to switch. And I'm not talking about miners here but people that do useful computations and modeling on their hardware, and all this on Linux to boot, a platform that most vendors could not care less about.

einpoklum 4y ago
> CUDA is the most direct and transparent way to work with the GPU Yes, but it's still not direct and transparent enough. The libraries and drivers are closed. > it doesn't have four vendors trying to pull it their way ending somewhere in the middle Well, no, but it does have "marketing-ware", i.e. features introduced mostly to be able to say: "Oh, we have feature X" - even if the feature does not help performance.
- TomVDB 4y ago
  I don’t know which features you’re talking about, but over the years, CUDA has received quite a bit of features where Nvidia was quite explicit that they were not for performance, but for ease of use. “If you want code to work with 90% performance, use this, if you want 100%, use the old way, but with significantly more developer pain.” Seems fair to me.
- dahart 4y ago
  > The libraries and drivers are closed. Are you including the new open source kernel modules? https://developer.nvidia.com/blog/nvidia-releases-open-source-gpu-kernel-modules/ https://developer.nvidia.com/blog/nvidia-releases-open-sourc... > it’s still not direct and transparent enough. Out of curiosity, not direct enough for what? What do you need access to that you don’t have at the moment? > features introduced mostly to be able to o say: “Oh we have feature X” - even if the feature does not help performance. Which features are you referring to? Are you suggesting that features that make programming easier and features that users request must not be added? Does your opinion extend to all computing platforms and all vendors equally? Do you have any examples of a widely used platform/language/compiler/hardware that has no features outside of performance?

pjmlp 4y ago

Competition has had 15 years to provide competitive graphical development experience, GPGPU libraries and polygon development experience. They don't have anyone to blame but themselves.

sennight 4y ago
Uh, and everyone else who isn't NVIDIA or the "Competition" aka AMD? Do we get the good old "Don't like the way Private Company does X, build your own X then!"? Everyone had years to provide a competitive petroleum company - apparently Standard Oil did nothing wrong after all.
- pjmlp 4y ago
  Definitely, they are the ones to blame by not upping their game, this includes Intel as well. They have had 15 years to improve OpenCL tooling to outmatch CUDA, they haven't done so.
  - sennight 4y ago
    Lazy end users, not writing drivers to hardware they don't have engineering specs to. Oh well, NVIDIA might get more than they bargained for with their scheme to dominate GPGPU with their proprietary API. It would be a real shame if people started using GPUs for rendering graphics, and just computed the prior work loads on DSPs and FPGAs sitting behind APIs already wired into LLVM, and silicon fab at process levels suited to the task only gets cheaper as new facilities are created to meet the demands of the next node level. That would be just awful, CUDA being so great and all - so beloved due to "Investments ... made 15-17 years ago" and no other reason. Huh, I wonder if that is why they unsuccessfully tried to buy Arm - because they knew the GPU carriage is at risk of turning back into a pumpkin, and they want to continue dominating the market of uncanny cat image generators.
    - pjmlp 4y ago
      Playing victim usually doesn't work. Again, AMD and Intel could have done their job to appeal to developers, they fully failed at it . ARM you say? Ironically Google never supported OpenCL on mobile, rather pushed their RenderScript dialect that no one cares, and now is pushing for Vulkan Compute, even worse than OpenCL in regards to tooling.

physicsguy 4y ago

It's not vendor locking when the functionality doesn't exist on the other platforms. Just take as an example cuFFT. That's a core library that's been there pretty much since the beginning with CUDA. It has a compatibility interface compatible with FFTW, which everybody knows how to use. So porting from a CPU code that used FFTW was trivial. rocFFT is not as mature, the documentation is poor, and the performance is worse. And that's where there is an equivalent library that exists. In other cases, there isn't one.