5 ms·

Interesting question. Press article aside, GPGPU applications like scientific compute, ML etc. have all mostly gravitated to Nvidia / CUDA. Not working in this

Interesting question. Press article aside, GPGPU applications like scientific compute, ML etc. have all mostly gravitated to Nvidia / CUDA. Not working in this space, I'm curious why this is the case. Is there something inherently better about CUDA? Or is it that Nvidia's performance is somehow better for these tasks? Or maybe something else?

naavis 5y ago
NVIDIA provides tools that just mostly do not exist for other GPUs, making it easier to build on CUDA instead of something else.
- hiptobecubic 5y ago
  Absolutely this. When cuda was first making headway it was the only thing even remotely close to a "developer environment" and made things significantly easier than any of the alternatives. It might be different now, but at that time, many of the users were not computer scientists, they were scientists with computers. Having an easier to use programming model and decent debugging tools means publishing more results, more quickly.
carlmr 5y ago
From my cursory knowledge of the topic, there are competitors like ROCm, but CUDA was the first that had a useable solution here. Also last time I checked ROCm doesn't have broad support on consumer cards, which makes it harder for people to try it out at home. But it seems ROCm is getting better and it has tensorflow and pytorch support, so there's reasons to be hopeful to see some competition here.
jjoonathan 5y ago
Back in 2014 or so I made the unfortunate mistake of buying AMD cards with the thought that I'd just use OpenCL. I knew that some codes wouldn't run, but I had catalogued the ones I really cared about and thought I was up for the challenge. I was so, so wrong. First of all, software that advertised OpenCL or AMD compatibility often worked poorly or not at all in that mode. Adobe creative suite just rendered solid black outputs whenever acceleration was enabled and forums revealed that it had been that way for years and nobody cared to fix it. Blender supported OpenCL for a while, but it was slower than CPU rendering and for a sticky reason (nvidia did the work to support big kernels with heavy branching and AMD didn't). Ironically, OpenCL mode had decent performance but only if you used it on an nvidia card. The situation was even worse in scientific codes, where "OpenCL version" typically meant "a half-finished blob of code that was abandoned before ever becoming functional, let alone anywhere near feature-parity." I quickly learned why this was the case: the OpenCL tooling and drivers weren't just a little behind their CUDA counterparts in terms of features, they were almost unusably bad. For instance, the OpenCL drivers didn't do memory (or other context object?) cleanup, so if your program was less than perfect you would be headed for a hard crash every few runs. Debugging never worked -- hard crashes all around. Basic examples didn't compile, documentation was scattered, and at the end of the day, it was also leagues behind CUDA in terms of features. After months of putting up with this, I finally bit the bullet, sold my AMD card, bought an NVidia card, ate the spread, the shipping, the eBay fees, and the green tax itself. It hurt, but it meant I was able to start shipping code. I'm a stubborn bastard so I didn't learn my lesson and repeated this process two years later on the next generation of cards. The second time, the lesson stuck.
mattkrause 5y ago
The products are good, but NVidia also cleverly bootstrapped a whole ecosystem around them. One of the other posts mentions 2014 as a turning point. At that time, GPGPU stuff was entering the (scientific) mainstream and NVidia was all over academia, convincing people to try it out in their research. They handed out demo accounts on boxes with beefy GPUs, and ran an extremely generous hardware grant proposal. There was tons of (free) training available: online CUDA MOOCs and in-person training sessions. The first-party tools were pretty decent. As a result, people built a lot of stuff using CUDA. Others, wanting to use those programs, basically had to buy NVidia. Lather, rinse, repeat. This is in stark contrast to the other “accelerator” vendors. Around the same time, I looked into using Intel Xenon Phi and they were way less aggressive: “here are some benchmarks, here’s how it works, your institution has one or two somewhere if you want to contact them and try it out.” As for OpenCL…crickets. I don’t even remember any AMD events, and the very broad standard made it hard to figure out what would work/work well and you might end up needing to port it too!
- jjoonathan 5y ago
  AMD's GPU OpenCL wasn't just not marketed, it was also a bad product, even for relatively tame scientific purposes, even when AMD made loud, repeated statements to the contrary. Hopefully now that AMD has money they can do better. I'm sure that NVidia's ecosystem building played a role (I remember events in 2009 and books before that), perhaps even a big role, but it wasn't the only factor. I paid a steep price in 2014 and 2016 for incorrectly assuming that it was.

naavis 5y ago

NVIDIA provides tools that just mostly do not exist for other GPUs, making it easier to build on CUDA instead of something else.

hiptobecubic 5y ago
Absolutely this. When cuda was first making headway it was the only thing even remotely close to a "developer environment" and made things significantly easier than any of the alternatives. It might be different now, but at that time, many of the users were not computer scientists, they were scientists with computers. Having an easier to use programming model and decent debugging tools means publishing more results, more quickly.

carlmr 5y ago

From my cursory knowledge of the topic, there are competitors like ROCm, but CUDA was the first that had a useable solution here. Also last time I checked ROCm doesn't have broad support on consumer cards, which makes it harder for people to try it out at home. But it seems ROCm is getting better and it has tensorflow and pytorch support, so there's reasons to be hopeful to see some competition here.

jjoonathan 5y ago

Back in 2014 or so I made the unfortunate mistake of buying AMD cards with the thought that I'd just use OpenCL. I knew that some codes wouldn't run, but I had catalogued the ones I really cared about and thought I was up for the challenge. I was so, so wrong. First of all, software that advertised OpenCL or AMD compatibility often worked poorly or not at all in that mode. Adobe creative suite just rendered solid black outputs whenever acceleration was enabled and forums revealed that it had been that way for years and nobody cared to fix it. Blender supported OpenCL for a while, but it was slower than CPU rendering and for a sticky reason (nvidia did the work to support big kernels with heavy branching and AMD didn't). Ironically, OpenCL mode had decent performance but only if you used it on an nvidia card. The situation was even worse in scientific codes, where "OpenCL version" typically meant "a half-finished blob of code that was abandoned before ever becoming functional, let alone anywhere near feature-parity." I quickly learned why this was the case: the OpenCL tooling and drivers weren't just a little behind their CUDA counterparts in terms of features, they were almost unusably bad. For instance, the OpenCL drivers didn't do memory (or other context object?) cleanup, so if your program was less than perfect you would be headed for a hard crash every few runs. Debugging never worked -- hard crashes all around. Basic examples didn't compile, documentation was scattered, and at the end of the day, it was also leagues behind CUDA in terms of features. After months of putting up with this, I finally bit the bullet, sold my AMD card, bought an NVidia card, ate the spread, the shipping, the eBay fees, and the green tax itself. It hurt, but it meant I was able to start shipping code. I'm a stubborn bastard so I didn't learn my lesson and repeated this process two years later on the next generation of cards. The second time, the lesson stuck.

mattkrause 5y ago

The products are good, but NVidia also cleverly bootstrapped a whole ecosystem around them. One of the other posts mentions 2014 as a turning point. At that time, GPGPU stuff was entering the (scientific) mainstream and NVidia was all over academia, convincing people to try it out in their research. They handed out demo accounts on boxes with beefy GPUs, and ran an extremely generous hardware grant proposal. There was tons of (free) training available: online CUDA MOOCs and in-person training sessions. The first-party tools were pretty decent. As a result, people built a lot of stuff using CUDA. Others, wanting to use those programs, basically had to buy NVidia. Lather, rinse, repeat. This is in stark contrast to the other “accelerator” vendors. Around the same time, I looked into using Intel Xenon Phi and they were way less aggressive: “here are some benchmarks, here’s how it works, your institution has one or two somewhere if you want to contact them and try it out.” As for OpenCL…crickets. I don’t even remember any AMD events, and the very broad standard made it hard to figure out what would work/work well and you might end up needing to port it too!

jjoonathan 5y ago
AMD's GPU OpenCL wasn't just not marketed, it was also a bad product, even for relatively tame scientific purposes, even when AMD made loud, repeated statements to the contrary. Hopefully now that AMD has money they can do better. I'm sure that NVidia's ecosystem building played a role (I remember events in 2009 and books before that), perhaps even a big role, but it wasn't the only factor. I paid a steep price in 2014 and 2016 for incorrectly assuming that it was.