4 ms·

>their equivalent of CUDA Nvidia didn't pull their AI leadership out of thin air overnight, but they shipped the first CUDA capable consumer cards with the Ge

>their equivalent of CUDA Nvidia didn't pull their AI leadership out of thin air overnight, but they shipped the first CUDA capable consumer cards with the GeForce 8000 series way back in 2007 and committed to this ecosystem over the years, consistently investing in the HW, SW. By the time AMD woke up and shipped ROCm in 2016, Nvidia already had nearly 10 years head start and a cemented moat in this field. AMD now has a huge mountain to climb to catch up to Nvidia.

mahkeiro 3y ago
I always had hope that ROCm will be able to compete with CUDA but it’s nowhere here despite the time. Seems funny to see that Intel is doing a better job at that with OneAPI.
bryanlarsen 3y ago
AMD invested significantly into OpenCL prior to 2016. It seemed like a safe bet -- the open industry standard usually ends up beating the proprietary standard in the long run. Especially for something like this, with massive open source / open standard companies like Google as heavy users. It seems surprising to me that Google didn't ensure that open standards won in an area that they are so heavily dependent on.
- ChuckNorris89 3y ago
  Because all the researchers that used GPUs for CV and ML used what they had at their disposal, which was Nvidia GPUs and CUDA. OpenCL brought no advantage here considering it only worked on AMD GPUs which were lackluster in performance and switching from CUDA to OpenCL meant extra work that researches already iterating on CUDA weren't willing to do.
  - cibyr 3y ago
    OpenCL works fine on Nvidia GPUs. It's more annoying to code for, and harder to get good performance out of than CUDA, but it works.
    - dragontamer 3y ago
      Poorly, in my experience. CUDA is compiled into PTX, an intermediate language. PTX is then compiled into a specific NVidia assembly language (often called SASS, though each SASS for each generation of cards is different). This way, NVidia can make huge changes to the underlying assembly code from generation-to-generation, but still have portability. OpenCL, especially OpenCL 1.2, (which is the version of OpenCL that works on the widest set of cards), does not have an intermediate language. SPIR is an OpenCL2.+ concept. This means that OpenCL 1.2 code is distributed in source and recompiled in practice. But that means that compiler errors can kill your code before it even runs. This is especially annoying because the OpenCL 1.2 compiler is part of a device-driver. Meaning if the end-user updates the device driver, the compiler may have a new bug (or old bug), that changes the behavior of your code. ------------- This doesn't matter for DirectX, because like CUDA, Microsoft compiles DirectX into DXIR / DirectX intermediate language. And then has device drivers compile the intermediate-language into the final assembly code on a per-device basis. ------------- It is this intermediate layer that AMD is missing, and IMO is the key to their problems in practice. SPIR (OpenCL's standard intermediate layer) has spotty support across cards. I'm guessing NVidia knows that PTX intermediate language is their golden goose and doesn't want to offer good SPIR support. Microsoft probably prefers people to use DirectX / DXIR as well. So that leaves AMD and Intel as the only groups who could possibly push SPIR and align together. SPIR is a good idea, but I'm not sure if the politics will allow it to happen.
      - KeplerBoy 3y ago
        OpenCL also gets compiled to PTX on Nvidia GPUs.
        dragontamer 3y ago
        But as an OpenCL programmer, you don't distribute PTX intermediate code. You distribute OpenCL kernels around and recompile every time. That's more or less the practice.
        KeplerBoy 3y ago
  - DeepYogurt 3y ago
    OpenCL did (and still does I think) work on nvidia cards. People I talked to back in the day complained more about OpenCL being "C but on GPUs" while cuda was more akin to C++. They could move faster and do more in cuda and the nvidia lock in didn't matter as the fastest cards of the day were nvidia. I think vega cards were faster (or faster per dollar maybe) for some of the code that was relevant, but not by much and by that point legacy code lock in had taken over.
- belval 3y ago
  If you ever get the chance to write something in OpenCL and then in CUDA I promise you will understand immediately why Google didn't push for it. There is a lot more boilerplate to it, read/write to the buffer, queueing being handled explicitly. Here's an example that illustrate what I mean: https://github.com/rsnemmen/OpenCL-examples/blob/master/mandelbrot/mandelbrot.c#L12 https://github.com/rsnemmen/OpenCL-examples/blob/master/mand... For comparison here is an implementation in CUDA: http://selkie.macalester.edu/csinparallel/modules/CUDAArchitecture/build/html/1-Mandelbrot/Mandelbrot.html http://selkie.macalester.edu/csinparallel/modules/CUDAArchit... Notice how the CUDA code is more readable.
  - bryanlarsen 3y ago
    Google is part of the Khronos group. They were well positioned to steer the standard towards one that doesn't suck. Or they could have championed a different standard. Google has the scale that only Google is to blame that they are still heavily dependent on a closed standard. Open Standards almost always beat closed ones. IMO AMD was right to bet on open standards. They lost the bet but I think it was the right bet.
    - barbariangrunge 3y ago
      > the Khronos group I still wake up at night, sweating, thinking about my time learning OpenGL in university. The docs were a nightmare back then
      - shaklee3 3y ago
        openCL != openGL
  - SuchAnonMuchWow 3y ago
    In the link provided, the CUDA example only show the compute kernel itself and not the boilerplate required to run it. On the other hand, your OpenCL example only show the boilerplate. This is the OpenCL kernel from the same repo, for a more fair comparison: https://github.com/rsnemmen/OpenCL-examples/blob/master/mandelbrot/mandelbrot.cl https://github.com/rsnemmen/OpenCL-examples/blob/master/mand... This is much more readable. OpenCL-C the language is fine: it's how you deploy the program on the cards that is complicated with opencl.
Keyframe 3y ago
Nvidia took a big gamble with CUDA and it took years and ton of investment to get there. Jensen Huang talks about it on commencement speech he did in Taiwan recently here https://www.youtube.com/watch?v=oi89u6q0_AY https://www.youtube.com/watch?v=oi89u6q0_AY It's a big moat to cross.

mahkeiro 3y ago

I always had hope that ROCm will be able to compete with CUDA but it’s nowhere here despite the time. Seems funny to see that Intel is doing a better job at that with OneAPI.

bryanlarsen 3y ago

AMD invested significantly into OpenCL prior to 2016. It seemed like a safe bet -- the open industry standard usually ends up beating the proprietary standard in the long run. Especially for something like this, with massive open source / open standard companies like Google as heavy users. It seems surprising to me that Google didn't ensure that open standards won in an area that they are so heavily dependent on.

ChuckNorris89 3y ago
Because all the researchers that used GPUs for CV and ML used what they had at their disposal, which was Nvidia GPUs and CUDA. OpenCL brought no advantage here considering it only worked on AMD GPUs which were lackluster in performance and switching from CUDA to OpenCL meant extra work that researches already iterating on CUDA weren't willing to do.
- cibyr 3y ago
  OpenCL works fine on Nvidia GPUs. It's more annoying to code for, and harder to get good performance out of than CUDA, but it works.
  - dragontamer 3y ago
    Poorly, in my experience. CUDA is compiled into PTX, an intermediate language. PTX is then compiled into a specific NVidia assembly language (often called SASS, though each SASS for each generation of cards is different). This way, NVidia can make huge changes to the underlying assembly code from generation-to-generation, but still have portability. OpenCL, especially OpenCL 1.2, (which is the version of OpenCL that works on the widest set of cards), does not have an intermediate language. SPIR is an OpenCL2.+ concept. This means that OpenCL 1.2 code is distributed in source and recompiled in practice. But that means that compiler errors can kill your code before it even runs. This is especially annoying because the OpenCL 1.2 compiler is part of a device-driver. Meaning if the end-user updates the device driver, the compiler may have a new bug (or old bug), that changes the behavior of your code. ------------- This doesn't matter for DirectX, because like CUDA, Microsoft compiles DirectX into DXIR / DirectX intermediate language. And then has device drivers compile the intermediate-language into the final assembly code on a per-device basis. ------------- It is this intermediate layer that AMD is missing, and IMO is the key to their problems in practice. SPIR (OpenCL's standard intermediate layer) has spotty support across cards. I'm guessing NVidia knows that PTX intermediate language is their golden goose and doesn't want to offer good SPIR support. Microsoft probably prefers people to use DirectX / DXIR as well. So that leaves AMD and Intel as the only groups who could possibly push SPIR and align together. SPIR is a good idea, but I'm not sure if the politics will allow it to happen.
    - KeplerBoy 3y ago
      OpenCL also gets compiled to PTX on Nvidia GPUs.
      - dragontamer 3y ago
        But as an OpenCL programmer, you don't distribute PTX intermediate code. You distribute OpenCL kernels around and recompile every time. That's more or less the practice.
        KeplerBoy 3y ago
- DeepYogurt 3y ago
  OpenCL did (and still does I think) work on nvidia cards. People I talked to back in the day complained more about OpenCL being "C but on GPUs" while cuda was more akin to C++. They could move faster and do more in cuda and the nvidia lock in didn't matter as the fastest cards of the day were nvidia. I think vega cards were faster (or faster per dollar maybe) for some of the code that was relevant, but not by much and by that point legacy code lock in had taken over.
belval 3y ago
If you ever get the chance to write something in OpenCL and then in CUDA I promise you will understand immediately why Google didn't push for it. There is a lot more boilerplate to it, read/write to the buffer, queueing being handled explicitly. Here's an example that illustrate what I mean: https://github.com/rsnemmen/OpenCL-examples/blob/master/mandelbrot/mandelbrot.c#L12 https://github.com/rsnemmen/OpenCL-examples/blob/master/mand... For comparison here is an implementation in CUDA: http://selkie.macalester.edu/csinparallel/modules/CUDAArchitecture/build/html/1-Mandelbrot/Mandelbrot.html http://selkie.macalester.edu/csinparallel/modules/CUDAArchit... Notice how the CUDA code is more readable.
- bryanlarsen 3y ago
  Google is part of the Khronos group. They were well positioned to steer the standard towards one that doesn't suck. Or they could have championed a different standard. Google has the scale that only Google is to blame that they are still heavily dependent on a closed standard. Open Standards almost always beat closed ones. IMO AMD was right to bet on open standards. They lost the bet but I think it was the right bet.
  - barbariangrunge 3y ago
    > the Khronos group I still wake up at night, sweating, thinking about my time learning OpenGL in university. The docs were a nightmare back then
    - shaklee3 3y ago
      openCL != openGL
- SuchAnonMuchWow 3y ago
  In the link provided, the CUDA example only show the compute kernel itself and not the boilerplate required to run it. On the other hand, your OpenCL example only show the boilerplate. This is the OpenCL kernel from the same repo, for a more fair comparison: https://github.com/rsnemmen/OpenCL-examples/blob/master/mandelbrot/mandelbrot.cl https://github.com/rsnemmen/OpenCL-examples/blob/master/mand... This is much more readable. OpenCL-C the language is fine: it's how you deploy the program on the cards that is complicated with opencl.

Keyframe 3y ago

Nvidia took a big gamble with CUDA and it took years and ton of investment to get there. Jensen Huang talks about it on commencement speech he did in Taiwan recently here https://www.youtube.com/watch?v=oi89u6q0_AY https://www.youtube.com/watch?v=oi89u6q0_AY It's a big moat to cross.