3 ms·

"just create a new instruction set"? It's taken the better part of a decade for workable alternatives to CUDA to emerge.

irrational 5y ago
By that logic there will never again be another workable alternative. Why not start now rather than wait a few decades?
- InvertedRhodium 5y ago
  There's nothing that prevents you from doing that as it stands.
- nerdponx 5y ago
  Innovation is not an on-off switch. Market forces create an incentive to attempt to innovate, but they don't magically make innovation happen.
- vxNsr 5y ago
  It's likely that there are many new designs being spun up everyday by various people, but that doesn't mean that we want to remove competition from the market.
techdragon 5y ago
I hope you’re not calling ROCm a workable alternative… it’s really not close. I love that they are trying, I just honestly feel they need to be trying with 10 times as much funding/resources behind them. Or do you mean Apple’s Metal Compute? useful only with inconvenient (at scale) Apple hardware. If workable CUDA alternatives were really here, I feel people would be talking about them more given how large the hate is for Nvidia on multi fronts from multiple different user groups.
- ivirshup 5y ago
  I was, but mostly through XLA or Vulkan layers. Also, I’d define workable as a pretty low bar.
  - techdragon 5y ago
    That’s fair, for me it never worked with any of the hardware or software I wanted it to and I’m not enough of a GPU expert to really fix that myself, so for me and many others I found with similar problems it definitely has yet to become a workable solution, but I was probably a bit harsh if I came across like I thought it didn’t work at all, I’ve seen it demoed and I know it works, but for such a restrictive set of hardware and software that Ive never seen it in use myself outside of demonstrations. As for Vulkan, I tend to think of CUDA at the driver level given how closely coupled it is to Nvidia’s hardware. So I wasn’t really thinking about broader cross compatible APIs like Vulkan which tend to get implemented on top of the drivers providing low level access like CUDA gives… but with the exception of Apple (because we know they will never change their mind) I’m hoping the broader industry gets behind Vulkan compute shaders enough we finally get something that delivers on the promises made when the OpenCL effort began. I want clean understandable abstractions over the top of all the different SIMD and MIMD capabilities we have these days it doesn’t have to magically compile my code down to FPGA hardware but how about actually being able to get the best matrix multiplication performance out of both my CPU or GPU or both if I have a CPU with the right kind of integrated GPU that they can efficiently enough share memory and both work together without slowing each other down when working on an embarrassingly parallel matrix multiplication task. Is this really too much to ask of software in an an era that gives us near magical JIT performance in multiple languages, and manages to build cross compile/transpilation tools that can convert entire assembly code bases into JavaScript and all the other nice things we have as software developers… it never felt like too much to me but the fate of OpenCL appears to disagree.
torginus 5y ago
I never understood the dominance of CUDA. I did GPGPU more than half a decade ago, and I feel like stuff has hardly changed. There are thread groups, group shared variables, global, and group shared atomics etc. Underneath, all GPUs run the same-ish wide SIMD architecture, so I'm pretty sure most stuff is semi-performance portable. DirectCompute, OpenCL and CUDA and probably all the others expose this same programming model. Why is CUDA so dominant?
- michaelt 5y ago
  > Why is CUDA so dominant? nvidia decided they wanted to be the market leader, and spent $$$$ developing tools, training materials, and libraries (like cuDNN) which they gave away for free. Other GPU vendors chose not to outspend them.
  - ivirshup 5y ago
    They also hire software engineers to integrate CUDA into open source projects. On the one hand, those projects can now take advantage of advances in hardware. On the other, users will have to buy NVIDIA to use it, so how open is it? Also NVIDIA will not provide any help with CI, so those costs are borne by the project – or the code isn't tested.
ekianjo 5y ago
Which alternative? When it comes to ML there is no alternative to CUDA.
- ivirshup 5y ago
  JAX/ XLA seem quite popular, especially since you can use them in Colab on TPUs. IIRC, there are people who have managed get jax compiled with ROCm support. Tensorflow lets you deploy to quite a few backends. oneAPI support in GPUArrays.jl seems to be coming along.

irrational 5y ago

By that logic there will never again be another workable alternative. Why not start now rather than wait a few decades?

InvertedRhodium 5y ago
There's nothing that prevents you from doing that as it stands.
nerdponx 5y ago
Innovation is not an on-off switch. Market forces create an incentive to attempt to innovate, but they don't magically make innovation happen.
vxNsr 5y ago
It's likely that there are many new designs being spun up everyday by various people, but that doesn't mean that we want to remove competition from the market.

techdragon 5y ago

I hope you’re not calling ROCm a workable alternative… it’s really not close. I love that they are trying, I just honestly feel they need to be trying with 10 times as much funding/resources behind them. Or do you mean Apple’s Metal Compute? useful only with inconvenient (at scale) Apple hardware. If workable CUDA alternatives were really here, I feel people would be talking about them more given how large the hate is for Nvidia on multi fronts from multiple different user groups.

ivirshup 5y ago
I was, but mostly through XLA or Vulkan layers. Also, I’d define workable as a pretty low bar.
- techdragon 5y ago
  That’s fair, for me it never worked with any of the hardware or software I wanted it to and I’m not enough of a GPU expert to really fix that myself, so for me and many others I found with similar problems it definitely has yet to become a workable solution, but I was probably a bit harsh if I came across like I thought it didn’t work at all, I’ve seen it demoed and I know it works, but for such a restrictive set of hardware and software that Ive never seen it in use myself outside of demonstrations. As for Vulkan, I tend to think of CUDA at the driver level given how closely coupled it is to Nvidia’s hardware. So I wasn’t really thinking about broader cross compatible APIs like Vulkan which tend to get implemented on top of the drivers providing low level access like CUDA gives… but with the exception of Apple (because we know they will never change their mind) I’m hoping the broader industry gets behind Vulkan compute shaders enough we finally get something that delivers on the promises made when the OpenCL effort began. I want clean understandable abstractions over the top of all the different SIMD and MIMD capabilities we have these days it doesn’t have to magically compile my code down to FPGA hardware but how about actually being able to get the best matrix multiplication performance out of both my CPU or GPU or both if I have a CPU with the right kind of integrated GPU that they can efficiently enough share memory and both work together without slowing each other down when working on an embarrassingly parallel matrix multiplication task. Is this really too much to ask of software in an an era that gives us near magical JIT performance in multiple languages, and manages to build cross compile/transpilation tools that can convert entire assembly code bases into JavaScript and all the other nice things we have as software developers… it never felt like too much to me but the fate of OpenCL appears to disagree.

torginus 5y ago

I never understood the dominance of CUDA. I did GPGPU more than half a decade ago, and I feel like stuff has hardly changed. There are thread groups, group shared variables, global, and group shared atomics etc. Underneath, all GPUs run the same-ish wide SIMD architecture, so I'm pretty sure most stuff is semi-performance portable. DirectCompute, OpenCL and CUDA and probably all the others expose this same programming model. Why is CUDA so dominant?

michaelt 5y ago
> Why is CUDA so dominant? nvidia decided they wanted to be the market leader, and spent $$$$ developing tools, training materials, and libraries (like cuDNN) which they gave away for free. Other GPU vendors chose not to outspend them.
- ivirshup 5y ago
  They also hire software engineers to integrate CUDA into open source projects. On the one hand, those projects can now take advantage of advances in hardware. On the other, users will have to buy NVIDIA to use it, so how open is it? Also NVIDIA will not provide any help with CI, so those costs are borne by the project – or the code isn't tested.

ekianjo 5y ago

Which alternative? When it comes to ML there is no alternative to CUDA.

ivirshup 5y ago
JAX/ XLA seem quite popular, especially since you can use them in Colab on TPUs. IIRC, there are people who have managed get jax compiled with ROCm support. Tensorflow lets you deploy to quite a few backends. oneAPI support in GPUArrays.jl seems to be coming along.