5 ms·

Does anyone familiar with the state of GPU programming think OpenCL will eventually 'win' over CUDA? Although CUDA has more adoption, I don't like the idea of u

Does anyone familiar with the state of GPU programming think OpenCL will eventually 'win' over CUDA? Although CUDA has more adoption, I don't like the idea of using it and being locked into a specific vendor. Of course nVidia is only supporting outdated versions of OpenCL for now. Am I a fool for hoping OpenCL eventually becomes the standard?

arcanus 10y ago
I think that the closed nature of CUDA will be its undoing. I think that a standard, like C++ amp or openMP-4.5 will be the the ultimate winner. I liked openCL but it seems to be dying.
- jupiter90000 10y ago
  I'm not familiar with those other standards, thanks for mentioning them. I'll check them out.
- ahelwer 10y ago
  Is C++ AMP still going? I learned it about five years ago back when I was involved in GPU programming, haven't heard a thing about it since then.
  - jupiter90000 10y ago
    Looks like it might be dead/dying: http://stackoverflow.com/questions/34969287/what-is-the-current-status-of-c-amp http://stackoverflow.com/questions/34969287/what-is-the-curr... Although perhaps would still be useful in its stale state, not sure.
- programmarchy 10y ago
  Apple seems to have abandoned OpenCL in favor of Metal, which speaks to your case of it dying. I found Metal Compute Shaders to be very nice to work with, though. Was much easier for me to understand than OpenCL.
  - arcanus 10y ago
    I also like metal, but it is not yet performant for high performance computing, which is more my wheelhouse. I'm also skeptical it will be popular if it does not get picked up by the GPGPU folks, but time will tell.
jlebar 10y ago
Honestly for those of us in machine learning, I think something like XLA will likely win over both paradigms. (Disclaimer, I work on XLA.) https://www.tensorflow.org/versions/master/experimental/xla/ https://www.tensorflow.org/versions/master/experimental/xla/ XLA much more closely matches what you want for ML than CUDA/opencl. Which isn't a surprise; it was designed specifically for ML. Kernel launches are expensive, so any fast CUDA system has to let you compose computations into a single kernel (e.g. multiply by 5 and then take tanh). It's possible to do this in CUDA, but it requires heoric C++ template metaprogramming. It's not uncommon to have files that take ten minutes to compile. Whereas in XLA kernel fusion is nbd, because it's a JIT. Also, because XLA is generating GPU code after it's seen your model, it can specialize computations specifically to your model. In regular TensorFlow (and I presume other ML frameworks, although I'm not at all familiar with them), you have to compile all of your kernels upfront. This means that the framework probably doesn't have the ideal set of kernels for your model, because the framework's set of kernels needs to be generic. For example, the framework probably isn't going to have a "multiply by 5 and then take tanh" kernel -- if you're lucky, it might have a "multiply by X and then take tanh", but notice that this may be slower because X is now not a constant. In contrast, not only can XLA specialize for your weird X==5 case, but it can also specialize all of the dimensions of your arrays. This is a really big advantage in many cases. As just one example, it's common for kernels to do something like int index = some computation based on threadIdx and blockIdx; if (index < array_len) { ... } But in XLA we know the size of the kernel, so we know the possible values for threadIdx and blockIdx, and we know the exact value of array_len. We can therefore often optimize out the if entirely.
- disposablename 10y ago
  Why no AMD GPU support?
  - jlebar 10y ago
    Why no AMD GPU support? I think it just reflects the team's internal priorities. Patches are welcome; we want people to use this system. It wouldn't even be tremendously hard. The XLA IR --> LLVM IR backend is relatively simple, and LLVM already has support for compiling to AMD GPUs. You'd have to split out the nvidia-isms in the generated IR. I think the biggest challenge would just be one of software engineering, namely figuring out a way to specialize the GPU backend for each of the two architectures while allowing it to share code in general.
joe_the_user 10y ago
I've been evaluating Cuda and OpenCL while trying to produce some target independent code. My impression is that while Cuda might not win, OpenCL will almost certainly lose. OpenCL seems to be a monster compromise interface which takes into account all the architectures of the members of a large consortium. It's the sort-of designed-by-committee api that a developer has to fight against to accomplish anything. Naturally its many years behind Cuda in features, etc. An open-source library with equivalent qualities to Cuda is needed- ie, a library intended to aid developers, allow abstract c++ to be easily become parallel code, provide reasonable tools and documentation etc. One promising example is amd's Hip "HIP allows developers to convert CUDA code to portable C++. The same source code can be compiled to run on NVIDIA or AMD GPUs." https://github.com/GPUOpen-ProfessionalCompute-Tools/HIP https://github.com/GPUOpen-ProfessionalCompute-Tools/HIP
- rayuela 10y ago
  Hadn't heard of this before. This is pretty cool. Does this project have official AMD support?
  - joe_the_user 10y ago
    What I remember from this being on hn months ago is that is this an official AMD project aiming to compete with nVidia.
    - joe_the_user 10y ago
      See https://www.phoronix.com/scan.php?page=news_item&px=AMD-GPUOpen-HIP-Summer https://www.phoronix.com/scan.php?page=news_item&px=AMD-GPUO... I think...
se6 10y ago
In 2013 we started GPU programming at the company I work for. We carefully evaluated CUDA and OpenCL and decided to go for OpenCL because it was a standard and we could chose between 2 vendors of GPU. I can tell you that in 2017 we do not regret our choice. It is great to be able to run our code on both AMD and NVidia GPUs, and to offer our customers to choose whichever GPU vendor they prefer. Many people criticise OpenCL because when you come from C++ it seems a lot of work. It is true that OpenCL has an API influenced by OpenGL and is verbose. However it is not difficult to write a small framework specific to your needs and domain to factorise much of this verbosity. NVidia does everything it can to hide the fact that their devices support OpenCL. People thinks that only ancients versions of OpenCL run on NVidia devices. That is not true: 1.2 is not ancient is still as of today the main version of OpenCL used. OpenCL 1.2 is fully supported and NVidia quietly say to its large customers who refuse to use CUDA, that they will starting to support soon some OpenCL 2.0 features. To answer your question, I am not sure either will win, but they will both exist for a long time.
- jupiter90000 10y ago
  Thank you, this is very helpful information I was hoping to hear.

arcanus 10y ago

I think that the closed nature of CUDA will be its undoing. I think that a standard, like C++ amp or openMP-4.5 will be the the ultimate winner. I liked openCL but it seems to be dying.

jupiter90000 10y ago
I'm not familiar with those other standards, thanks for mentioning them. I'll check them out.
ahelwer 10y ago
Is C++ AMP still going? I learned it about five years ago back when I was involved in GPU programming, haven't heard a thing about it since then.
- jupiter90000 10y ago
  Looks like it might be dead/dying: http://stackoverflow.com/questions/34969287/what-is-the-current-status-of-c-amp http://stackoverflow.com/questions/34969287/what-is-the-curr... Although perhaps would still be useful in its stale state, not sure.
programmarchy 10y ago
Apple seems to have abandoned OpenCL in favor of Metal, which speaks to your case of it dying. I found Metal Compute Shaders to be very nice to work with, though. Was much easier for me to understand than OpenCL.
- arcanus 10y ago
  I also like metal, but it is not yet performant for high performance computing, which is more my wheelhouse. I'm also skeptical it will be popular if it does not get picked up by the GPGPU folks, but time will tell.

jlebar 10y ago

Honestly for those of us in machine learning, I think something like XLA will likely win over both paradigms. (Disclaimer, I work on XLA.) https://www.tensorflow.org/versions/master/experimental/xla/ https://www.tensorflow.org/versions/master/experimental/xla/ XLA much more closely matches what you want for ML than CUDA/opencl. Which isn't a surprise; it was designed specifically for ML. Kernel launches are expensive, so any fast CUDA system has to let you compose computations into a single kernel (e.g. multiply by 5 and then take tanh). It's possible to do this in CUDA, but it requires heoric C++ template metaprogramming. It's not uncommon to have files that take ten minutes to compile. Whereas in XLA kernel fusion is nbd, because it's a JIT. Also, because XLA is generating GPU code after it's seen your model, it can specialize computations specifically to your model. In regular TensorFlow (and I presume other ML frameworks, although I'm not at all familiar with them), you have to compile all of your kernels upfront. This means that the framework probably doesn't have the ideal set of kernels for your model, because the framework's set of kernels needs to be generic. For example, the framework probably isn't going to have a "multiply by 5 and then take tanh" kernel -- if you're lucky, it might have a "multiply by X and then take tanh", but notice that this may be slower because X is now not a constant. In contrast, not only can XLA specialize for your weird X==5 case, but it can also specialize all of the dimensions of your arrays. This is a really big advantage in many cases. As just one example, it's common for kernels to do something like int index = some computation based on threadIdx and blockIdx; if (index < array_len) { ... } But in XLA we know the size of the kernel, so we know the possible values for threadIdx and blockIdx, and we know the exact value of array_len. We can therefore often optimize out the if entirely.

disposablename 10y ago
Why no AMD GPU support?
- jlebar 10y ago
  Why no AMD GPU support? I think it just reflects the team's internal priorities. Patches are welcome; we want people to use this system. It wouldn't even be tremendously hard. The XLA IR --> LLVM IR backend is relatively simple, and LLVM already has support for compiling to AMD GPUs. You'd have to split out the nvidia-isms in the generated IR. I think the biggest challenge would just be one of software engineering, namely figuring out a way to specialize the GPU backend for each of the two architectures while allowing it to share code in general.

joe_the_user 10y ago

I've been evaluating Cuda and OpenCL while trying to produce some target independent code. My impression is that while Cuda might not win, OpenCL will almost certainly lose. OpenCL seems to be a monster compromise interface which takes into account all the architectures of the members of a large consortium. It's the sort-of designed-by-committee api that a developer has to fight against to accomplish anything. Naturally its many years behind Cuda in features, etc. An open-source library with equivalent qualities to Cuda is needed- ie, a library intended to aid developers, allow abstract c++ to be easily become parallel code, provide reasonable tools and documentation etc. One promising example is amd's Hip "HIP allows developers to convert CUDA code to portable C++. The same source code can be compiled to run on NVIDIA or AMD GPUs." https://github.com/GPUOpen-ProfessionalCompute-Tools/HIP https://github.com/GPUOpen-ProfessionalCompute-Tools/HIP

rayuela 10y ago
Hadn't heard of this before. This is pretty cool. Does this project have official AMD support?
- joe_the_user 10y ago
  What I remember from this being on hn months ago is that is this an official AMD project aiming to compete with nVidia.
  - joe_the_user 10y ago
    See https://www.phoronix.com/scan.php?page=news_item&px=AMD-GPUOpen-HIP-Summer https://www.phoronix.com/scan.php?page=news_item&px=AMD-GPUO... I think...

se6 10y ago

In 2013 we started GPU programming at the company I work for. We carefully evaluated CUDA and OpenCL and decided to go for OpenCL because it was a standard and we could chose between 2 vendors of GPU. I can tell you that in 2017 we do not regret our choice. It is great to be able to run our code on both AMD and NVidia GPUs, and to offer our customers to choose whichever GPU vendor they prefer. Many people criticise OpenCL because when you come from C++ it seems a lot of work. It is true that OpenCL has an API influenced by OpenGL and is verbose. However it is not difficult to write a small framework specific to your needs and domain to factorise much of this verbosity. NVidia does everything it can to hide the fact that their devices support OpenCL. People thinks that only ancients versions of OpenCL run on NVidia devices. That is not true: 1.2 is not ancient is still as of today the main version of OpenCL used. OpenCL 1.2 is fully supported and NVidia quietly say to its large customers who refuse to use CUDA, that they will starting to support soon some OpenCL 2.0 features. To answer your question, I am not sure either will win, but they will both exist for a long time.

jupiter90000 10y ago
Thank you, this is very helpful information I was hoping to hear.