3 ms·

OpenCL is old, and everyone seemingly wants to abandon it, but it's your only option for a bunch of configurations (such as AMD on Windows, even though AMD has

OpenCL is old, and everyone seemingly wants to abandon it, but it's your only option for a bunch of configurations (such as AMD on Windows, even though AMD has pulled all their OpenCL stuff from their website). As someone who's done a fair bit of SIMD programming on CPU, and heard many scream that wide SIMD (like AVX512) is pointless when you have GPGPU, it's certainly eye-opening to see how poor a state cross-platform GPGPU development is in. Well, I suppose if you only care about Nvidia, CUDA does seem to be pretty good. Too bad it's Nvidia only. I've heard people consider Vulkan Compute as an alternative. I had a quick look, and it doesn't seem like it supports integer operations (what I'm mostly doing), so doesn't seem viable for me, but I guess it could for a bunch of folk. Not familiar with Vulkan myself though, so corrections welcome.

pavlov 5y ago
I'm genuinely surprised that AMD didn't put in an effort to implement CUDA themselves. It's been almost 15 years. They should have started a skunkworks project to get CUDA running immediately. Intel didn't take long to eat crow and ship the AMD64 instruction set in their CPUs as soon as it became clear they'd lost the 64-bit ISA game. AMD should have taken a lesson from that: if the market wants the other guy's API, you can implement it, customers will be pleased, and that gives you power over the API that you wouldn't have otherwise.
- RaisingSpear 5y ago
  > I'm genuinely surprised that AMD didn't put in an effort to implement CUDA themselves I think HIP is their attempt at that, as it's very similar to CUDA and is meant to be easy to port. Maybe there's reasons why they (or anyone else really) can't just adopt CUDA (licensing?), though that's beyond my knowledge.
raphlinus 5y ago
Vulkan supports integer operations just fine, but tool support for just about everything is extremely primitive. Sizes other than 32 bits are available, but generally as an option. You can check https://vulkan.gpuinfo.org/listfeaturescore10.php https://vulkan.gpuinfo.org/listfeaturescore10.php to see the fraction of drivers that support the various integer sizes (and lots other optional features).
- RaisingSpear 5y ago
  Thanks! I guess I got confused with the information out there. Most examples seem to be using GLSL shaders for the kernel, but posts seem to indicate it uses SPIR-V as input [https://community.khronos.org/t/is-a-vulkan-compute-shader-different-from-a-glsl-compute-shader/7473 https://community.khronos.org/t/is-a-vulkan-compute-shader-d...]. And then you have threads like this [https://community.amd.com/t5/drivers-software/amd-dropped-spir-support/td-p/405177 https://community.amd.com/t5/drivers-software/amd-dropped-sp...] saying that SPIR-V isn't supported on the AMD's Windows driver (I had similar issues myself trying to get SYCL examples to run), though I see other places running Vulkan Compute stuff on AMD+Windows. Maybe that only applies to SPIR-V with an OpenCL runtime? Diagrams [https://www.khronos.org/assets/uploads/apis/2020-spir-landing-page-01_1.jpg https://www.khronos.org/assets/uploads/apis/2020-spir-landin...] seem to indicate that you can feed stuff like OpenCL and SYCL into SPIR-V (and then Vulkan Compute) instead of GLSL. For the former case, would you essentially still be using OpenCL, but just with a different runtime? (though articles like this [https://linuxreviews.org/The_State_Of_OpenCL_To_Vulkan_Compute_Layers_On_Linux_In_April_2021 https://linuxreviews.org/The_State_Of_OpenCL_To_Vulkan_Compu...] seem to suggest OpenCL -> Vulkan isn't in a good state)
  - raphlinus 5y ago
    There's a lot here, let me try to clarify a bit. Most of the time, when people say OpenCL, they mean that an OpenCL driver is provided by the GPU vendor. That's what's in a particularly sorry state. Many vendors ship OpenCL but have deprecated it. AMD's ROCm is based on OpenCL, but they don't support it on all cards and there are problems. The other thing that's picking up steam lately is using a lower level API such as Vulkan as the interface to the graphics hardware, and having a layer that runs the compute workload (whether OpenCL or something else) on top of that. In my opinion, this actually has a pretty good future. On Linux, Vulkan is obviously the way to go, on Windows it's supported by all major cards, and on mac it's possible to fake it with MoltenVK. This is what clspv and clvk are about, but from the article you posted these are not in usable shape yet. I think that might say more about the level of interest around OpenCL than anything else though. It's entirely possible that running compute on Vulkan becomes mainstream through other efforts like IREE than by porting OpenCL workloads. In a couple years or so, there's a good chance the landscape will shift again, as WebGPU might be capable of running compute workloads well, and is likely to be supported by all vendors. Note that despite "web" in the name, it doesn't require a browser or other web technology. You can run prototypes of it today, but there are still huge chunks of the ecosystem missing.
jdashg 5y ago
Vulkan Compute is the alternative, and you should look again if you're serious.
- RaisingSpear 5y ago
  Thanks. According to this post [https://community.khronos.org/t/opencl-vs-vulkan-compute/7132/2 https://community.khronos.org/t/opencl-vs-vulkan-compute/713...], Vulkan Compute seems to be mostly focused on rendering over general compute. This is probably fine for Blender's renderer, but may not be suitable for some compute loads. Unless something has changed since then, or the post is inaccurate?
  - raphlinus 5y ago
    That post is 4 years old. Since then, Vulkan compute has made enormous strides forward, while OpenCL has stagnated. Examples of the things Vulkan has now are subgroup size control, explicit float16 and int8, pointers (previously you had to fake them with arrays), and a memory model. The memory model alone is a pretty big advance. What is still a mess is the tool situation. You have to write your shaders in GLSL or HLSL (the latter is still missing some subgroup operation) and compile to SPIR-V. You also have to have CPU-side code to manage all resources, including memory allocation and putting in explicit pipeline barriers and other ways to manage the asynchrony. For someone who just wants to get their code running, it's pretty painful. My own application is 2D rendering, but a lot of the motivation for improving Vulkan Compute is machine learning workloads. One of the projects to watch is IREE (https://google.github.io/iree/ https://google.github.io/iree/). I've got some blog posts and tutorials in the pipeline, stay tuned.
    - RaisingSpear 5y ago
      I see, thanks. I presume the end-user running the application will need to have up-to-date drivers to take advantage of these changes in the past 4 years?
      - raphlinus 5y ago

pavlov 5y ago

I'm genuinely surprised that AMD didn't put in an effort to implement CUDA themselves. It's been almost 15 years. They should have started a skunkworks project to get CUDA running immediately. Intel didn't take long to eat crow and ship the AMD64 instruction set in their CPUs as soon as it became clear they'd lost the 64-bit ISA game. AMD should have taken a lesson from that: if the market wants the other guy's API, you can implement it, customers will be pleased, and that gives you power over the API that you wouldn't have otherwise.

RaisingSpear 5y ago
> I'm genuinely surprised that AMD didn't put in an effort to implement CUDA themselves I think HIP is their attempt at that, as it's very similar to CUDA and is meant to be easy to port. Maybe there's reasons why they (or anyone else really) can't just adopt CUDA (licensing?), though that's beyond my knowledge.

raphlinus 5y ago

Vulkan supports integer operations just fine, but tool support for just about everything is extremely primitive. Sizes other than 32 bits are available, but generally as an option. You can check https://vulkan.gpuinfo.org/listfeaturescore10.php https://vulkan.gpuinfo.org/listfeaturescore10.php to see the fraction of drivers that support the various integer sizes (and lots other optional features).

RaisingSpear 5y ago
Thanks! I guess I got confused with the information out there. Most examples seem to be using GLSL shaders for the kernel, but posts seem to indicate it uses SPIR-V as input [https://community.khronos.org/t/is-a-vulkan-compute-shader-different-from-a-glsl-compute-shader/7473 https://community.khronos.org/t/is-a-vulkan-compute-shader-d...]. And then you have threads like this [https://community.amd.com/t5/drivers-software/amd-dropped-spir-support/td-p/405177 https://community.amd.com/t5/drivers-software/amd-dropped-sp...] saying that SPIR-V isn't supported on the AMD's Windows driver (I had similar issues myself trying to get SYCL examples to run), though I see other places running Vulkan Compute stuff on AMD+Windows. Maybe that only applies to SPIR-V with an OpenCL runtime? Diagrams [https://www.khronos.org/assets/uploads/apis/2020-spir-landing-page-01_1.jpg https://www.khronos.org/assets/uploads/apis/2020-spir-landin...] seem to indicate that you can feed stuff like OpenCL and SYCL into SPIR-V (and then Vulkan Compute) instead of GLSL. For the former case, would you essentially still be using OpenCL, but just with a different runtime? (though articles like this [https://linuxreviews.org/The_State_Of_OpenCL_To_Vulkan_Compute_Layers_On_Linux_In_April_2021 https://linuxreviews.org/The_State_Of_OpenCL_To_Vulkan_Compu...] seem to suggest OpenCL -> Vulkan isn't in a good state)
- raphlinus 5y ago
  There's a lot here, let me try to clarify a bit. Most of the time, when people say OpenCL, they mean that an OpenCL driver is provided by the GPU vendor. That's what's in a particularly sorry state. Many vendors ship OpenCL but have deprecated it. AMD's ROCm is based on OpenCL, but they don't support it on all cards and there are problems. The other thing that's picking up steam lately is using a lower level API such as Vulkan as the interface to the graphics hardware, and having a layer that runs the compute workload (whether OpenCL or something else) on top of that. In my opinion, this actually has a pretty good future. On Linux, Vulkan is obviously the way to go, on Windows it's supported by all major cards, and on mac it's possible to fake it with MoltenVK. This is what clspv and clvk are about, but from the article you posted these are not in usable shape yet. I think that might say more about the level of interest around OpenCL than anything else though. It's entirely possible that running compute on Vulkan becomes mainstream through other efforts like IREE than by porting OpenCL workloads. In a couple years or so, there's a good chance the landscape will shift again, as WebGPU might be capable of running compute workloads well, and is likely to be supported by all vendors. Note that despite "web" in the name, it doesn't require a browser or other web technology. You can run prototypes of it today, but there are still huge chunks of the ecosystem missing.

jdashg 5y ago

Vulkan Compute is the alternative, and you should look again if you're serious.

RaisingSpear 5y ago
Thanks. According to this post [https://community.khronos.org/t/opencl-vs-vulkan-compute/7132/2 https://community.khronos.org/t/opencl-vs-vulkan-compute/713...], Vulkan Compute seems to be mostly focused on rendering over general compute. This is probably fine for Blender's renderer, but may not be suitable for some compute loads. Unless something has changed since then, or the post is inaccurate?
- raphlinus 5y ago
  That post is 4 years old. Since then, Vulkan compute has made enormous strides forward, while OpenCL has stagnated. Examples of the things Vulkan has now are subgroup size control, explicit float16 and int8, pointers (previously you had to fake them with arrays), and a memory model. The memory model alone is a pretty big advance. What is still a mess is the tool situation. You have to write your shaders in GLSL or HLSL (the latter is still missing some subgroup operation) and compile to SPIR-V. You also have to have CPU-side code to manage all resources, including memory allocation and putting in explicit pipeline barriers and other ways to manage the asynchrony. For someone who just wants to get their code running, it's pretty painful. My own application is 2D rendering, but a lot of the motivation for improving Vulkan Compute is machine learning workloads. One of the projects to watch is IREE (https://google.github.io/iree/ https://google.github.io/iree/). I've got some blog posts and tutorials in the pipeline, stay tuned.
  - RaisingSpear 5y ago
    I see, thanks. I presume the end-user running the application will need to have up-to-date drivers to take advantage of these changes in the past 4 years?
    - raphlinus 5y ago