6 ms·

We need a CUDA alternative for non NVIDIA gpu's, specially on a Mac.

by suyash 7y ago

nevi-me 7y ago
Would this from AMD count? https://rocm.github.io/ https://rocm.github.io/
- PeterisP 7y ago
  It's not supported on Mac, and even on Linux seems to have weird limitations on specific kernel versions. It also supports just a subset of AMD chips, which doesn't seem to include the actual AMD chips of the different Macs that I have available. ROCm would count if it's mature enough so that the setup "just works" on any reasonable environment (in the way that it mostly is so for the major ML platforms on nvidia/CUDA) but it does not yet seem ready for that.
  - nevi-me 7y ago
    You make a good point, I forgot that it's not Mac or Windows compatible, and the limited hardware support.
Athas 7y ago
OpenCL is decent, and although it's deprecated, it's probably going to continue working for years. It's not as nice as CUDA for direct programming, but don't most people access CUDA through higher-level libraries anyway? You probably would not be able to tell whether those use OpenCL or CUDA behind the scenes.
- fulafel 7y ago
  Deprecated by Apple in regards native OS X support for OpenCL, important distinction.
ActorNightly 7y ago
I dont really understand this, why would you use a Mac for anything compute intensive?
- mikkelam 7y ago
  hackintosh
  - vojta_letal 7y ago
    That's such a narrow segment of users. Would it be worth supporting?
    - rob74 7y ago
      Even if it was worth supporting, I guess it's understandable that Apple wouldn't go out of its way to do it...
  - deleted 7y ago
    [deleted]
zozbot234 7y ago
AIUI, Vulkan can be used for compute - so why not do that? It might work reasonably well on both nVidia and non-nVidia hardware. Of course, this requires GPU-enabled software to implement Vulkan support, but that's one-time work.
- Athas 7y ago
  Last year, I supervised a student project that involved writing a Vulkan backend for a compiler that already possessed CUDA and OpenCL backends. While I recall that everything ended up working, it didn't run all that fast. The vast majority of the problems were related to the language used for encoding the shaders, SPIR-V. Among the difficulties we encountered, I recall the need to use an offline optimiser to do trivial things like CSE and constant folding. CUDA and OpenCL will do this during kernel compilation, but Vulkan implementations seems to be designed as non-optimising (in order for loading to be faster, I assume). This is a perfectly understandable design, but it means it's a little more awkward to use. For direct programming (as compared to a compilation target), it's a lot more awkward. Another issue was that basic Vulkan is very restricted, as I assume it's supposed to be usable on simple hardware. Additional functionality can be enabled through a collection of extensions. One particular problem I recall is that our code generator assumes the existence of general pointers, for example such that the same memory can be used for different types of values at different times. SPIR-V has/had a very strict notion of pointers, and no way to cast between different types without an extension, and that extension was IIRC not implemented by NVIDIA or AMD. There were lots of these kinds of issues. In the end, Vulkan is usable for compute - after all, a single master's student (admittedly one who's very bright) managed to implement a Vulkan-targeting compiler backend in half a year. However, at the time I concluded that it's not as mature or as practical as CUDA or OpenCL, for reasons that seem perfectly solvable. However, for direct programming, the SPIR-V that is needed for shaders is completely inaccessible. It would be like writing machine code (not assembly) by hand. I assume graphics programmers have some layer on top to provide a sane interface, but since we were writing a compiler anyway, it wasn't a big deal for us.
  - gmueckl 7y ago
    Vulkan compute shaders are roughly on the level of OpenGL compute shaders, which never were as flexible and general as CUDA and OpenCL kernels. Even though compute in Vulkan appears as a first class citizen instead of the weird tacked on feature it is in OpenGL, the design is still focused very much on embedding compute steps in a process that is designed to deliver images to a screen. And that uses a lot more opaque object representations like textures and samplers. GLSL has also always been a very restrictive language and that carried over into SPIR-V.
sorenjan 7y ago
I recently looked into GPGPU programming, and it seems to be a bit of a mess, which is surprising seeing how long it's been around and how much it is used. I think a large part of it is due to Nvidia using their strong position to hamper anything other than Cuda. OpenCL 2.0 was announced in 2013, Nvidia added OpenCL 2.0 "for evaluation purposes" in 2017. I don't think they have final support yet. OpenCL 2.1 uses the same intermediate language as Vulkan, SPIR-V. Nvidia does not seem to support this. When OpenCL 2.2 was announced in 2017 Khronos said they were working on converging OpenCL and Vulkan compute, but OpenCL will remain separate. I think this means that they will both use the same SPIR-V backend to run shader code, but I don't know. Khronos also has SYCL. The latest version, 1.2.1, was released a few days ago and uses OpenCL 1.2 as the backend. I'm guessing it doesn't use a newer version because of Nvidia's poor support for them, but AMD doesn't seem to support it well either. Then there's a multitude of other libraries/platforms, like POCL, HCC, Acoran, that I don't know anything about but found mentions of. AMD has HIP which can convert Cuda to run on AMD hardware, so maybe Cuda is the best option for AMD as well? I find it a bit of an embarrassment for the industry that it's this messy and complicated, and I don't see it getting better in the near future. It seems as if the proprietary Cuda is still the best option, and that's a big failure for everyone other than Nvidia. I don't want to use a proprietary language, but I want the code to run well on all the major platforms. I still don't know what to use. https://www.khronos.org/sycl/ https://www.khronos.org/sycl/ http://portablecl.org/ http://portablecl.org/ https://gpuopen.com/compute-product/hcc-heterogeneous-compute-compiler/ https://gpuopen.com/compute-product/hcc-heterogeneous-comput... https://www.codeplay.com/products/acoran/ https://www.codeplay.com/products/acoran/ https://gpuopen.com/compute-product/hip-convert-cuda-to-portable-c-code/ https://gpuopen.com/compute-product/hip-convert-cuda-to-port...
- dahart 7y ago
  > I don't want to use a proprietary language CUDA is mostly just limited C++. A couple of the interesting points in the release notes: “Added support for CUDA Virtual Memory Management APIs.; 10.2 now includes libcu++, a parallel standard C++ library for GPUs.” Those things make CUDA even easier to treat like regular C++. The language isn’t really the force keeping people in or out, the libraries & tools are. cuDNN, for example, is something you can’t get in OpenCL.
en4bz 7y ago
It's called SYCL and no one uses it. It's been around since 2014.

We need a CUDA alternative for non NVIDIA gpu's, specially on a Mac.

by suyash 7y ago

We need a CUDA alternative for non NVIDIA gpu's, specially on a Mac.

nevi-me 7y ago

Would this from AMD count? https://rocm.github.io/ https://rocm.github.io/

PeterisP 7y ago
It's not supported on Mac, and even on Linux seems to have weird limitations on specific kernel versions. It also supports just a subset of AMD chips, which doesn't seem to include the actual AMD chips of the different Macs that I have available. ROCm would count if it's mature enough so that the setup "just works" on any reasonable environment (in the way that it mostly is so for the major ML platforms on nvidia/CUDA) but it does not yet seem ready for that.
- nevi-me 7y ago
  You make a good point, I forgot that it's not Mac or Windows compatible, and the limited hardware support.

Athas 7y ago

OpenCL is decent, and although it's deprecated, it's probably going to continue working for years. It's not as nice as CUDA for direct programming, but don't most people access CUDA through higher-level libraries anyway? You probably would not be able to tell whether those use OpenCL or CUDA behind the scenes.

fulafel 7y ago
Deprecated by Apple in regards native OS X support for OpenCL, important distinction.

ActorNightly 7y ago

I dont really understand this, why would you use a Mac for anything compute intensive?

mikkelam 7y ago
hackintosh
- vojta_letal 7y ago
  That's such a narrow segment of users. Would it be worth supporting?
  - rob74 7y ago
    Even if it was worth supporting, I guess it's understandable that Apple wouldn't go out of its way to do it...
- deleted 7y ago
  [deleted]

zozbot234 7y ago

AIUI, Vulkan can be used for compute - so why not do that? It might work reasonably well on both nVidia and non-nVidia hardware. Of course, this requires GPU-enabled software to implement Vulkan support, but that's one-time work.

Athas 7y ago
Last year, I supervised a student project that involved writing a Vulkan backend for a compiler that already possessed CUDA and OpenCL backends. While I recall that everything ended up working, it didn't run all that fast. The vast majority of the problems were related to the language used for encoding the shaders, SPIR-V. Among the difficulties we encountered, I recall the need to use an offline optimiser to do trivial things like CSE and constant folding. CUDA and OpenCL will do this during kernel compilation, but Vulkan implementations seems to be designed as non-optimising (in order for loading to be faster, I assume). This is a perfectly understandable design, but it means it's a little more awkward to use. For direct programming (as compared to a compilation target), it's a lot more awkward. Another issue was that basic Vulkan is very restricted, as I assume it's supposed to be usable on simple hardware. Additional functionality can be enabled through a collection of extensions. One particular problem I recall is that our code generator assumes the existence of general pointers, for example such that the same memory can be used for different types of values at different times. SPIR-V has/had a very strict notion of pointers, and no way to cast between different types without an extension, and that extension was IIRC not implemented by NVIDIA or AMD. There were lots of these kinds of issues. In the end, Vulkan is usable for compute - after all, a single master's student (admittedly one who's very bright) managed to implement a Vulkan-targeting compiler backend in half a year. However, at the time I concluded that it's not as mature or as practical as CUDA or OpenCL, for reasons that seem perfectly solvable. However, for direct programming, the SPIR-V that is needed for shaders is completely inaccessible. It would be like writing machine code (not assembly) by hand. I assume graphics programmers have some layer on top to provide a sane interface, but since we were writing a compiler anyway, it wasn't a big deal for us.
- gmueckl 7y ago
  Vulkan compute shaders are roughly on the level of OpenGL compute shaders, which never were as flexible and general as CUDA and OpenCL kernels. Even though compute in Vulkan appears as a first class citizen instead of the weird tacked on feature it is in OpenGL, the design is still focused very much on embedding compute steps in a process that is designed to deliver images to a screen. And that uses a lot more opaque object representations like textures and samplers. GLSL has also always been a very restrictive language and that carried over into SPIR-V.

sorenjan 7y ago

I recently looked into GPGPU programming, and it seems to be a bit of a mess, which is surprising seeing how long it's been around and how much it is used. I think a large part of it is due to Nvidia using their strong position to hamper anything other than Cuda. OpenCL 2.0 was announced in 2013, Nvidia added OpenCL 2.0 "for evaluation purposes" in 2017. I don't think they have final support yet. OpenCL 2.1 uses the same intermediate language as Vulkan, SPIR-V. Nvidia does not seem to support this. When OpenCL 2.2 was announced in 2017 Khronos said they were working on converging OpenCL and Vulkan compute, but OpenCL will remain separate. I think this means that they will both use the same SPIR-V backend to run shader code, but I don't know. Khronos also has SYCL. The latest version, 1.2.1, was released a few days ago and uses OpenCL 1.2 as the backend. I'm guessing it doesn't use a newer version because of Nvidia's poor support for them, but AMD doesn't seem to support it well either. Then there's a multitude of other libraries/platforms, like POCL, HCC, Acoran, that I don't know anything about but found mentions of. AMD has HIP which can convert Cuda to run on AMD hardware, so maybe Cuda is the best option for AMD as well? I find it a bit of an embarrassment for the industry that it's this messy and complicated, and I don't see it getting better in the near future. It seems as if the proprietary Cuda is still the best option, and that's a big failure for everyone other than Nvidia. I don't want to use a proprietary language, but I want the code to run well on all the major platforms. I still don't know what to use. https://www.khronos.org/sycl/ https://www.khronos.org/sycl/ http://portablecl.org/ http://portablecl.org/ https://gpuopen.com/compute-product/hcc-heterogeneous-compute-compiler/ https://gpuopen.com/compute-product/hcc-heterogeneous-comput... https://www.codeplay.com/products/acoran/ https://www.codeplay.com/products/acoran/ https://gpuopen.com/compute-product/hip-convert-cuda-to-portable-c-code/ https://gpuopen.com/compute-product/hip-convert-cuda-to-port...

dahart 7y ago
> I don't want to use a proprietary language CUDA is mostly just limited C++. A couple of the interesting points in the release notes: “Added support for CUDA Virtual Memory Management APIs.; 10.2 now includes libcu++, a parallel standard C++ library for GPUs.” Those things make CUDA even easier to treat like regular C++. The language isn’t really the force keeping people in or out, the libraries & tools are. cuDNN, for example, is something you can’t get in OpenCL.

en4bz 7y ago

It's called SYCL and no one uses it. It's been around since 2014.