4 ms·

How does CUDA support any of these (.Net, Java, etc?). It's the first time I hear this claim. There are 3rd party wrappers in Java, .Net, etc. that call CUDA's

How does CUDA support any of these (.Net, Java, etc?). It's the first time I hear this claim. There are 3rd party wrappers in Java, .Net, etc. that call CUDA's C++ API, and that's all. Equivalet APIs exist for OpenCL too...

dogma1138 6y ago
There are Java and C# compilers for CUDA such as JCUDA and http://www.altimesh.com/hybridizer-essentials/ http://www.altimesh.com/hybridizer-essentials/ but the CUDA runtime, libraries and the first party compiler only supports C/C++ and FORTRAN, for Python you need to use something like Numba. Most non C++ frameworks and implementations tho would simply use wrappers and bindings. I also am not aware of any high performance lib for CUDA that wasn’t written in C++.
pjmlp 6y ago
"Hybridizer: High-Performance C# on GPUs" https://developer.nvidia.com/blog/hybridizer-csharp/ https://developer.nvidia.com/blog/hybridizer-csharp/ "Simplifying GPU Access: A Polyglot Binding for GPUs with GraalVM" https://developer.nvidia.com/gtc/2020/video/s21269-vid https://developer.nvidia.com/gtc/2020/video/s21269-vid And then you can browse for products on https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/tesla-product-literature/gpu-applications-catalog.pdf https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent...
- dogma1138 6y ago
  Hybridizer simply creates CUDA C++ code from C# which is then compiled to PTX it also does it for AVX which you can the compile with Intel’s compiler or gcc, it’s not particularly good and you often need to debug the generated CUDA source code yourself, it’s also doesn’t always play well with the CUDA programming model especially its more advanced features. And again it’s a commercial product developed by a 3rd party, whilst someone uses it I wouldn’t even put it as a rounding error when accounting for why CUDA has the market share it has.
  - pjmlp 6y ago
    It is like everyone arguing about C++ for AAA studios, as if everyone was doing Crysis and Fortnight clones, while forgetting the legions of people making money selling A games. Or forgetting the days when games written in C were actually full of inline Assembly. It is still CUDA, regardless if it goes through PTX or CUDA C++ as implementation detail for the high level code.
    - dogma1138 6y ago
      You aren’t seeing the forest for the trees. The market for these secondary implementations is tiny, and that is coming from someone who worked at a company that had CUDA executed from a spreadsheet. The C#/Java et. al isn’t what made CUDA popular nor what would make OneAPI succeed or fail. CUDA became popular because of its architecture, using an intermediate assembly to allow backward and forward compatibility, it had exe Elle to support across the entire NVIDIA GPU stack which means that it could run on everything form bargain bin laptops with the cheapest dGPU to HPC cards. It came with a large library of high performant libraries and yes the C++ programming model is why it was adopted so well by the big players. And even arguably more importantly is that when ML and GPU compute exploded and that wasn’t that long ago NVIDIA from a business perspective was the top dog in town, CUDA could’ve been dog shit but when AMD could barely launch a GPU that could compete with NVIDIA’s mid range for multiple generations it wouldn’t have mattered.
      - dannyw 6y ago
        Yep, CUDA running on literally anything and everything definitely helped its success. So many data scientists, ML engineers, who got into cuda by playing with their gaming GPUs.
        dogma1138 6y ago
        Which is exactly the advantage Intel has over AMD, they aren’t locked to Linux only and they support iGPUs, ROCm is essentially an extension of the Linux display driver stack at this point and barely supports any consumer hardware and most importantly APUs. I would really want to be able to find the people at AMD who are responsible for the ROCm roadmap and ask them WTF were they thinking...
- my123 6y ago
  https://www.ibm.com/support/knowledgecenter/SSYKE2_8.0.0/com.ibm.java.vm.80.doc/docs/gpu_enabling.html https://www.ibm.com/support/knowledgecenter/SSYKE2_8.0.0/com... goes to a level above: "Alternatively you can let the virtual machine (VM) make this decision automatically by setting a system property on the command line. The JIT can also offload certain processing tasks based on performance heuristics." A lot of what ultimately limits GPUs today is that they are connected over a relatively slow bus (PCIe), this will change in the future, allowing smaller and smaller tasks to be offloaded.
my123 6y ago
The CUDA runtime gets as input the PTX intermediate language. The toolkit ships with compilers from C++ and Fortran to NVVM, and provides you documentation about the PTX virtual machine at https://docs.nvidia.com/cuda/parallel-thread-execution/index.html https://docs.nvidia.com/cuda/parallel-thread-execution/index... and about the higher-level NVVM (which compiles down to PTX) at https://docs.nvidia.com/cuda/nvvm-ir-spec/index.html https://docs.nvidia.com/cuda/nvvm-ir-spec/index.html.
- navaati 6y ago
  Oooh, I didn’t know PTX was an intermediate representation and explicitly documented as such, I really thought it was the actual assembly ran by the chips…
  - my123 6y ago
    You can get the GPU-targeted assembly (sometimes called SASS by NVIDIA) through specifically compiling to a given GPU then using nvdisasm, which also has a very terse definition of the underlying instruction set in the docs (https://docs.nvidia.com/cuda/cuda-binary-utilities/index.html#instruction-set-ref https://docs.nvidia.com/cuda/cuda-binary-utilities/index.htm...). But it's one way only, NVIDIA ships a disassembler, but explicitly doesn't ship an assembler.
  - dogma1138 6y ago
    PTX is a virtual ISA and the reason why ROCm is doomed to fail well beyond its horrendous bugs. ROCm produces hardware specific binaries which not only means you need to produce binaries for multiple GPUs but you also don’t have any guarantee for forward compatibility. A CUDA binary from 10 years ago will still run today on modern hardware, ROCm breaks compatibility between minor releases sometimes and it’s often not documented.
The_rationalist 6y ago
https://github.com/NVIDIA/grcuda https://github.com/NVIDIA/grcuda

dogma1138 6y ago

There are Java and C# compilers for CUDA such as JCUDA and http://www.altimesh.com/hybridizer-essentials/ http://www.altimesh.com/hybridizer-essentials/ but the CUDA runtime, libraries and the first party compiler only supports C/C++ and FORTRAN, for Python you need to use something like Numba. Most non C++ frameworks and implementations tho would simply use wrappers and bindings. I also am not aware of any high performance lib for CUDA that wasn’t written in C++.

pjmlp 6y ago

"Hybridizer: High-Performance C# on GPUs" https://developer.nvidia.com/blog/hybridizer-csharp/ https://developer.nvidia.com/blog/hybridizer-csharp/ "Simplifying GPU Access: A Polyglot Binding for GPUs with GraalVM" https://developer.nvidia.com/gtc/2020/video/s21269-vid https://developer.nvidia.com/gtc/2020/video/s21269-vid And then you can browse for products on https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/tesla-product-literature/gpu-applications-catalog.pdf https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent...

dogma1138 6y ago
Hybridizer simply creates CUDA C++ code from C# which is then compiled to PTX it also does it for AVX which you can the compile with Intel’s compiler or gcc, it’s not particularly good and you often need to debug the generated CUDA source code yourself, it’s also doesn’t always play well with the CUDA programming model especially its more advanced features. And again it’s a commercial product developed by a 3rd party, whilst someone uses it I wouldn’t even put it as a rounding error when accounting for why CUDA has the market share it has.
- pjmlp 6y ago
  It is like everyone arguing about C++ for AAA studios, as if everyone was doing Crysis and Fortnight clones, while forgetting the legions of people making money selling A games. Or forgetting the days when games written in C were actually full of inline Assembly. It is still CUDA, regardless if it goes through PTX or CUDA C++ as implementation detail for the high level code.
  - dogma1138 6y ago
    You aren’t seeing the forest for the trees. The market for these secondary implementations is tiny, and that is coming from someone who worked at a company that had CUDA executed from a spreadsheet. The C#/Java et. al isn’t what made CUDA popular nor what would make OneAPI succeed or fail. CUDA became popular because of its architecture, using an intermediate assembly to allow backward and forward compatibility, it had exe Elle to support across the entire NVIDIA GPU stack which means that it could run on everything form bargain bin laptops with the cheapest dGPU to HPC cards. It came with a large library of high performant libraries and yes the C++ programming model is why it was adopted so well by the big players. And even arguably more importantly is that when ML and GPU compute exploded and that wasn’t that long ago NVIDIA from a business perspective was the top dog in town, CUDA could’ve been dog shit but when AMD could barely launch a GPU that could compete with NVIDIA’s mid range for multiple generations it wouldn’t have mattered.
    - dannyw 6y ago
      Yep, CUDA running on literally anything and everything definitely helped its success. So many data scientists, ML engineers, who got into cuda by playing with their gaming GPUs.
      - dogma1138 6y ago
        Which is exactly the advantage Intel has over AMD, they aren’t locked to Linux only and they support iGPUs, ROCm is essentially an extension of the Linux display driver stack at this point and barely supports any consumer hardware and most importantly APUs. I would really want to be able to find the people at AMD who are responsible for the ROCm roadmap and ask them WTF were they thinking...
my123 6y ago
https://www.ibm.com/support/knowledgecenter/SSYKE2_8.0.0/com.ibm.java.vm.80.doc/docs/gpu_enabling.html https://www.ibm.com/support/knowledgecenter/SSYKE2_8.0.0/com... goes to a level above: "Alternatively you can let the virtual machine (VM) make this decision automatically by setting a system property on the command line. The JIT can also offload certain processing tasks based on performance heuristics." A lot of what ultimately limits GPUs today is that they are connected over a relatively slow bus (PCIe), this will change in the future, allowing smaller and smaller tasks to be offloaded.

my123 6y ago

The CUDA runtime gets as input the PTX intermediate language. The toolkit ships with compilers from C++ and Fortran to NVVM, and provides you documentation about the PTX virtual machine at https://docs.nvidia.com/cuda/parallel-thread-execution/index.html https://docs.nvidia.com/cuda/parallel-thread-execution/index... and about the higher-level NVVM (which compiles down to PTX) at https://docs.nvidia.com/cuda/nvvm-ir-spec/index.html https://docs.nvidia.com/cuda/nvvm-ir-spec/index.html.

navaati 6y ago
Oooh, I didn’t know PTX was an intermediate representation and explicitly documented as such, I really thought it was the actual assembly ran by the chips…
- my123 6y ago
  You can get the GPU-targeted assembly (sometimes called SASS by NVIDIA) through specifically compiling to a given GPU then using nvdisasm, which also has a very terse definition of the underlying instruction set in the docs (https://docs.nvidia.com/cuda/cuda-binary-utilities/index.html#instruction-set-ref https://docs.nvidia.com/cuda/cuda-binary-utilities/index.htm...). But it's one way only, NVIDIA ships a disassembler, but explicitly doesn't ship an assembler.
- dogma1138 6y ago
  PTX is a virtual ISA and the reason why ROCm is doomed to fail well beyond its horrendous bugs. ROCm produces hardware specific binaries which not only means you need to produce binaries for multiple GPUs but you also don’t have any guarantee for forward compatibility. A CUDA binary from 10 years ago will still run today on modern hardware, ROCm breaks compatibility between minor releases sometimes and it’s often not documented.

The_rationalist 6y ago

https://github.com/NVIDIA/grcuda https://github.com/NVIDIA/grcuda