4 ms·

> And we have CUDA which is an attempt to, yes, get back to the C model CUDA allows different programming models, it's the point of having the PTX virtual mach

by my123 5y ago

> And we have CUDA which is an attempt to, yes, get back to the C model CUDA allows different programming models, it's the point of having the PTX virtual machine as the exposed programming model. CUDA C++ is popular, but is far from the only option people have. (and it can go very deep, https://www.ibm.com/docs/en/sdk-java-technology/8?topic=egpulwo-how-jit-compiler-uses-gpu-linux-windows-only https://www.ibm.com/docs/en/sdk-java-technology/8?topic=egpu... for example)

paulmd 5y ago
It's far less popular than the C/C++ bindings (at least at an end-user level), but NVIDIA actually has official support for FORTRAN on CUDA as well. iirc you can mix and match at the linker level, and there's a fair amount of library code that uses FORTRAN for performance reasons. FORTRAN has tighter restrictions around what you can do in terms of aliasing/etc, which allows compilers to be more aggressive about optimizations without as much undefined-behavior "this looks correct and will run fine until we add another optimization and it doesn't" shenanigans. So in practice it can be faster than C, and it's actually quite popular among the HPC community and other performance-sensitive segments. It's not like C has to be this way, with all the "undefined behavior" nonsense, it just is an artifact of a programming language written in the early 70s. C makes a lot of sense when a compiler looks a lot more like an assembler with a bit of optimization sprinkled in, and isn't performing super deep introspection and deciding that this entire function can be optimized away because of some arcane "undefined behavior" rule.
- my123 5y ago
  > at least at an end-user level I'd say that the most popular option at the end-user level today is a whole level of abstraction above: using Python w/ PyTorch (or less frequently, TensorFlow). Combined with CuPy and cuNumeric, which is available at https://developer.nvidia.com/cunumeric https://developer.nvidia.com/cunumeric and handles scaling to multiple machines with Python-written code, through the Legate runtime (https://nv-legate.github.io/legate.core/README.html https://nv-legate.github.io/legate.core/README.html). This covers a huge amount of what the CUDA userbase uses.

by my123 5y ago

paulmd 5y ago

It's far less popular than the C/C++ bindings (at least at an end-user level), but NVIDIA actually has official support for FORTRAN on CUDA as well. iirc you can mix and match at the linker level, and there's a fair amount of library code that uses FORTRAN for performance reasons. FORTRAN has tighter restrictions around what you can do in terms of aliasing/etc, which allows compilers to be more aggressive about optimizations without as much undefined-behavior "this looks correct and will run fine until we add another optimization and it doesn't" shenanigans. So in practice it can be faster than C, and it's actually quite popular among the HPC community and other performance-sensitive segments. It's not like C has to be this way, with all the "undefined behavior" nonsense, it just is an artifact of a programming language written in the early 70s. C makes a lot of sense when a compiler looks a lot more like an assembler with a bit of optimization sprinkled in, and isn't performing super deep introspection and deciding that this entire function can be optimized away because of some arcane "undefined behavior" rule.

my123 5y ago
> at least at an end-user level I'd say that the most popular option at the end-user level today is a whole level of abstraction above: using Python w/ PyTorch (or less frequently, TensorFlow). Combined with CuPy and cuNumeric, which is available at https://developer.nvidia.com/cunumeric https://developer.nvidia.com/cunumeric and handles scaling to multiple machines with Python-written code, through the Legate runtime (https://nv-legate.github.io/legate.core/README.html https://nv-legate.github.io/legate.core/README.html). This covers a huge amount of what the CUDA userbase uses.