4 ms·

Note that this is from 2022. My guess is that people nowadays are gradually moving away from raw CUDA programming and moving towards things like Triton etc, an

Note that this is from 2022. My guess is that people nowadays are gradually moving away from raw CUDA programming and moving towards things like Triton etc, and you won't be focusing on pure GEMM since you tend to do some fusion. The Triton tutorial claims their performance is on par with cuBLAS. https://triton-lang.org/main/getting-started/tutorials/03-matrix-multiplication.html https://triton-lang.org/main/getting-started/tutorials/03-ma...

machinekob 2y ago
But Triton is just abstraction over CUDA for Python (same like cupy, numba etc.). If you need low lvl access you still will use CUDA if you want high level you can use Triton or numba even higer you'll just use wrappers like pytorch/jax.
bee_rider 2y ago
I’m sure they are using cool super-modern stuff but I wonder if Eigen can somehow be made to spit out CUDA code?
- lights0123 2y ago
  Since 2016, you can use Eigen in CUDA C++ code and it just works: https://eigen.tuxfamily.org/dox/TopicCUDA.html https://eigen.tuxfamily.org/dox/TopicCUDA.html
  - bee_rider 2y ago
    Oh wow, that is cool as heck.
imjonse 2y ago
It is still an excellent article if you care about how the GPU works. Triton doesn't magically hide all the hardware details.
dang 2y ago
Year added above. Thanks!
almostgotcaught 2y ago
> My guess is that people nowadays are gradually moving away from raw CUDA programming and moving towards things like Triton etc Your guess is wrong. Besides the fact that there's much more to life than matmul (for which triton is just ok), the other obvious fact is that triton has exactly 1 frontend (python) and there's much more to life than that frontend. I find that basically in every thread about low-level work there's someone making some weird comment about how triton or mojo or XYZ supplants CUDA or assembly or whatever. I can't understand how this comes about because absolutely no one working in these areas thinks XYZ is going to supplant anything. So it's invariably outsiders making these claims and I cannot fathom why any outsider would be motivated to make claims from the outside.
- the_svd_doctor 2y ago
  FWIW Triton can actually be used pretty easily from C++ straight using MLIR. XLA does that. Maybe not exactly a frontend but not that far.
  - almostgotcaught 2y ago
    > FWIW Triton i have PRs in Triton - i'm well familiar with the fact that triton is an MLIR project. > C++ straight using MLIR that's like saying llvm ir is usable through C++ ... or hell that's like saying NVPTX is usable through C++. it's not just not a frontend it's the exact opposite: it's emitting IR using IR builders.
- flakiness 2y ago
  Great comment! My wrong comment at least served as a MacDonal's theory https://jonbell.medium.com/mcdonalds-theory-9216e1c9da7d https://jonbell.medium.com/mcdonalds-theory-9216e1c9da7d As an outsider CUDA is so intimidating so the promise of Triton etc is very appearing and I wanted to get sold.

machinekob 2y ago

But Triton is just abstraction over CUDA for Python (same like cupy, numba etc.). If you need low lvl access you still will use CUDA if you want high level you can use Triton or numba even higer you'll just use wrappers like pytorch/jax.

bee_rider 2y ago

I’m sure they are using cool super-modern stuff but I wonder if Eigen can somehow be made to spit out CUDA code?

lights0123 2y ago
Since 2016, you can use Eigen in CUDA C++ code and it just works: https://eigen.tuxfamily.org/dox/TopicCUDA.html https://eigen.tuxfamily.org/dox/TopicCUDA.html
- bee_rider 2y ago
  Oh wow, that is cool as heck.

imjonse 2y ago

It is still an excellent article if you care about how the GPU works. Triton doesn't magically hide all the hardware details.

dang 2y ago

Year added above. Thanks!

almostgotcaught 2y ago

> My guess is that people nowadays are gradually moving away from raw CUDA programming and moving towards things like Triton etc Your guess is wrong. Besides the fact that there's much more to life than matmul (for which triton is just ok), the other obvious fact is that triton has exactly 1 frontend (python) and there's much more to life than that frontend. I find that basically in every thread about low-level work there's someone making some weird comment about how triton or mojo or XYZ supplants CUDA or assembly or whatever. I can't understand how this comes about because absolutely no one working in these areas thinks XYZ is going to supplant anything. So it's invariably outsiders making these claims and I cannot fathom why any outsider would be motivated to make claims from the outside.

the_svd_doctor 2y ago
FWIW Triton can actually be used pretty easily from C++ straight using MLIR. XLA does that. Maybe not exactly a frontend but not that far.
- almostgotcaught 2y ago
  > FWIW Triton i have PRs in Triton - i'm well familiar with the fact that triton is an MLIR project. > C++ straight using MLIR that's like saying llvm ir is usable through C++ ... or hell that's like saying NVPTX is usable through C++. it's not just not a frontend it's the exact opposite: it's emitting IR using IR builders.
flakiness 2y ago
Great comment! My wrong comment at least served as a MacDonal's theory https://jonbell.medium.com/mcdonalds-theory-9216e1c9da7d https://jonbell.medium.com/mcdonalds-theory-9216e1c9da7d As an outsider CUDA is so intimidating so the promise of Triton etc is very appearing and I wanted to get sold.