3 ms·

> which are never, ever written in Python nah never ever ever ever ever ... except https://github.com/FlagOpen/FlagGems https://github.com/FlagOpen/FlagGems

> which are never, ever written in Python nah never ever ever ever ever ... except https://github.com/FlagOpen/FlagGems https://github.com/FlagOpen/FlagGems https://github.com/linkedin/Liger-Kernel https://github.com/linkedin/Liger-Kernel https://github.com/meta-pytorch/applied-ai https://github.com/meta-pytorch/applied-ai https://github.com/gpu-mode/triton-index https://github.com/gpu-mode/triton-index https://github.com/AlibabaPAI/FLASHNN https://github.com/AlibabaPAI/FLASHNN https://github.com/unslothai/unsloth https://github.com/unslothai/unsloth the number of people commenting on this stuff that don't know what they're actually talking about grows by leaps and bounds every day...

Hizonner 1y ago
I stand corrected. I should have known people would be doing that in Python. How many of the world's total FLOPs are going through those?
- almostgotcaught 1y ago
  Triton is a backend for PyTorch. Lately it is the backend. So it's definitely double digits percentage if not over 50%.
  - lairv 1y ago
    It's the backend for torch.compile, pytorch eager mode will still use cuBLAS/cuDNN/custom CUDA kernels, not sure what's the usage of torch.compile
    - almostgotcaught 1y ago
      > not sure what's the usage of torch.compile consider that at minimum both FB and OAI themselves definitely make heavy use of the Triton backend in PyTorch.
  - cavisne 1y ago
    Doesn’t triton write its own intermediate language that then compiles to PTX?
    - almostgotcaught 1y ago
      Yes and?
    - saagarjha 1y ago
      It has a fairly standard MLIR pipeline
- saagarjha 1y ago
  A lot. OpenAI uses Triton for their critical kernels. Meta has torch.compile using it too. I know Anthropic is not using Triton but I think their homegrown compiler is also Python. xAI is using CUTLASS which is C++ but I wouldn't be surprised if they start using the Python API moving forward.
  - almostgotcaught 1y ago
    Anthropic is a Jax shop
    - saagarjha 1y ago
      Surprised they got it working for Tranium

Hizonner 1y ago

I stand corrected. I should have known people would be doing that in Python. How many of the world's total FLOPs are going through those?

almostgotcaught 1y ago
Triton is a backend for PyTorch. Lately it is the backend. So it's definitely double digits percentage if not over 50%.
- lairv 1y ago
  It's the backend for torch.compile, pytorch eager mode will still use cuBLAS/cuDNN/custom CUDA kernels, not sure what's the usage of torch.compile
  - almostgotcaught 1y ago
    > not sure what's the usage of torch.compile consider that at minimum both FB and OAI themselves definitely make heavy use of the Triton backend in PyTorch.
- cavisne 1y ago
  Doesn’t triton write its own intermediate language that then compiles to PTX?
  - almostgotcaught 1y ago
    Yes and?
  - saagarjha 1y ago
    It has a fairly standard MLIR pipeline
saagarjha 1y ago
A lot. OpenAI uses Triton for their critical kernels. Meta has torch.compile using it too. I know Anthropic is not using Triton but I think their homegrown compiler is also Python. xAI is using CUTLASS which is C++ but I wouldn't be surprised if they start using the Python API moving forward.
- almostgotcaught 1y ago
  Anthropic is a Jax shop
  - saagarjha 1y ago
    Surprised they got it working for Tranium