My claim is pretty subjective, but the idea is that there aren't many distinct kernels used in machine learning. It's all tensor contractions and element-wise

My claim is pretty subjective, but the idea is that there aren't many distinct kernels used in machine learning. It's all tensor contractions and element-wise operations. I'd argue that this can be maintained by hand without need for automation or high level abstraction. Triton is used in a templated way for a very specific albeit pervasive hardware (PTX compatible GPUs), which is why it works so well. Here's some of the code: https://github.com/pytorch/pytorch/blob/a66625da3bcdf1e262ddf7bc7fedaa070147c78e/torch/_inductor/kernel/mm.py#L16-L71 https://github.com/pytorch/pytorch/blob/a66625da3bcdf1e262dd... Generalized kernel generation (i.e. synthesis of optimal performance from non-expert user defined kernels and novel hardware) would be fantastic to have, but it just doesn't seem particularly necessary in the field.