3 ms·
It'd be curious to see how those AI generated kernels compare to kernels generated by https://github.com/tinygrad/tinygrad https://github.com/tinygrad/tinygrad
by Tiberium 1y ago
It'd be curious to see how those AI generated kernels compare to kernels generated by https://github.com/tinygrad/tinygrad https://github.com/tinygrad/tinygrad
- xiphias2 1y agoAs they wrote most of the wins are because of fusion and TimyGrad started to have fusion optimizations in the last few weeks. GeoHot didn't want to make it only FlashAttention specific, he worked on FlashAttenrion being automatically generated by the optimizer. It's going well