3 ms·
> Thus, I wonder what makes this AI-generated kernel faster than the standard pyTorch kernel All of this stuff is way outside my wheelhouse, but maybe "the sta
by zahlman 1y ago
> Thus, I wonder what makes this AI-generated kernel faster than the standard pyTorch kernel
All of this stuff is way outside my wheelhouse, but maybe "the standard pyTorch kernel" is just a low bar? (https://news.ycombinator.com/item?id=44144346 https://news.ycombinator.com/item?id=44144346)
- david-gpu 1y agoThanks for the link. I did have a quick look at the github repo yesterday, trying to find where this "default pytorch implementation" lives, but it wasn't immediately apparent due to the layers of abstraction in the code. There's little doubt in my mind that the authors of the blog post went after the largest performance delta between their best AI-generated code and the default pytorch implementation, without looking into real-world nuances like "picking a good kernel out of a zillion options" or "production-quality output accuracy". It's cool that this sort of thing is being researched, but the results need to be taken with a grain of salt. Maybe I'm too jaded.