Thanks for the link. I did have a quick look at the github repo yesterday, trying to find where this "default pytorch implementation" lives, but it wasn't immed

Thanks for the link. I did have a quick look at the github repo yesterday, trying to find where this "default pytorch implementation" lives, but it wasn't immediately apparent due to the layers of abstraction in the code. There's little doubt in my mind that the authors of the blog post went after the largest performance delta between their best AI-generated code and the default pytorch implementation, without looking into real-world nuances like "picking a good kernel out of a zillion options" or "production-quality output accuracy". It's cool that this sort of thing is being researched, but the results need to be taken with a grain of salt. Maybe I'm too jaded.