I don't see how it _can_ improve anything by anywhere near the claimed amounts. Most practical work is basically dot product, which has been so heavily hand-opt

I don't see how it _can_ improve anything by anywhere near the claimed amounts. Most practical work is basically dot product, which has been so heavily hand-optimized (and then some - via Fourier/Winograd for convolutions), there's no blood to squeeze out from that particular stone. Kernels of this quality can not be arrived at by the compiler - a competent human always beats the compiler when she has to. As a result, your optimization efforts will necessarily have to improve things elsewhere, where a competently written deep learning framework spends single digit percentages of its inference time. As a result of that, in turn, the best you can really hope for is single digit percentage gains, barring some exotic architecture which doesn't consist almost entirely of dot products or some degenerate cases where hand-coded convolution and dot product kernels suck, which are, by definition, uncommon. I mean it's great to have this - a rigorous, testable, theoretically grounded approach to execution graph manipulation is a hard thing to achieve for something as general as this, and everyone will benefit from not rolling everything from scratch every time, but let's be realistic here - this is not going to "fix" Moore's law or anything like that.