The response to this comment is a bit sad. Yes, "we just needs a sufficiently advanced compiler" is a meme, but 1) not everyone knows that and 2) that's not wh

The response to this comment is a bit sad. Yes, "we just needs a sufficiently advanced compiler" is a meme, but 1) not everyone knows that and 2) that's not what this comment is asking for: > but here one domain specific language can have a huge impact There's a big difference between a sufficiently advanced compiler for general C code, and a compiler for a domain-specific array processing language, for example. Have a look at halide: https://halide-lang.org https://halide-lang.org , where you specify the computation to perform and a schedule (loop order, blocking, vectorisation) separately. There's even an automatic scheduler that can find a good schedule given some example array sizes and parameters. The fftw paper is worth a read, too -- this shows that for a restricted domain it's possible to automatically find code that works as well as or better than hand-tuned algorithms. Generalising this is of course very hard. For deep learning I think a lot of this is already done, see XLA for example.