I feel like a lot of performance could still be extracted from Python libs everyone is using. I.e. NumPy should probably use MKL on Intel out of the box - an ea

I feel like a lot of performance could still be extracted from Python libs everyone is using. I.e. NumPy should probably use MKL on Intel out of the box - an easy 2x improvement in performance. On dense linear algebra MKL is pretty impressive. All the stuff that has to happen in Python will be slow as molasses, though, no question there. Parallelization is also currently much harder than it needs to be. I have a bit of code that does elastic deformation on images. It uses a single core for whatever reason (it does call out into numpy and scipy, obviously, this is not done in Python). All my options for parallelizing this suck mightily. I might end up doing exactly what you're opposed to: drop it down to C++ and use pybind, because currently this stuff is single handedly responsible for quadrupling the duration of training runs, which weren't super fast to begin with.