Universal models of computations are still distinguishable by their complexity classes. Every universal model can simulate the others but these simulations may

Universal models of computations are still distinguishable by their complexity classes. Every universal model can simulate the others but these simulations may not be practical. The whole insane teraflop showoff of the big players of NN is fun and all (saddening actually) but at some point everyone has a limit. Then you have to think about how to get qualitative improvement, and since ML basically boils down to optimizing in computable function spaces just take a better computation model. If you're keen on gradient descent take some language of computable derivable functions (there are richer ones than finite space linear functions + some primitives). Cf Conal Elliott talking about how getting stuck on the idea of matrix operation graph is completely stupid (among other things): "A Functional Reboot for Deep Learning".