The efficacy of neural networks really boil down to the efficacy of linear operations, which in turn – I suspect – are efficacious because all smooth functions

The efficacy of neural networks really boil down to the efficacy of linear operations, which in turn – I suspect – are efficacious because all smooth functions are linear when you look closely enough. That might help with the intuition for why neural networks seem to represent such a fundamentally useful operation. I'm the wrong person to speculate about the future.