3 ms·
> In what sense are neural nets not rigorous? In the sense that for non-trivial applications, we struggle to define what approach is successful and why it's su
by karpierz 6y ago
> In what sense are neural nets not rigorous?
In the sense that for non-trivial applications, we struggle to define what approach is successful and why it's successful without leaning on empirical metrics.
- GregarianChild 6y agoThat's not because of lacking rigour, but because of complexity! You cannot, in general, predict arbitrary program properties without running programs, that's the essence of Rice's theorem [1]. By the famous Universal Approximation Theorems for neural networks, in general, NNs are Turing complete, which precludes general and simple mathematical "silver bullets" that will help you overcome those challenges. [1] https://en.wikipedia.org/wiki/Rice%27s_theorem https://en.wikipedia.org/wiki/Rice%27s_theorem