3 ms·
Agreed. NFL is general and we need to discuss a specific landscape. Deep learning is continuous <nonlinear> multidimensional optimization - yes. What defines
by viewtransform 9y ago
Agreed. NFL is general and we need to discuss a specific landscape.
Deep learning is continuous <nonlinear> multidimensional optimization - yes.
What defines the subset of problems ? a general nonlinear mapping from R^n to R^m n>m ?
or are you limiting to image classification ? speech recognition ? which would be a subset.
We have empirical evidence that deep-learning works but I'm not confident that we have the mathematical tools to understand why.
- aoeusnth1 9y agohttps://arxiv.org/abs/1710.05468 https://arxiv.org/abs/1710.05468 was an interesting paper that came out recently. It showed that large CNN models which have far greater capacity than the data they are shown (and could have memorized it) still tend to learn very good generalizable minima. See proposition 1: (i) For any model class F whose model complexity is large enough to memorize any dataset and which includes f∗ possibly at an arbitrarily sharp minimum, there exists (A, Sm) such that the generalization gap is at most epsilon, and (ii) For any dataset Sm, there exist arbitrarily unstable and arbitrarily non-robust algorithms A such that the generalization gap of f_A(Sm) is at most epsilon.