3 ms·
What results are there that show that deep NNs fundamentally cannot learn to tightly approximate some function or class of functions? (Honest question.) Outside
by Sharlin 2y ago
What results are there that show that deep NNs fundamentally cannot learn to tightly approximate some function or class of functions? (Honest question.) Outside pathological nowhere-differentiable cases, of course.
- eviks 2y agoI'm not sure it makes sense to talk in such absolutes (requiring a proof of impossibility) in such a fluid category - after all, you could include the imaginary perfect replica of the human brain in the NN category, and obviously human brains can do this task. But then again, this categorization provides no information to make any assessment of whether that is in any way achievable.
- Sharlin 2y agoSure. Not in the sense of "here’s a proof that a perceptron cannot learn XOR", but at least something that would hint that there are qualitative gaps in the expressive power of DLNNs, or even LLMs specifically.