3 ms·
What the author means by "close approximation" is clarified later (under the "Two caveats" header). The point is that you can get an arbitrarily close approxima
by throwaway39202 11y ago
What the author means by "close approximation" is clarified later (under the "Two caveats" header). The point is that you can get an arbitrarily close approximation of any continuous function, just by choosing a sufficiently large number of hidden units. That is, for any epsilon > 0, a neural network can approximate any function within epsilon of the function's exact value at all points, with enough hidden units. The smaller the epsilon, the better the approximation you want, and the more hidden units you need.
- TheOtherHobbes 11y agoWhat if you're not approximating a continuous function?
- leni536 11y agoI think it's mostly accurate in L2 norm and not point by point. So every function in L2 can be approximated arbitrarily in L2 norm.
- chestervonwinch 11y agoThis is not true. The theorem proves point-wise convergence for a continuous functions on a compact subset of R^n. See theorem 2, page 6: http://deeplearning.cs.cmu.edu/pdfs/Cybenko.pdf http://deeplearning.cs.cmu.edu/pdfs/Cybenko.pdf
- chestervonwinch 11y agoWell, compact continuous functions are dense in L1, so if you're discontinuous but at least integrable, you probably will still get a nice approximation. Take a box function for instance. Strictly speaking this is not continuous, but it's easy to see it can be approximated arbitrarily well by a couple of sigmoids.