40 ms·
Wait a minute, isn't it also the case that according to the Weierstrass approximation theorem any continuous function on a closed interval can be approximated b
by 13415 8y ago
Wait a minute, isn't it also the case that according to the Weierstrass approximation theorem any continuous function on a closed interval can be approximated by a polynomial function? And isn't that kind of pointless for practical applications because we also need to avoid overfitting?
To clarify, I'm not trying to make a snippy remark, I just happened to have used polynomial curve fitting before and looked up the Wikipedia page for the Stone-Weierstrass theorem and am trying to figure out the relevance of that post on NNs. Is it essentially the same claim?
Any clarification appreciated!
- xtacy 8y agoYes, the claims are pretty much in the same spirit. Although the first (Weierstrass's) theorem [1] was stated for real-valued functions in a 1-D closed interval [a, b], Stone-Weirstrass is a generalisation of the above theorem [2] that's applicable in more general scenarios. Here is the formal statement: - [1] http://mathworld.wolfram.com/WeierstrassApproximationTheorem.html http://mathworld.wolfram.com/WeierstrassApproximationTheorem... - [2] http://mathworld.wolfram.com/Stone-WeierstrassTheorem.html http://mathworld.wolfram.com/Stone-WeierstrassTheorem.html Neural Networks use a different "basis" (sigmoid, ReLU, etc.), but the underlying idea shares the same spirit.
- mturmon 8y agoYes. A NN consisting of, for example, sigmoids will form an “algebra” that separates points in the sense of the Stone Weierstrass theorem, and the NN approximation result will follow. All that’s really needed is that the limit of the NN basis function is different at plus versus minus infinity on the real line. This will give you the “separates points “ property.
- Certhas 8y agoThere are many such ways to approximate somewhat arbitrary functions. Reproducing Kernel Hilbert Spaces for example. The fact that NN can reproduce arbitrary functions is decidedly not what makes them special...
- deleted 8y ago[deleted]