2 ms·
Yes, the claims are pretty much in the same spirit. Although the first (Weierstrass's) theorem [1] was stated for real-valued functions in a 1-D closed interva
by xtacy 7y ago
Yes, the claims are pretty much in the same spirit. Although the first (Weierstrass's) theorem [1] was stated for real-valued functions in a 1-D closed interval [a, b], Stone-Weirstrass is a generalisation of the above theorem [2] that's applicable in more general scenarios. Here is the formal statement:
- [1] http://mathworld.wolfram.com/WeierstrassApproximationTheorem.html http://mathworld.wolfram.com/WeierstrassApproximationTheorem...
- [2] http://mathworld.wolfram.com/Stone-WeierstrassTheorem.html http://mathworld.wolfram.com/Stone-WeierstrassTheorem.html
Neural Networks use a different "basis" (sigmoid, ReLU, etc.), but the underlying idea shares the same spirit.
- mturmon 7y agoYes. A NN consisting of, for example, sigmoids will form an “algebra” that separates points in the sense of the Stone Weierstrass theorem, and the NN approximation result will follow. All that’s really needed is that the limit of the NN basis function is different at plus versus minus infinity on the real line. This will give you the “separates points “ property.
- Certhas 7y agoThere are many such ways to approximate somewhat arbitrary functions. Reproducing Kernel Hilbert Spaces for example. The fact that NN can reproduce arbitrary functions is decidedly not what makes them special...