3 ms·
>You don't need non-linearities, an infinite series of sine functions is enough The sine function is nonlinear.
by meindnoch 4y ago
>You don't need non-linearities, an infinite series of sine functions is enough
The sine function is nonlinear.
- somewhereoutth 4y agoAh so it is! I guess my point was that even this simple function is enough, you don't need any 'magic' beyond that. In particular, this continuous function can be used (infinitely) to represent discontinuous functions - the step function being the usual example. That is the more interesting and relevant mathematical fact I think.
- meindnoch 4y agoWell, yes. The 'magic' is the nonlinearity. It's because the composition of linear (actually also affine) functions is still just linear (affine). You don't get any additional power by combining many of them - which is also the reason why linear functions are so well understood and easy to work with. You give sprinkle in a tiny nonlinearity (e.g. x^2 instead of x) and suddenly you can get infinite complexity by weighted composition - which is also the reason why we're so helpless with nonlinear functions and reach for linear approximations immediately (cf. gradient descent).
- imtringued 4y ago>It's because the composition of linear (actually also affine) functions is still just linear (affine). Except you can "compose" affine functions using Horner's schema to get any polynomial... No need to sprinkle tiny non linearities. It's a buffet. Grab as much as you need.
- dmd 4y agoSo is IEEE floating point https://www.youtube.com/watch?v=Ae9EKCyI1xU https://www.youtube.com/watch?v=Ae9EKCyI1xU