4 ms·

Something that blew my mind the first time I learned it: You can think of a function f(x) as the limit of an infinitely big vector where the entries index the

Something that blew my mind the first time I learned it: You can think of a function f(x) as the limit of an infinitely big vector where the entries index the infinitesimal. Eg f(x) = [...f(-2dx), f(-dx), f(0), f(dx), f(2dx)...]. The dot product of two functions (f,g) is still the usual sum[f(x)*g(x)] but the sum is replaced with an integral. Sin and Cos of integer frequencies happen to have a dot product of 0 (check it) which means doing a change of basis into those functions (aka the Fourier transform) happens to work out really nicely. Other than that Sin and Cos are not really privileged. For example you can do a transform into the basis of polynomial functions if you wanted to (aka the Taylor series). Any basis you can cook up would do just as well so long as its a complete basis. Just like in linear algebra, you have a complete basis if you can use linear combinations to construct the vectors ... [...1,0,0...], [...0,1,0...], [...0,0,1...] ... (aka the dirac delta functions). Differentiation is a matrix with dx on the off diagonal and -dx on the diagonal.

laretluval 7y ago
What should I google to read more about this kind of thing?
- mlevental 7y ago
  https://en.wikipedia.org/wiki/Functional_analysis https://en.wikipedia.org/wiki/Functional_analysis
- liuyao 7y ago
  These kind of heuristics are supposedly a prerequisite to the formal study, but sadly most textbooks don't spend the time talking about it (since logically it's not necessary). Instead of reading up on functional analysis (Banach spaces and Hilbert spaces), read bits of its history to get a taste. Here's one that's not too long http://courses.mai.liu.se/GU/TATM85/FA-history.pdf http://courses.mai.liu.se/GU/TATM85/FA-history.pdf
ska 7y ago
It's a bit weirder than this suggests. You are right as long as you have a complete basis you can transform code your function by a weighted sum of bases vectors, where the weights are inner products of the function against he dual vectors of that basis, and the basis vector form a complete orthonormal basis (we can relax that last bit). So that's great, but what do the basis vectors look like? You need a family of functions so that scalings and translations of them have inner product 0 against each other. So this is easy if the shared support is 0, or if the +ve and -ve parts cancel out. It doesn't take too much playing about to see how this works with Haar basis, or sin/cos (i.e. FFT), so it looks pretty intuitive. It also seem kind of intuitive that these might be the only way to do this... but that's wrong. It turns out you can construct other families that work, unlike the nicely behaved sin/cos, or step functions, they are not smooth, not symmetric (at least for orthonormal basis) - quite odd. They don't have closed forms, so if you want to see what one looks like you'll have to realized it as a fixed point or by some other approximation method: e.g. https://en.wikipedia.org/wiki/Daubechies_wavelet#/media/File:Daubechies4-functions.svg https://en.wikipedia.org/wiki/Daubechies_wavelet#/media/File...
jacobolus 7y ago
> For example you can do a transform into the basis of polynomial functions if you wanted to (aka the Taylor series). Any basis you can cook up would do just as well so long as its a complete basis. Just be warned that the polynomial you get depends on where you center your Taylor series. There are multiple different Taylor-series approximations for the same function, and each one only converges within a disk in the complex plane which does not include any poles. If you try to fit the Runge function 1 / (1 + 25x^2) with a Taylor series centered at 0 you’re going to have a bad time. https://www.desmos.com/calculator/uyjyjug07m https://www.desmos.com/calculator/uyjyjug07m

laretluval 7y ago

What should I google to read more about this kind of thing?

mlevental 7y ago
https://en.wikipedia.org/wiki/Functional_analysis https://en.wikipedia.org/wiki/Functional_analysis
liuyao 7y ago
These kind of heuristics are supposedly a prerequisite to the formal study, but sadly most textbooks don't spend the time talking about it (since logically it's not necessary). Instead of reading up on functional analysis (Banach spaces and Hilbert spaces), read bits of its history to get a taste. Here's one that's not too long http://courses.mai.liu.se/GU/TATM85/FA-history.pdf http://courses.mai.liu.se/GU/TATM85/FA-history.pdf

ska 7y ago

It's a bit weirder than this suggests. You are right as long as you have a complete basis you can transform code your function by a weighted sum of bases vectors, where the weights are inner products of the function against he dual vectors of that basis, and the basis vector form a complete orthonormal basis (we can relax that last bit). So that's great, but what do the basis vectors look like? You need a family of functions so that scalings and translations of them have inner product 0 against each other. So this is easy if the shared support is 0, or if the +ve and -ve parts cancel out. It doesn't take too much playing about to see how this works with Haar basis, or sin/cos (i.e. FFT), so it looks pretty intuitive. It also seem kind of intuitive that these might be the only way to do this... but that's wrong. It turns out you can construct other families that work, unlike the nicely behaved sin/cos, or step functions, they are not smooth, not symmetric (at least for orthonormal basis) - quite odd. They don't have closed forms, so if you want to see what one looks like you'll have to realized it as a fixed point or by some other approximation method: e.g. https://en.wikipedia.org/wiki/Daubechies_wavelet#/media/File:Daubechies4-functions.svg https://en.wikipedia.org/wiki/Daubechies_wavelet#/media/File...

jacobolus 7y ago

> For example you can do a transform into the basis of polynomial functions if you wanted to (aka the Taylor series). Any basis you can cook up would do just as well so long as its a complete basis. Just be warned that the polynomial you get depends on where you center your Taylor series. There are multiple different Taylor-series approximations for the same function, and each one only converges within a disk in the complex plane which does not include any poles. If you try to fit the Runge function 1 / (1 + 25x^2) with a Taylor series centered at 0 you’re going to have a bad time. https://www.desmos.com/calculator/uyjyjug07m https://www.desmos.com/calculator/uyjyjug07m