3 ms·

I would think that a tutorial on convolutions this extensive would mention the convolution theorem. Definitely the easiest way to visualize for most application

I would think that a tutorial on convolutions this extensive would mention the convolution theorem. Definitely the easiest way to visualize for most applications. Or is Fourier space not assumed to be known by the reader?

king_magic 8y ago
I had a similar thought the first time I read this a few years back. I mean, I get what the author is trying to say here (e.g. the ball dropping/rolling probability thing), but it’s just a little more convoluted (ouch) than it needs to be, IMO.
jsharf 8y ago
> Fourier space not assumed to be known by the reader This sounds like a big assumption. I think this is a deep-learning oriented blog post, and not for people with a background in systems & signals (basically information theory). I don't understand mathematical convolutions super well (I know the vague derivation, of integration of f(t)g(t-T)dT, and the metaphor of multiplication in frequency space), but I do get how convolutions work with neural networks, and maybe this is a failure on my part, but it doesn't feel like there's a super strong connection between the two. Is there something I'm missing out on?
slavik81 8y ago
The introduction to convolution via signals and systems is definitely simpler. You can basically introduce the Dirac delta (impulse), introduce the idea of the impulse response, and then point out that every function is just a series of impulses scaled by the function value. Then convolution is just the sum of the impulse responses. (This description is probably leaving mathematicians quietly screaming, but it's just a short skip from that understanding to a more formal one.) I found the article a bit roundabout. Perhaps I'm not part of the intended audience; I understand convolutions, but have only limited familiarity with the internals of neural networks.

3 ms·

I would think that a tutorial on convolutions this extensive would mention the convolution theorem. Definitely the easiest way to visualize for most application

by vortico 8y ago

king_magic 8y ago
I had a similar thought the first time I read this a few years back. I mean, I get what the author is trying to say here (e.g. the ball dropping/rolling probability thing), but it’s just a little more convoluted (ouch) than it needs to be, IMO.
jsharf 8y ago
> Fourier space not assumed to be known by the reader This sounds like a big assumption. I think this is a deep-learning oriented blog post, and not for people with a background in systems & signals (basically information theory). I don't understand mathematical convolutions super well (I know the vague derivation, of integration of f(t)g(t-T)dT, and the metaphor of multiplication in frequency space), but I do get how convolutions work with neural networks, and maybe this is a failure on my part, but it doesn't feel like there's a super strong connection between the two. Is there something I'm missing out on?
slavik81 8y ago
The introduction to convolution via signals and systems is definitely simpler. You can basically introduce the Dirac delta (impulse), introduce the idea of the impulse response, and then point out that every function is just a series of impulses scaled by the function value. Then convolution is just the sum of the impulse responses. (This description is probably leaving mathematicians quietly screaming, but it's just a short skip from that understanding to a more formal one.) I found the article a bit roundabout. Perhaps I'm not part of the intended audience; I understand convolutions, but have only limited familiarity with the internals of neural networks.