4 ms·

I also enjoy Terence Tao's explanation [0]: > I remember as a graduate student that Ingrid Daubechies frequently referred to convolution by a bump function as

I also enjoy Terence Tao's explanation [0]: > I remember as a graduate student that Ingrid Daubechies frequently referred to convolution by a bump function as "blurring" - its effect on images is similar to what a short-sighted person experiences when taking off his or her glasses (and, indeed, if one works through the geometric optics, convolution is not a bad first approximation for this effect). I found this to be very helpful, not just for understanding convolution per se, but as a lesson that one should try to use physical intuition to model mathematical concepts whenever one can. > More generally, if one thinks of functions as fuzzy versions of points, then convolution is the fuzzy version of addition (or sometimes multiplication, depending on the context). The probabilistic interpretation is one example of this (where the fuzz is a a probability distribution), but one can also have signed, complex-valued, or vector-valued fuzz, of course. [0] https://mathoverflow.net/questions/5892/what-is-convolution-intuitively https://mathoverflow.net/questions/5892/what-is-convolution-...

nabla9 6y ago
Tao's explanation seems to be the deepest one so far. Understanding convolution different contexts: geometry, vector algebra, statistics, signal progressing, control theory, functional analysis .. etc. just deepens the understanding. Instead of finding one definition and then seeing everything trough it, one should try to find how different viewpoints are the same and learn to switch.
salty_biscuits 6y ago
I might be wrong (so happy to be corrected by a native speaker) but I liked when I heard that the German word for convolution is "faltung" which translates to "folding". This is a much nicer word to metaphorically understand the operation. Maybe it still confuses the hell out of German undergrads though?
- dasudasu 6y ago
  The etymology is the same in English. To convolve : to roll together : writhe. Convolution : a form or shape that is folded in curved or tortuous windings.
  - ReactiveJelly 6y ago
    That confuses the hell out of me. I first encountered it as a young gamedev hobbyist, doing image post-processing in OpenGL. One kind of kernel makes images blurry, one (attempts to) make them sharp. It doesn't bring folding to mind at all. And in FP, "fold" means "reduce". How much of dyscalculia is just one person thinking an analogy is completely obvious and the other person thinking it makes no sense?
    - salty_biscuits 6y ago
      Probably due to the really idiosyncratic way I think about things and from the post it is not at all clear what I mean, sorry for the imprecision! I think of cross correlation of 1D signals with a kernel as sliding one of the functions over the other [0]. In my head I imagine drawing the signal and kernel next to each other one a big piece of paper and literally folding it over and sliding it back over the signal to intuit the convolution (with an imaginary see through piece of paper). I really don't think that was obvious from what I posted, so definitely not dyscalculia! https://commons.wikimedia.org/wiki/File:Cross_correlation_animation.gif https://commons.wikimedia.org/wiki/File:Cross_correlation_an...
    - enriquto 6y ago
      > One kind of kernel makes images blurry, one (attempts to) make them sharp. Any kernel with positive values will blur the image, and the effect will be quite similar to Gaussian blur (by the Central Limit Theorem). It's kernels with positive and negative coefficients that may "enhance" or "sharpen" the image.
jonahx 6y ago
"if one thinks of functions as fuzzy versions of points" Can anyone elaborate on this?
- peppery 6y ago
  Tao is likely inviting us (just as many physical/probabilistic laws do) to view any arbitrary function as relatively "thicker"/"fuzzier" than an infinitely-thin, infinitely-tall spike function at a certain value: the Dirac delta function (https://en.wikipedia.org/wiki/Dirac_delta_function https://en.wikipedia.org/wiki/Dirac_delta_function). If you convolve ≡ integrate this Dirac delta function (located at some value x) against any function g(t), by construction the integral is zero everywhere except at t=x, so the result is an infinitely thin slice of g at x, exactly g(x) (the 'sifting property,' https://math.stackexchange.com/questions/1015498/convolution-with-delta-function https://math.stackexchange.com/questions/1015498/convolution...). Now imagine you begin to thicken/fuzz the spike; now you begin to accumulate the behavior of g(t) not just exactly at x, but also at points nearby, getting a schmeared representation of g. Deforming our spike to an arbitrary function of interest in this way gives an arbitrary convolution schmear.
  - jonahx 6y ago
    Great explanation, thank you.

nabla9 6y ago

Tao's explanation seems to be the deepest one so far. Understanding convolution different contexts: geometry, vector algebra, statistics, signal progressing, control theory, functional analysis .. etc. just deepens the understanding. Instead of finding one definition and then seeing everything trough it, one should try to find how different viewpoints are the same and learn to switch.

salty_biscuits 6y ago

I might be wrong (so happy to be corrected by a native speaker) but I liked when I heard that the German word for convolution is "faltung" which translates to "folding". This is a much nicer word to metaphorically understand the operation. Maybe it still confuses the hell out of German undergrads though?

dasudasu 6y ago
The etymology is the same in English. To convolve : to roll together : writhe. Convolution : a form or shape that is folded in curved or tortuous windings.
- ReactiveJelly 6y ago
  That confuses the hell out of me. I first encountered it as a young gamedev hobbyist, doing image post-processing in OpenGL. One kind of kernel makes images blurry, one (attempts to) make them sharp. It doesn't bring folding to mind at all. And in FP, "fold" means "reduce". How much of dyscalculia is just one person thinking an analogy is completely obvious and the other person thinking it makes no sense?
  - salty_biscuits 6y ago
    Probably due to the really idiosyncratic way I think about things and from the post it is not at all clear what I mean, sorry for the imprecision! I think of cross correlation of 1D signals with a kernel as sliding one of the functions over the other [0]. In my head I imagine drawing the signal and kernel next to each other one a big piece of paper and literally folding it over and sliding it back over the signal to intuit the convolution (with an imaginary see through piece of paper). I really don't think that was obvious from what I posted, so definitely not dyscalculia! https://commons.wikimedia.org/wiki/File:Cross_correlation_animation.gif https://commons.wikimedia.org/wiki/File:Cross_correlation_an...
  - enriquto 6y ago
    > One kind of kernel makes images blurry, one (attempts to) make them sharp. Any kernel with positive values will blur the image, and the effect will be quite similar to Gaussian blur (by the Central Limit Theorem). It's kernels with positive and negative coefficients that may "enhance" or "sharpen" the image.

jonahx 6y ago

"if one thinks of functions as fuzzy versions of points" Can anyone elaborate on this?

peppery 6y ago
Tao is likely inviting us (just as many physical/probabilistic laws do) to view any arbitrary function as relatively "thicker"/"fuzzier" than an infinitely-thin, infinitely-tall spike function at a certain value: the Dirac delta function (https://en.wikipedia.org/wiki/Dirac_delta_function https://en.wikipedia.org/wiki/Dirac_delta_function). If you convolve ≡ integrate this Dirac delta function (located at some value x) against any function g(t), by construction the integral is zero everywhere except at t=x, so the result is an infinitely thin slice of g at x, exactly g(x) (the 'sifting property,' https://math.stackexchange.com/questions/1015498/convolution-with-delta-function https://math.stackexchange.com/questions/1015498/convolution...). Now imagine you begin to thicken/fuzz the spike; now you begin to accumulate the behavior of g(t) not just exactly at x, but also at points nearby, getting a schmeared representation of g. Deforming our spike to an arbitrary function of interest in this way gives an arbitrary convolution schmear.
- jonahx 6y ago
  Great explanation, thank you.