4 ms·

This is very cool and impressive, really good job! Image filtering is a popular subject these days thanks partly to Instagram, and this subject is on the bound

This is very cool and impressive, really good job! Image filtering is a popular subject these days thanks partly to Instagram, and this subject is on the boundary between art and science, which is nice for a change of pace sometimes. Anyway, to master image filtering in photoshop/GIMP for example requires learning a (very) large number of words and concepts which seemingly have no organizational structure. These words/concepts exist for historical reasons, and not because they represent some sort of cohesive or unified approach to filtering in general. The exact same problem exists for simulating effects pedals for guitars in programs like Pro Tools, interestingly. Only Fourier methods can possibly unify this type of filtering, for a number of reasons which are actually obvious to anyone who is familiar with the fundamental role played by these methods in analysis. The problem seems to be that the people who actually use this stuff all the time do not know Fourier methods and are not in a position to learn it; and this has created a massive inertia against re-formulating the entire UI/UX for image filtering based on the only possible unified approach. It might be thought that Fourier methods are too hard for a graphic designer to understand, but I believe that it is up to people like us who do understand these things to redo the entire approach to filtering from square one, based on Fourier methods. In fact, I believe such an approach will make this subject much more accessible to everyone, and will be significantly easier for power users as well.

sharpneli 12y ago
"It might be thought that Fourier methods are too hard for a graphic designer to understand" I find this argument funny in general. Just look at sound engineers working with the mixboard and their equalizers. Equalizer basically allows you to do the same as what is done in that webpage. And yet the persons using it won't necessarily even know what the term "Fourier transform" means. An useful UI so artists can play with it is probably the only thing needed for this.
- calhoun137 12y ago
  Since you used the example of sound engineer's, I would like to expand on the point I made in my previous comment about pro tools; and also clarify that I believe sound engineer's and graphic designers are a smart bunch, and are more than capable of handling it. I actually recently went to a studio and worked with a professional sound engineer. Most of the time, he would right click on a track, and then select from a list of filters to apply to some segment. These filters have a GUI that looks like an effects pedal with knobs etc... Having used many of these programs myself, I can assure you virtually nothing is based on fourier methods, even though this is all that these effects pedals are doing, often with large amounts of redundancy. A curious fact is that a large majority of the popular types of effects pedals actually correspond to the most well studied convolutions, which is something I hope and plan to research more carefully and write up in a blog post one day soon. Effects pedals for guitars were originally circuits discovered basically by chance; and it was only much later that these things were made digital. This clearly demonstrates the problem I was talking about when I said there is a "massive inertia against re-formulating the entire UI/UX for ... filtering based on the only possible unified approach"
  - tfinniga 12y ago
    A convolution in image space is a component-wise multiplication in fourier space. As the size of your convolution kernel increases, sometimes it is more efficient to convert the kernel to fourier space, multiply pixel by pixel, and then convert back. This is what's going on with the presets at the bottom of the article.. for example, if you took the gaussian blur filter and converted it to image space, you'd have a gaussian blur convolution kernel.
    - smorrow 12y ago
      > [...] convolution [...] component-wise multiplication in fourier space [...] convolution kernel [...] gaussian blur filter [...] Can I ask where people learn this stuff? Programming computers is pretty easy to learn by yourself, but the more I look at technical subjects other than programming computers, the less I can understand how it's possible to learn that stuff other than someone showing you.
      - sharpneli 12y ago
        School. Books. Or the way I did it: Access to Matlab while in University. I learned signal processing just like I learned to program, by playing around and hacking stuff together. Naturally no-one learns anything in isolation. So bunch of reading is needed regardless of what you do.
      - tfinniga 12y ago
        Learned this in my image/signal processing class at university. Most of the terms are just fancy / specific ways of saying simple things. There are a few key concepts to learn, and then the rest is implementation and learning the vocabulary so you can discuss it unambiguously.
      - tripzilch 12y ago
        Many many years ago, a fellow demoscener pointed me to this: Yehar's DSP Tutorial for the Braindead http://yehar.com/blog/?p=121 http://yehar.com/blog/?p=121 It skips over some of the more fundamental math, but it serves as a great introduction to general ideas of digital signal processing. You can then proceed to look up the fiddly bits and deeper theory in more detail as you see fit (start from Wikipedia, hit the external links--that's how I'd do it). This particular tutorial speaks mainly about digital audio signal processing, which is the 1D case, but it generalizes to 2D image processing just as easily, if you've got the imagination (although you need some tricks if you don't want an IIR filter to look weird on an image).
icegreentea 12y ago
I think a lot of the simpler filtering is understood just fine in spatial domain. It's just that a lot of the filtering isn't even taught in that context either. Hell, a lot of what is called 'filtering' in the instagram sense doesn't even touch frequency domain - they're global manipulation of color balance, contrast and levels. There's certainly space for improvement - it would certainly be nice to expose the underlying frequency domain components when manipulating filters, but I really think that you've overstated the level of lack of cohesion in Photoshop terminology, especially as a case for major UI changes. And certainly, anyone who -has- knowledge of Fourier methods will also understand that by its very nature, spatial formulations of filters are every bit as legitimate as frequency domain formulations. Which one you should use should be based on whatever is more convenient. A nice box/circle blur is certainly easier to reason about than its frequency domain counterpart.
frozenport 12y ago
>>Only Fourier methods can possibly unify this type of filtering You are wrong in this. The last 60 years of signal processing history involve developing methods that are improvements on the FT. >>It might be thought that Fourier methods are too hard for a graphic designer to understand Fourier methods are bad because they are the least efficient basis to represent your image. It is not intuitive to increase the gain of the low frequencies when you want to increase something like the shadow. There is no mapping between features in an object and the Fourier domain. More advanced, spatial domain techniques might be appropriate and these already exist and are being used by graphics design professionals.

sharpneli 12y ago

"It might be thought that Fourier methods are too hard for a graphic designer to understand" I find this argument funny in general. Just look at sound engineers working with the mixboard and their equalizers. Equalizer basically allows you to do the same as what is done in that webpage. And yet the persons using it won't necessarily even know what the term "Fourier transform" means. An useful UI so artists can play with it is probably the only thing needed for this.

calhoun137 12y ago
Since you used the example of sound engineer's, I would like to expand on the point I made in my previous comment about pro tools; and also clarify that I believe sound engineer's and graphic designers are a smart bunch, and are more than capable of handling it. I actually recently went to a studio and worked with a professional sound engineer. Most of the time, he would right click on a track, and then select from a list of filters to apply to some segment. These filters have a GUI that looks like an effects pedal with knobs etc... Having used many of these programs myself, I can assure you virtually nothing is based on fourier methods, even though this is all that these effects pedals are doing, often with large amounts of redundancy. A curious fact is that a large majority of the popular types of effects pedals actually correspond to the most well studied convolutions, which is something I hope and plan to research more carefully and write up in a blog post one day soon. Effects pedals for guitars were originally circuits discovered basically by chance; and it was only much later that these things were made digital. This clearly demonstrates the problem I was talking about when I said there is a "massive inertia against re-formulating the entire UI/UX for ... filtering based on the only possible unified approach"
- tfinniga 12y ago
  A convolution in image space is a component-wise multiplication in fourier space. As the size of your convolution kernel increases, sometimes it is more efficient to convert the kernel to fourier space, multiply pixel by pixel, and then convert back. This is what's going on with the presets at the bottom of the article.. for example, if you took the gaussian blur filter and converted it to image space, you'd have a gaussian blur convolution kernel.
  - smorrow 12y ago
    > [...] convolution [...] component-wise multiplication in fourier space [...] convolution kernel [...] gaussian blur filter [...] Can I ask where people learn this stuff? Programming computers is pretty easy to learn by yourself, but the more I look at technical subjects other than programming computers, the less I can understand how it's possible to learn that stuff other than someone showing you.
    - sharpneli 12y ago
      School. Books. Or the way I did it: Access to Matlab while in University. I learned signal processing just like I learned to program, by playing around and hacking stuff together. Naturally no-one learns anything in isolation. So bunch of reading is needed regardless of what you do.
    - tfinniga 12y ago
      Learned this in my image/signal processing class at university. Most of the terms are just fancy / specific ways of saying simple things. There are a few key concepts to learn, and then the rest is implementation and learning the vocabulary so you can discuss it unambiguously.
    - tripzilch 12y ago
      Many many years ago, a fellow demoscener pointed me to this: Yehar's DSP Tutorial for the Braindead http://yehar.com/blog/?p=121 http://yehar.com/blog/?p=121 It skips over some of the more fundamental math, but it serves as a great introduction to general ideas of digital signal processing. You can then proceed to look up the fiddly bits and deeper theory in more detail as you see fit (start from Wikipedia, hit the external links--that's how I'd do it). This particular tutorial speaks mainly about digital audio signal processing, which is the 1D case, but it generalizes to 2D image processing just as easily, if you've got the imagination (although you need some tricks if you don't want an IIR filter to look weird on an image).

icegreentea 12y ago

I think a lot of the simpler filtering is understood just fine in spatial domain. It's just that a lot of the filtering isn't even taught in that context either. Hell, a lot of what is called 'filtering' in the instagram sense doesn't even touch frequency domain - they're global manipulation of color balance, contrast and levels. There's certainly space for improvement - it would certainly be nice to expose the underlying frequency domain components when manipulating filters, but I really think that you've overstated the level of lack of cohesion in Photoshop terminology, especially as a case for major UI changes. And certainly, anyone who -has- knowledge of Fourier methods will also understand that by its very nature, spatial formulations of filters are every bit as legitimate as frequency domain formulations. Which one you should use should be based on whatever is more convenient. A nice box/circle blur is certainly easier to reason about than its frequency domain counterpart.

frozenport 12y ago

>>Only Fourier methods can possibly unify this type of filtering You are wrong in this. The last 60 years of signal processing history involve developing methods that are improvements on the FT. >>It might be thought that Fourier methods are too hard for a graphic designer to understand Fourier methods are bad because they are the least efficient basis to represent your image. It is not intuitive to increase the gain of the low frequencies when you want to increase something like the shadow. There is no mapping between features in an object and the Fourier domain. More advanced, spatial domain techniques might be appropriate and these already exist and are being used by graphics design professionals.