5 ms·

Moiré no more (2021)

andreareina 4y ago
Past discussion: https://news.ycombinator.com/item?id=28663719 https://news.ycombinator.com/item?id=28663719 Relevant xkcd: https://xkcd.com/1814/ https://xkcd.com/1814/
- PuddleOfSausage 4y ago
  Ah, thanks. I did search before submitting but nothing came up at the time (must have been my connection).
  - dang 4y ago
    Reposts are fine after a year or so! This is in the FAQ: https://news.ycombinator.com/newsfaq.html https://news.ycombinator.com/newsfaq.html.
- dang 4y ago
  Thanks! Macroexpanded: Moiré No More - https://news.ycombinator.com/item?id=28663719 https://news.ycombinator.com/item?id=28663719 - Sept 2021 (112 comments) Moiré no more (How to de-moiré a halftoned photo) - https://news.ycombinator.com/item?id=28394884 https://news.ycombinator.com/item?id=28394884 - Sept 2021 (2 comments)
anotheryou 4y ago
> All the demoireing guides tell you to apply a sharpening filter after this to compensate, but that’s like reheating cold pizza. You will never get back to where you were before, no matter how hard you try Actually the noise from the moire can fake detail that wasn't there in the first place. Put film grain or a raster on top of a blurry image and it will subjectively improve in quality.
bargle0 4y ago
A certain YouTube creator wears striped shirts on camera a lot, but I feel like he should probably know better. I wonder if it’s an intentional, subtle troll?
- aargh_aargh 4y ago
  What's the point of your comment without naming him?
zokier 4y ago
I wonder what would be the more systematic approach than manually painting over the fft image? That constellation of peaks in fft should be reasonably easy to recognize (semi-)automatically, then you'd need to figure out good mask for those.. is it a circular blob, or maybe diamond/star shape? And what size is best? Etc etc.
- d_tr 4y ago
  I think I 'd start with simple generated images first, like a sine wave rendered with dots, to get some initial insights.
- deleted 4y ago
  [deleted]
- dafelst 4y ago
  It has been a long time since I have done anything in this domain (20+ years), but if I understand correctly, the original article author is essentially filtering out high frequency signals by painting over the bright FFT spots, essentially creating a localized low-pass filter or maybe band-pass filter. FFT essentially just transforms data into its frequency domain, so I assume the representation shown in the article is just that, brighter spots indicate higher frequency information (which would come from all the little spaced dots), so killing those essentially smooths the noise out of the data. I imagine that the same effect could be automatically achieved by identifying which frequencies to filter out to get the desired effect.
  - BoatyPrint 4y ago
    I dont believe that by painting over the bright FFT spots you are necessarily removing the high frequencies. Instead, you are removing the strong frequencies from the image. The strong frequencies cause peaks in the FFT image. Its the repetitive nature of the moire effect that cause peaks in the FFT image.
    - dafelst 4y ago
      Ahhh, so perhaps the position indicates the frequency and the brightness indicates the magnitude of the frequency? That makes sense - in that case the atricle's approach feels more like you said, a manually applied band pass or comb filter, i.e. it either removes or attenuates signals within certain frequency ranges.
planede 4y ago
When you get Moiré from the sensor of a camera or scanner, then there is nothing much you can do. But extreme Moiré effect from resizing is not really Moiré. It's aliasing. And it typically happens because the resizing happens in a non-linear colorspace and/or with a poor kernel. Unfortunately this mistake is quite common in many image processing software.
- stared 4y ago
  Aliasing IS a type of Moiré. Take an image, and impose it on a grid of dots - and you get the same afect as from naive resampling.
  - TheRealPomax 4y ago
    Technically it's not, but your description is, because of the words "grid of dots", which means there's white space between the dots (as dots are circular), a necessary requirement for the Moiré effect. We see this in image sensors as well, where all photosites have a tiny amount of space between them. Image resizing isn't a grid of dots, though. At the data level, pixels don't have space between each pixel to cause Moiré effects. Instead, interpolation of the colors leads to patterns similar to, but distinct from, the Moiré effect. The Moiré effect is "not really there" in that it's a perceptual effect: you can reproduce it with an SVG with the same dot pattern on two layers, and transforming one of them a little. A human observer will "see" patterns that aren't actually there, and as you zoom in the effect disappears (if you're now thinking "because it's on a monitor!", then no: it's not. The same is true for two physical transparencies with the same dot pattern placed slightly shifted on an old school projector, and then changing the zoom on the projector) Antialiasing is a data property: it can result from lossy interpolating of pixels to a new grid, and is there no matter how much you zoom in because zooming in on a pixel doesn't change its color. The confusion comes from talking about Moiré patterns on grid based displays. Those will "lock in" or even intensify the effect, but they don't cause the effect.
- jbay808 4y ago
  There's not much you can do to fix it in software, but you can fix it in the optics. You can add a Moiré filter to the camera, which is an optical low-pass filter, ensuring that the image that reaches the sensor does not have any spatial-frequency components above the Nyquist frequency of the sensor (ie, twice the pixel pitch).
moloch-hai 4y ago
So all the people who insisted nothing could be done were just ignorant of what could be done?
- maxbond 4y ago
  A sage reminder to all of us that sometimes it's better to upvote & listen (or in this case heart/retweet) to see if a better response surfaces than to offer our two cents.
wazoox 4y ago
I obtained a similar effect with The Gimp and the wavelet decomposition plugin[1] that splits the picture in various layers containing details at different granularity. You can then simply remove the "bad" ones. On portraits, you can remove freckles and wrinkles very easily this way, too. [1]: https://docs.gimp.org/2.10/en/plug-in-wavelet-decompose.html https://docs.gimp.org/2.10/en/plug-in-wavelet-decompose.html So I tried it on his image and here's how it looks with default settings: https://demo.intellique.org/nextcloud/index.php/s/SsgWGczo6GMe5yx/download/decomposed.jpg https://demo.intellique.org/nextcloud/index.php/s/SsgWGczo6G... Much less work :)
DrNosferatu 4y ago
Done since the mid-1960's :) but, nevertheless, always an interesting read! This is classical (as in pre-GPU AI/ML) Image Processing. The reference text book in the field, "Digital Image Processing" by Gonzalez et al, shows how to do it: like the pioneers at JPL's Image Processing Laboratory (est. 1965) denoised Mars pictures from the Mariner probes using Frequency Domain methods like the posted article uses - see an example below. [https://pixelcraft.photo.blog/2021/09/29/the-early-days-of-image-processing-to-mars-and-beyond/ https://pixelcraft.photo.blog/2021/09/29/the-early-days-of-i...]
- DrNosferatu 4y ago
  Speaking of which, with the advent of AI/ML methods, how are Image Processing and Machine Vision courses taught today at Universities? - Has Python or other free languages replaced Matlab yet? - Is Gonzales' course structure still followed somehow?
- jbay808 4y ago
  It's so classical, it can be done without a digital computer at all, using fourier optics! The true fastest fourier transform, done at the speed of light. https://en.wikipedia.org/wiki/Fourier_optics https://en.wikipedia.org/wiki/Fourier_optics At the focal plane of a lens, an image is converted into its spatial fourier components. By adding blocking elements at that focal plane (eg. an aperture, or a piece of glass with marker dots), unwanted frequency components can be cut out from the image.
  - DrNosferatu 4y ago
    Aha - very nice! My background with Fourier optics is mostly theoretical and the applications numerical. Any references (books or online) exist about physical applications, like the one you mention?
Gordonjcp 4y ago
Hang on then, if you're just notching out certain frequencies by painting over them in a 2D FFT could you do the same thing by treating the image like it was raster-scanned video and passing it through a notch filter at the Moiré frequency, or for that matter a comb filter? After all that's how you remove PAL subcarrier speckle from composite video.
tantalor 4y ago
> occurring when two similar frequencies overlap and create strange, pulsating patterns I just made the connection between moire and beats. Nice! https://en.wikipedia.org/wiki/Beat_(acoustics) https://en.wikipedia.org/wiki/Beat_(acoustics)
lowbloodsugar 4y ago
There's literally a button for this in Photoshop, and I'm sure every other tool for photos / scanning.
dvh 4y ago
Here's my favorite "alternative" explanation of moiré: sin(a) * sin(b) = 0.5 * (cos(a - b) - cos(a + b)) You may remember this formula from high school. The arguments a and b are basically frequencies. When you multiply two frequencies a and b, you will get their difference cos(a - b) and their sum cos(a + b). If two frequencies are similar (e.g. 101Hz and 100Hz) you get one very low frequency (101Hz - 100Hz = 1Hz) and the other one will be much higher number (101Hz + 100Hz = 201Hz). It's hard to see that higher frequency 201Hz, you have to come really close, but it is easy to see that low frequency, that 1Hz would manifest as 1 dark blob across entire area.
hilbert42 4y ago
"This is why nobody will allow you to wear stripes on television:" This used to be cardinal rule in television broadcasting but it's much less a problem nowadays with the increased resolution/line rate of HD TV and video processing (filters) designed to eliminate it. Nevertheless, whilst once discouraged, moiré was often a useful tool as camera and CCU (Camera Control Unit) operators would use it to focus an image—the more pronounced the moiré the sharper the camera's focus. As bad as moiré is perceived a much greater cardinal 'sin' in both television broadcasting and photography is lateral inversion. This is where the image has been swapped in the horizontal direction (in some places it was a dismissible offence), as it's a complete distortion/misrepresentation of the image, which in some instances, may go undiscovered—there being no visual clues to indicate the problem. Lateral inversion is very obvious when writing, street signs etc., appear backwards but for some reason it's very much less so when images of humans are involved. Despite the fact that laterally-inverted images put men's clothes on women and vice versa—as blouses, shitrts, coats and pants flies appear the wrong way around—few people seem to notice. There's much evidence for this, one I often cite is that there are images from WWII on the US National Archive by the US Army Signal Corps that are still laterally inverted after 70-plus years that no one has bothered to correct (it's not a recent scanning error either as the Signal Corps logo (which is embedded within the photo) is not laterally inverted but the image content is). BTW, laterally inverting a film image reduces the resolution as the negative or slide is no longer in the normal focus plane.

Past discussion: https://news.ycombinator.com/item?id=28663719 https://news.ycombinator.com/item?id=28663719 Relevant xkcd: https://xkcd.com/1814/ https://xkcd.com/1814/

PuddleOfSausage 4y ago
Ah, thanks. I did search before submitting but nothing came up at the time (must have been my connection).
- dang 4y ago
  Reposts are fine after a year or so! This is in the FAQ: https://news.ycombinator.com/newsfaq.html https://news.ycombinator.com/newsfaq.html.
dang 4y ago
Thanks! Macroexpanded: Moiré No More - https://news.ycombinator.com/item?id=28663719 https://news.ycombinator.com/item?id=28663719 - Sept 2021 (112 comments) Moiré no more (How to de-moiré a halftoned photo) - https://news.ycombinator.com/item?id=28394884 https://news.ycombinator.com/item?id=28394884 - Sept 2021 (2 comments)

anotheryou 4y ago

> All the demoireing guides tell you to apply a sharpening filter after this to compensate, but that’s like reheating cold pizza. You will never get back to where you were before, no matter how hard you try Actually the noise from the moire can fake detail that wasn't there in the first place. Put film grain or a raster on top of a blurry image and it will subjectively improve in quality.

bargle0 4y ago

A certain YouTube creator wears striped shirts on camera a lot, but I feel like he should probably know better. I wonder if it’s an intentional, subtle troll?

aargh_aargh 4y ago
What's the point of your comment without naming him?

zokier 4y ago

I wonder what would be the more systematic approach than manually painting over the fft image? That constellation of peaks in fft should be reasonably easy to recognize (semi-)automatically, then you'd need to figure out good mask for those.. is it a circular blob, or maybe diamond/star shape? And what size is best? Etc etc.

d_tr 4y ago
I think I 'd start with simple generated images first, like a sine wave rendered with dots, to get some initial insights.
deleted 4y ago
[deleted]
dafelst 4y ago
It has been a long time since I have done anything in this domain (20+ years), but if I understand correctly, the original article author is essentially filtering out high frequency signals by painting over the bright FFT spots, essentially creating a localized low-pass filter or maybe band-pass filter. FFT essentially just transforms data into its frequency domain, so I assume the representation shown in the article is just that, brighter spots indicate higher frequency information (which would come from all the little spaced dots), so killing those essentially smooths the noise out of the data. I imagine that the same effect could be automatically achieved by identifying which frequencies to filter out to get the desired effect.
- BoatyPrint 4y ago
  I dont believe that by painting over the bright FFT spots you are necessarily removing the high frequencies. Instead, you are removing the strong frequencies from the image. The strong frequencies cause peaks in the FFT image. Its the repetitive nature of the moire effect that cause peaks in the FFT image.
  - dafelst 4y ago
    Ahhh, so perhaps the position indicates the frequency and the brightness indicates the magnitude of the frequency? That makes sense - in that case the atricle's approach feels more like you said, a manually applied band pass or comb filter, i.e. it either removes or attenuates signals within certain frequency ranges.

planede 4y ago

When you get Moiré from the sensor of a camera or scanner, then there is nothing much you can do. But extreme Moiré effect from resizing is not really Moiré. It's aliasing. And it typically happens because the resizing happens in a non-linear colorspace and/or with a poor kernel. Unfortunately this mistake is quite common in many image processing software.

stared 4y ago
Aliasing IS a type of Moiré. Take an image, and impose it on a grid of dots - and you get the same afect as from naive resampling.
- TheRealPomax 4y ago
  Technically it's not, but your description is, because of the words "grid of dots", which means there's white space between the dots (as dots are circular), a necessary requirement for the Moiré effect. We see this in image sensors as well, where all photosites have a tiny amount of space between them. Image resizing isn't a grid of dots, though. At the data level, pixels don't have space between each pixel to cause Moiré effects. Instead, interpolation of the colors leads to patterns similar to, but distinct from, the Moiré effect. The Moiré effect is "not really there" in that it's a perceptual effect: you can reproduce it with an SVG with the same dot pattern on two layers, and transforming one of them a little. A human observer will "see" patterns that aren't actually there, and as you zoom in the effect disappears (if you're now thinking "because it's on a monitor!", then no: it's not. The same is true for two physical transparencies with the same dot pattern placed slightly shifted on an old school projector, and then changing the zoom on the projector) Antialiasing is a data property: it can result from lossy interpolating of pixels to a new grid, and is there no matter how much you zoom in because zooming in on a pixel doesn't change its color. The confusion comes from talking about Moiré patterns on grid based displays. Those will "lock in" or even intensify the effect, but they don't cause the effect.
jbay808 4y ago
There's not much you can do to fix it in software, but you can fix it in the optics. You can add a Moiré filter to the camera, which is an optical low-pass filter, ensuring that the image that reaches the sensor does not have any spatial-frequency components above the Nyquist frequency of the sensor (ie, twice the pixel pitch).

moloch-hai 4y ago

So all the people who insisted nothing could be done were just ignorant of what could be done?

maxbond 4y ago
A sage reminder to all of us that sometimes it's better to upvote & listen (or in this case heart/retweet) to see if a better response surfaces than to offer our two cents.

wazoox 4y ago

I obtained a similar effect with The Gimp and the wavelet decomposition plugin[1] that splits the picture in various layers containing details at different granularity. You can then simply remove the "bad" ones. On portraits, you can remove freckles and wrinkles very easily this way, too. [1]: https://docs.gimp.org/2.10/en/plug-in-wavelet-decompose.html https://docs.gimp.org/2.10/en/plug-in-wavelet-decompose.html So I tried it on his image and here's how it looks with default settings: https://demo.intellique.org/nextcloud/index.php/s/SsgWGczo6GMe5yx/download/decomposed.jpg https://demo.intellique.org/nextcloud/index.php/s/SsgWGczo6G... Much less work :)

DrNosferatu 4y ago

Done since the mid-1960's :) but, nevertheless, always an interesting read! This is classical (as in pre-GPU AI/ML) Image Processing. The reference text book in the field, "Digital Image Processing" by Gonzalez et al, shows how to do it: like the pioneers at JPL's Image Processing Laboratory (est. 1965) denoised Mars pictures from the Mariner probes using Frequency Domain methods like the posted article uses - see an example below. [https://pixelcraft.photo.blog/2021/09/29/the-early-days-of-image-processing-to-mars-and-beyond/ https://pixelcraft.photo.blog/2021/09/29/the-early-days-of-i...]

DrNosferatu 4y ago
Speaking of which, with the advent of AI/ML methods, how are Image Processing and Machine Vision courses taught today at Universities? - Has Python or other free languages replaced Matlab yet? - Is Gonzales' course structure still followed somehow?
jbay808 4y ago
It's so classical, it can be done without a digital computer at all, using fourier optics! The true fastest fourier transform, done at the speed of light. https://en.wikipedia.org/wiki/Fourier_optics https://en.wikipedia.org/wiki/Fourier_optics At the focal plane of a lens, an image is converted into its spatial fourier components. By adding blocking elements at that focal plane (eg. an aperture, or a piece of glass with marker dots), unwanted frequency components can be cut out from the image.
- DrNosferatu 4y ago
  Aha - very nice! My background with Fourier optics is mostly theoretical and the applications numerical. Any references (books or online) exist about physical applications, like the one you mention?

Gordonjcp 4y ago

Hang on then, if you're just notching out certain frequencies by painting over them in a 2D FFT could you do the same thing by treating the image like it was raster-scanned video and passing it through a notch filter at the Moiré frequency, or for that matter a comb filter? After all that's how you remove PAL subcarrier speckle from composite video.

tantalor 4y ago

> occurring when two similar frequencies overlap and create strange, pulsating patterns I just made the connection between moire and beats. Nice! https://en.wikipedia.org/wiki/Beat_(acoustics) https://en.wikipedia.org/wiki/Beat_(acoustics)

lowbloodsugar 4y ago

There's literally a button for this in Photoshop, and I'm sure every other tool for photos / scanning.

dvh 4y ago

Here's my favorite "alternative" explanation of moiré: sin(a) * sin(b) = 0.5 * (cos(a - b) - cos(a + b)) You may remember this formula from high school. The arguments a and b are basically frequencies. When you multiply two frequencies a and b, you will get their difference cos(a - b) and their sum cos(a + b). If two frequencies are similar (e.g. 101Hz and 100Hz) you get one very low frequency (101Hz - 100Hz = 1Hz) and the other one will be much higher number (101Hz + 100Hz = 201Hz). It's hard to see that higher frequency 201Hz, you have to come really close, but it is easy to see that low frequency, that 1Hz would manifest as 1 dark blob across entire area.

hilbert42 4y ago

"This is why nobody will allow you to wear stripes on television:" This used to be cardinal rule in television broadcasting but it's much less a problem nowadays with the increased resolution/line rate of HD TV and video processing (filters) designed to eliminate it. Nevertheless, whilst once discouraged, moiré was often a useful tool as camera and CCU (Camera Control Unit) operators would use it to focus an image—the more pronounced the moiré the sharper the camera's focus. As bad as moiré is perceived a much greater cardinal 'sin' in both television broadcasting and photography is lateral inversion. This is where the image has been swapped in the horizontal direction (in some places it was a dismissible offence), as it's a complete distortion/misrepresentation of the image, which in some instances, may go undiscovered—there being no visual clues to indicate the problem. Lateral inversion is very obvious when writing, street signs etc., appear backwards but for some reason it's very much less so when images of humans are involved. Despite the fact that laterally-inverted images put men's clothes on women and vice versa—as blouses, shitrts, coats and pants flies appear the wrong way around—few people seem to notice. There's much evidence for this, one I often cite is that there are images from WWII on the US National Archive by the US Army Signal Corps that are still laterally inverted after 70-plus years that no one has bothered to correct (it's not a recent scanning error either as the Signal Corps logo (which is embedded within the photo) is not laterally inverted but the image content is). BTW, laterally inverting a film image reduces the resolution as the negative or slide is no longer in the normal focus plane.