6 ms·

Removing blur from images – deconvolution and using simple image filters

131 points by bartwr 4y ago

iphoneeveryone 4y ago
dymk 4y ago
Getting rate limit exceeded for most of the images that try to load
- siegelzero 4y ago
  https://web.archive.org/web/20220526162025/https://bartwronski.com/2022/05/26/removing-blur-from-images-deconvolution-and-using-optimized-simple-filters/ https://web.archive.org/web/20220526162025/https://bartwrons...
- dark-star 4y ago
  Wouldn't it be great if instead of stopping to deliver the images, Google would just massively scale them down and still serve them? That way they could be scaled up and deconvoluted to give back (almost) the original images... ;-)
Beldin 4y ago
Despite not having read the article, I feel compelled to plug my students' port of a deblur algorithm to Gimp. https://www.open.ou.nl/hjo/stud-finished.htm#elseenton https://www.open.ou.nl/hjo/stud-finished.htm#elseenton Thesis (unfortunately) in Dutch, but a writeup in English (on SoylentNews) is linked. Moreover, its pictures speak volumes. And the plugin worked back then - if it doesn't, let me know and I'll see what I can do (ie. reach out to the authors)
- gus_massa 4y ago
  I think combining a version of your post in SoylentNews and adding a few examples would make an interesting post. Are you still working on this? In the page 51 of your tesis is a form with a lot of parameters to tweak. Is that a manual process or there is some automatic adjustment? I just noticed that you have some examples in the "Appendix b" of your thesis (page 72 of the pdf). The first time I stopped reading once I reached the bibliography.
  - bee_rider 4y ago
    Note that this is their students' thesis, not theirs.
  - Beldin 4y ago
    I really appreciate the interest! I'll try to find some time for a reply going into some details, but I need to suit down at a computer undisturbed for an hour or so (also to refresh some things), which is hard to fit in the next few days. So don't hold your breath; I'll do my best.
  - wazoox 4y ago
    See this page for examples: https://sites.google.com/site/jspanhomepage/l0rigdeblur https://sites.google.com/site/jspanhomepage/l0rigdeblur impressive!
    - gus_massa 4y ago
      Nice work, but IIUC it's made by a different research group.
    - rasz 4y ago
      sadly no, this is it https://www.open.ou.nl/hjo/software/text-deblur.htm https://www.open.ou.nl/hjo/software/text-deblur.htm and the results https://www.open.ou.nl/hjo/supervision/2016-deblur-bsc-thesis/TestSuite_TestResults.zip https://www.open.ou.nl/hjo/supervision/2016-deblur-bsc-thesi... dont look as mindblowing as google ones
HPsquared 4y ago
This kind of thing doesn't work so well if the blurred image has been compressed (e.g. jpeg). A lot of information is hidden in the blurred image which gets removed by the compression algorithm.
nomel 4y ago
At one time I wanted to see if I could pre-deconvolve what's shown on a computer screen so that a person with a slight vision problem could use it without having to wear glasses, using their blurry eyes to convolve the images back to the original image. I found a paper/website that did this (can't find it anymore), with some example images of text, and it worked fairly well, but spatial resolution was very limited. One interesting limitation is that negative brightness is needed to make it work without distortion. Limiting the brightness range of the image, so that black can act as a sort of "darker than possible" level helps.
- bartwr 4y ago
  That's a super interesting idea and I think should work very well. If it was combined with the use of camera (for tracking the distance away from the monitor / screen) I think it could work very well; especially that noise / quantization / compression is not a problem in such a case.
  - anon_123g987 4y ago
    The paper: O. Keleş, E. Anarım: Adjustment of Digital Screens to Compensate the Eye Refractive Errors via Deconvolution. Link to PDF: https://www.researchgate.net/profile/Emin-Anarim/publication/338072758_Adjustment_of_Digital_Screens_to_Compensate_the_Eye_Refractive_Errors_via_Deconvolution/links/5e4a5590a6fdccd965ac46bb/Adjustment-of-Digital-Screens-to-Compensate-the-Eye-Refractive-Errors-via-Deconvolution.pdf https://www.researchgate.net/profile/Emin-Anarim/publication...
    - loxias 4y ago
      Dude. I have... multiple books on my shelf about Deconvolution (though more 1 dimensional than 2 dimensional)... never even heard of this and am tickled pink :D Thanks! (and thanks grandparent for remembering it) Will 100% post to HN ~should~ when I try implementing it.
      - credit_guy 4y ago
omarhaneef 4y ago
Now I have to file this link for when someone makes fun of "computer, enhance" in sci fi (Blade Runner comes to mind)
- thanatos519 4y ago
  Speaking of Blade Runner, back when I had an 800x600 projector, I was having a hard time not noticing the DLP squares. I had a 1080p copy of Blade Runner. I downscaled it with a strong Lanczos filter, so the video looked too sharp. Then I put the projector slightly out of focus. The DLP squares disappeared, but the blur also compensated for the sharpening and the video looked amazingly non-digital.
  - Sharlin 4y ago
    Basically what the low-pass filter in digital cameras does.
- IAmEveryone 4y ago
  That was always stupid and there were always demonstrations of said stupidity. It’s one of those snarky contrarian takes people repeat ad nauseam. See also: correlation and causation.
- anon_123g987 4y ago
  I think the most famous example of using this technique, that's maybe easier to remember and refer later, is the correction of the flawed mirror of the Hubble Space Telescope. https://www.nasa.gov/content/hubbles-mirror-flaw https://www.nasa.gov/content/hubbles-mirror-flaw https://en.wikipedia.org/wiki/Hubble_Space_Telescope#Flawed_mirror https://en.wikipedia.org/wiki/Hubble_Space_Telescope#Flawed_...
Scene_Cast2 4y ago
This is an ongoing area of study. Here's a list of research (ML-focused) on the topic - https://github.com/subeeshvasu/Awesome-Deblurring https://github.com/subeeshvasu/Awesome-Deblurring I personally used a technique (based on a paper from that list) that learns a kernel based on the idea that it's easier to learn a latent source image and a blur compared to learning a blurred image.
EricAAJohnson 4y ago
Some great questions and answers on deconvolution at Signal Processing Stack Exchange: https://dsp.stackexchange.com/questions/tagged/deconvolution?tab=Votes https://dsp.stackexchange.com/questions/tagged/deconvolution....
dimatura 4y ago
Deblurring is a classical ill-posed inverse problem where no matter how much math you throw at it (and there's lots of very interesting work in that vein) at the end of the day, data-driven solutions (i.e., ML) will work best for most cases. The only caveat is that ML solutions can be somewhat riskier in that you increase the chances of hallucinating false details, but given the quality of current SOTA it's often an acceptable trade-off (application-dependent, of course). Another caveat: for ML-based solutions, data is key, and a lot of early work on ML-based deblurring suffered from the fact that the "blurry" image data was synthetically created by blurring sharp images. Thankfully, the community has come to realize this and has taken steps to fix it, by collecting "real" blurry datasets and creating more realistic synthetic blurry datasets.
- SilverBirch 4y ago
  Well this depends on the application right, if I zoom in on an image in twitter and see some detail that wasn't really there... who cares? If I then go on to present that detail in a court of law as evidence of image manipulation, or some characteristic that doesn't exist, or to persuade people the evidence is fallible even though it originally wasn't. Well that's a problem, and it's a difficult problem because it comes down to difficult problems around what you're adveritising you're doing and what people trust you to do. Maybe it doesn't come down to a point of law. Maybe it comes down to trolls on twitter repeating those arguments, but without the burden of proof.
  - dimatura 4y ago
    Absolutely! Hence my caveat. Other scenarios where I'd be worried about hallucinations are in algorithms that control self-driving vehicles, or in medical-image analysis. I think accurately quantifying uncertainty for these problems (so that an algorithm, rather than just hallucinate, might say "I don't know") is an important and currently active research topic.
balaji1 4y ago
is there a simple way to detect blur? I want the iPhone to show an alert that the previous photo was blurred - I ended up with a blurred photo of text on a poster as I walked past it.
- loxias 4y ago
  Look at the frequency spectrum. If you consider focus, or "blurness" a parameter that varies, the value where the picture is "least blurry" or "most in focus" tends to be a local maxima of the power of the higher frequency components. When the picture is crisper edges will be more pronounced, which is high(er) frequency information. So the high frequency power would be at a local maxima. (If that doesn't make sense yet, try imagining the opposite :D When a picture is the "most blurred" or "most out of focus" it looks like a soft colorfield. low frequency :D)
- anon_123g987 4y ago
  There's no simple and easy way as far as I know, but there are many approaches both for blur detection and automatic correction (kernel estimation). It's called blind deconvolution. https://en.wikipedia.org/wiki/Blind_deconvolution https://en.wikipedia.org/wiki/Blind_deconvolution
melony 4y ago
The next step is belief propagation and Ising models

131 points by bartwr 4y ago

iphoneeveryone 4y ago

dymk 4y ago

Getting rate limit exceeded for most of the images that try to load

siegelzero 4y ago
https://web.archive.org/web/20220526162025/https://bartwronski.com/2022/05/26/removing-blur-from-images-deconvolution-and-using-optimized-simple-filters/ https://web.archive.org/web/20220526162025/https://bartwrons...
dark-star 4y ago
Wouldn't it be great if instead of stopping to deliver the images, Google would just massively scale them down and still serve them? That way they could be scaled up and deconvoluted to give back (almost) the original images... ;-)

Beldin 4y ago

Despite not having read the article, I feel compelled to plug my students' port of a deblur algorithm to Gimp. https://www.open.ou.nl/hjo/stud-finished.htm#elseenton https://www.open.ou.nl/hjo/stud-finished.htm#elseenton Thesis (unfortunately) in Dutch, but a writeup in English (on SoylentNews) is linked. Moreover, its pictures speak volumes. And the plugin worked back then - if it doesn't, let me know and I'll see what I can do (ie. reach out to the authors)

gus_massa 4y ago
I think combining a version of your post in SoylentNews and adding a few examples would make an interesting post. Are you still working on this? In the page 51 of your tesis is a form with a lot of parameters to tweak. Is that a manual process or there is some automatic adjustment? I just noticed that you have some examples in the "Appendix b" of your thesis (page 72 of the pdf). The first time I stopped reading once I reached the bibliography.
- bee_rider 4y ago
  Note that this is their students' thesis, not theirs.
- Beldin 4y ago
  I really appreciate the interest! I'll try to find some time for a reply going into some details, but I need to suit down at a computer undisturbed for an hour or so (also to refresh some things), which is hard to fit in the next few days. So don't hold your breath; I'll do my best.
- wazoox 4y ago
  See this page for examples: https://sites.google.com/site/jspanhomepage/l0rigdeblur https://sites.google.com/site/jspanhomepage/l0rigdeblur impressive!
  - gus_massa 4y ago
    Nice work, but IIUC it's made by a different research group.
  - rasz 4y ago
    sadly no, this is it https://www.open.ou.nl/hjo/software/text-deblur.htm https://www.open.ou.nl/hjo/software/text-deblur.htm and the results https://www.open.ou.nl/hjo/supervision/2016-deblur-bsc-thesis/TestSuite_TestResults.zip https://www.open.ou.nl/hjo/supervision/2016-deblur-bsc-thesi... dont look as mindblowing as google ones

HPsquared 4y ago

This kind of thing doesn't work so well if the blurred image has been compressed (e.g. jpeg). A lot of information is hidden in the blurred image which gets removed by the compression algorithm.

nomel 4y ago

At one time I wanted to see if I could pre-deconvolve what's shown on a computer screen so that a person with a slight vision problem could use it without having to wear glasses, using their blurry eyes to convolve the images back to the original image. I found a paper/website that did this (can't find it anymore), with some example images of text, and it worked fairly well, but spatial resolution was very limited. One interesting limitation is that negative brightness is needed to make it work without distortion. Limiting the brightness range of the image, so that black can act as a sort of "darker than possible" level helps.

bartwr 4y ago
That's a super interesting idea and I think should work very well. If it was combined with the use of camera (for tracking the distance away from the monitor / screen) I think it could work very well; especially that noise / quantization / compression is not a problem in such a case.
- anon_123g987 4y ago
  The paper: O. Keleş, E. Anarım: Adjustment of Digital Screens to Compensate the Eye Refractive Errors via Deconvolution. Link to PDF: https://www.researchgate.net/profile/Emin-Anarim/publication/338072758_Adjustment_of_Digital_Screens_to_Compensate_the_Eye_Refractive_Errors_via_Deconvolution/links/5e4a5590a6fdccd965ac46bb/Adjustment-of-Digital-Screens-to-Compensate-the-Eye-Refractive-Errors-via-Deconvolution.pdf https://www.researchgate.net/profile/Emin-Anarim/publication...
  - loxias 4y ago
    Dude. I have... multiple books on my shelf about Deconvolution (though more 1 dimensional than 2 dimensional)... never even heard of this and am tickled pink :D Thanks! (and thanks grandparent for remembering it) Will 100% post to HN ~should~ when I try implementing it.
    - credit_guy 4y ago

omarhaneef 4y ago

Now I have to file this link for when someone makes fun of "computer, enhance" in sci fi (Blade Runner comes to mind)

thanatos519 4y ago
Speaking of Blade Runner, back when I had an 800x600 projector, I was having a hard time not noticing the DLP squares. I had a 1080p copy of Blade Runner. I downscaled it with a strong Lanczos filter, so the video looked too sharp. Then I put the projector slightly out of focus. The DLP squares disappeared, but the blur also compensated for the sharpening and the video looked amazingly non-digital.
- Sharlin 4y ago
  Basically what the low-pass filter in digital cameras does.
IAmEveryone 4y ago
That was always stupid and there were always demonstrations of said stupidity. It’s one of those snarky contrarian takes people repeat ad nauseam. See also: correlation and causation.
anon_123g987 4y ago
I think the most famous example of using this technique, that's maybe easier to remember and refer later, is the correction of the flawed mirror of the Hubble Space Telescope. https://www.nasa.gov/content/hubbles-mirror-flaw https://www.nasa.gov/content/hubbles-mirror-flaw https://en.wikipedia.org/wiki/Hubble_Space_Telescope#Flawed_mirror https://en.wikipedia.org/wiki/Hubble_Space_Telescope#Flawed_...

Scene_Cast2 4y ago

This is an ongoing area of study. Here's a list of research (ML-focused) on the topic - https://github.com/subeeshvasu/Awesome-Deblurring https://github.com/subeeshvasu/Awesome-Deblurring I personally used a technique (based on a paper from that list) that learns a kernel based on the idea that it's easier to learn a latent source image and a blur compared to learning a blurred image.

EricAAJohnson 4y ago

Some great questions and answers on deconvolution at Signal Processing Stack Exchange: https://dsp.stackexchange.com/questions/tagged/deconvolution?tab=Votes https://dsp.stackexchange.com/questions/tagged/deconvolution....

dimatura 4y ago

Deblurring is a classical ill-posed inverse problem where no matter how much math you throw at it (and there's lots of very interesting work in that vein) at the end of the day, data-driven solutions (i.e., ML) will work best for most cases. The only caveat is that ML solutions can be somewhat riskier in that you increase the chances of hallucinating false details, but given the quality of current SOTA it's often an acceptable trade-off (application-dependent, of course). Another caveat: for ML-based solutions, data is key, and a lot of early work on ML-based deblurring suffered from the fact that the "blurry" image data was synthetically created by blurring sharp images. Thankfully, the community has come to realize this and has taken steps to fix it, by collecting "real" blurry datasets and creating more realistic synthetic blurry datasets.

SilverBirch 4y ago
Well this depends on the application right, if I zoom in on an image in twitter and see some detail that wasn't really there... who cares? If I then go on to present that detail in a court of law as evidence of image manipulation, or some characteristic that doesn't exist, or to persuade people the evidence is fallible even though it originally wasn't. Well that's a problem, and it's a difficult problem because it comes down to difficult problems around what you're adveritising you're doing and what people trust you to do. Maybe it doesn't come down to a point of law. Maybe it comes down to trolls on twitter repeating those arguments, but without the burden of proof.
- dimatura 4y ago
  Absolutely! Hence my caveat. Other scenarios where I'd be worried about hallucinations are in algorithms that control self-driving vehicles, or in medical-image analysis. I think accurately quantifying uncertainty for these problems (so that an algorithm, rather than just hallucinate, might say "I don't know") is an important and currently active research topic.

balaji1 4y ago

is there a simple way to detect blur? I want the iPhone to show an alert that the previous photo was blurred - I ended up with a blurred photo of text on a poster as I walked past it.

loxias 4y ago
Look at the frequency spectrum. If you consider focus, or "blurness" a parameter that varies, the value where the picture is "least blurry" or "most in focus" tends to be a local maxima of the power of the higher frequency components. When the picture is crisper edges will be more pronounced, which is high(er) frequency information. So the high frequency power would be at a local maxima. (If that doesn't make sense yet, try imagining the opposite :D When a picture is the "most blurred" or "most out of focus" it looks like a soft colorfield. low frequency :D)
anon_123g987 4y ago
There's no simple and easy way as far as I know, but there are many approaches both for blur detection and automatic correction (kernel estimation). It's called blind deconvolution. https://en.wikipedia.org/wiki/Blind_deconvolution https://en.wikipedia.org/wiki/Blind_deconvolution

melony 4y ago

The next step is belief propagation and Ising models