8 ms·
Photoshop 'unblur' leaves MAX audience gasping for air
- benwerd 15y agoForensic police drama writers everywhere: vindicated. This is seriously cool technology.
- ookblah 15y agohahaha i was just thinking the same thing. "zoom in on that. good. now....ENHANCE."
- mathgladiator 15y ago"... and ROTATE"
- aguynamedben 15y agoThe reflection... http://www.youtube.com/watch?v=Vxq9yj2pVWk http://www.youtube.com/watch?v=Vxq9yj2pVWk
- deleted 15y ago[deleted]
- evo 15y agoI assume this just undoes the effects of an unsteady camera during exposure--all the information is present in the image, just smeared along a predictable path, and can be reversed by something akin to a deconvolution (but more complicated as it involves arbitrary nearby points). Upscaling an intrinsically low-resolution image is still in the realm of creating information where there was none, I think.
- cubicle67 15y agonot really, because it's not the same thing what CSI does it add information that wasn't initially there whereas what this is doing is just unscrambling the information. This is only for photos where the camera has moved during exposure, so should be great for low light shots (indoors etc) where you need a shutter speed a bit slower than optimal, say 1/5 second
- InclinedPlane 15y agoYes. This achieves the maximum possible theoretical resolution of an imaging system. The CSI-ish "enhance" feature typically goes well beyond the maximum possible resolution of an image.
- po 15y agoDoes this work with just motion blur or also with aperture blur? It seems like they are calculating the motion of the camera so perhaps just the former.
- klodolph 15y agoDefocus (or "aperture blur") cannot be corrected by the methods they mention in the video. However, there are other kinds of blur you can correct.
- vilius 15y agoThis is an interesting way of completely avoiding defocus blur: http://www.lytro.com/picture_gallery http://www.lytro.com/picture_gallery
- nethsix 15y agoI suppose this is more of image sharpening rather than reconstruction. Is this very different from technology on cameras/phones that tries to reduce of photo blurness due to unsteady hands?
- klodolph 15y agoWhat makes you suppose that? On an abstract level, you can model blur and camera shake with a convolution kernel. You can then invert the kernel and get back the original image. As an analogy, imagine that someone gives you an audio file with an echo. You can subtract the echo with a filter. Camera shake is harder because of the extra dimension. (Of course, you only get back the exact original in the world of mathematics)
- jjm 15y agoShould hopefully get easier with a dedicated asic crunching on the extra dimension gathered from future built-in gyroscopes in cameras.
- viraptor 15y agoI don't believe the echo is a simpler example. It's just as complicated as blur really. You can remove both to a large extent, providing you have the correct model with correct parameters. Discovering right parameters is a hard problem in both cases.
- beagle3 15y agoWhat's the difference between "sharpening" and "reconstruction"? Both take an input image, apply some filter which is the best estimate of an inverse of a filter originally applied (e.g., wrong focus). The technique to unsteady hands artifacts often has access to motion data, which this thing does not.
- zwischenzug 15y agoSo who shot Kennedy?
- dlsspy 15y agoLet me load the specially constructed set of parameters specific to this image so that when I do the next step you get a really clear image. That was a little too hand-wavy. I'm a little dubious until I see what went into that phase.
- jczhang 15y agoHe used the same parameter profile each time.
- sdfx 15y agoNo, he's loading a different parameter file each time.
- artursapek 15y agoEither way, there's no point in watching a tech preview and being cynical about it.
- elliottkember 15y agoI think there is. I much prefer cynicism to the blind adulation of "this tech is amazing and magical". Otherwise it's almost marketing spin.
- artursapek 15y agoI think both are extreme ends of a spectrum on which you should try to stay in the middle somewhere.
- stan_rogers 15y agoTo an extent, this is already available -- for example in the Topaz Labs InFocus Photoshop plugin. There are some params to play with that make it easier to find the blur trajectory when the blur is motion-related (although if you leave it in "figure it out for yourself" mode, it gets it right often enough). InFocus (the current version) will only do linear trajectories, though -- it can't handle curves as well as this Photoshop sneak does. The parameter preload isn't cheating -- if they're anything like the InFocus params, they're pretty obvious but somewhat tedious. They're things like telling it that you're trying to correct motion blur rather than focus blur, what level of artifacting you're willing to put up with (for forensics or text recovery, you can put up with a lot of noise in the uninteresting part of the picture), the desired hardness of recovered edges, that sort of thing. It would have just been a time-waster for the demo (and, like in the demo, InFocus allows you to save the params as a preset).
- 51Cards 15y agoNow that is a feature I would upgrade for.
- alanh 15y agoBeen hoping for this for a while! The information is there, it’s just distorted. Great to see Adobe keep pushing this kind of photo editing magic forward. I bet the maths are crazy.
- mturmon 15y agoThe information is not really there, because the phase is not captured by the sensor. All you have is the intensity of the light.
- deleted 15y ago[deleted]
- oomkiller 15y agoMaybe they could apply this technology to Flash, so that video streaming on YouTube isn't so blurry :)
- deleted 15y ago[deleted]
- chrislo 15y agoI believe the speaker mentioned this algorithm was based on the Point Spread Function[1] but modified to model the movement of the point in time. Dougherty has[2] a static PSF deconvolution implementation that is fun to play with. [1] http://en.wikipedia.org/wiki/Point_spread_function http://en.wikipedia.org/wiki/Point_spread_function [2] http://www.optinav.com/Iterative-Deconvolution.htm http://www.optinav.com/Iterative-Deconvolution.htm
- Kliment 15y agoThe random anti-intellectual comments from the guys in the wheely chairs were extremely annoying and unfunny. This guy is there, showing something truly amazing, and they're all "What's an algorithm? Haha!". And they'll get away with it too.
- artursapek 15y agoRainn Wilson is actually a really smart guy. He was playing the classic stupid guy being wowed by a genius card, it was an actor's way of complimenting him.
- elliottkember 15y agoAnti-intellectualism aside, they still interrupted an amazing talk without adding any value. An actor's way of dominating the stage even when it's not your turn. A bit rude and unnecessary.
- wahnfrieden 15y agoHe was paid by the company to interject with quips during the demos, so any rudeness is understood by both parties. An awkward situation for all involved.
- DavidSJ 15y agoBeing paid to be rude doesn't make it not rude.
- bluedanieru 15y agoHe was paid to be rude to the people who were paying him to be rude? That might not make it not rude, but I can't imagine anything that would come closer.
- atomicdog 15y ago
- ck2 15y agoI'm more impressed with that overhead display - seems impossible? How does it disappear at the end - or is that a virtual digital overlay? Wait, is the entire background rear projected, like a borderless movie theater screen? Must be massive resolution ?!
- nknight 15y agoI'm inclined to say much or all of the background is indeed all rear-projected. If you look at the top-right around 5:06 or 5:07, you see what looks a lot like it might be light-emitting text floating in midair. (EDIT: Correction! It's not just floating there, it scrolls left just as the camera is coming down, I missed that on the first viewing. So it's definitely not just on a physical banner.) As for "massive resolution", slicing up a framebuffer and shooting out the components to multiple projectors wouldn't be a new idea, and I'll bet that's what was done here.
- voidpointer 15y agoYes, it was all projected. (also the side walls of the theatre). Some 20+ projectors pushing 300 million pixels/sec. The intro to the keynotes was pretty amazing as well, meshing the projection with light effects and live performance: http://www.youtube.com/watch?v=VrDPgUjqTQ8&feature=related http://www.youtube.com/watch?v=VrDPgUjqTQ8&feature=relat...
- powertower 15y agoI was thinking the same thing.
- bartwe 15y agoTo me it seems the magic is in getting the blur kernel in the first place, how do you get that ?
- Geee 15y agoIt's called blind deconvolution. Blind means that they have to first estimate the original convolution/blur kernel and in the second phase, apply the deconvolution. If there's acceleration sensor on the camera, you can use data from that for the blur kernel. It's nothing new really, but algorithms for it have advanced tremendously. For example, there's some results from 2009 http://www.youtube.com/watch?v=uqMW3OleLM4 http://www.youtube.com/watch?v=uqMW3OleLM4 Teuobk on HN also made a startup/app based on this, but it seems to be down now: http://news.ycombinator.com/item?id=2460887 http://news.ycombinator.com/item?id=2460887
- mey 15y agoDoes this algorithm lend itself to video? Probably would take a massive cloud of systems to correct any highdef video signal, but it would be impressive for many applications (news broadcast, sports, or any live event, security, or remote robots) Granted to achieve performance on the order of near realtime dsp, it would require an impressive hardware system. Then again, when I can spend the price of a coffee and get access to a cloud of cpu's...
- shabble 15y agoWithout having looked into it too much (mostly just the video), there seem to be fairly strong parallels with the motion tracking/estimation that modern video codecs do. As I understand it, their system is trying to estimate the causes of noise in the image (camera shake, badly focused or dirty lens, etc) and apply an inverse transformation to produce the original 'real' image. They can't extract detail out of nowhere, so a lot of it will likely be heuristically driven, accounting for specific sources of error and applying adaptive transforms with some sort of 'looks like what we think it should look like' cost function. Figuring out the cost function is probably one of the hardest parts here. In terms of video, you've got more data to work with, which in theory means more constraints to your solution making it potentially more accurate, but I suspect it'd also be very easy to get bogged down in the quantity of data. Combining and merging data from multiple independent sources is a different problem entirely, probably more like synthetic aperture imaging: http://vision.ucsd.edu/kriegman-grp/research/synthetic_ap_tracking/ http://vision.ucsd.edu/kriegman-grp/research/synthetic_ap_tr...
- waitwhat 15y agoHow is this different from what FocusMagic http://www.focusmagic.com/ http://www.focusmagic.com/ has been offering for over a decade?
- ck2 15y agoWe're obviously going to need many more independent samples to compare both.
- teuobk 15y agoFocusMagic handles only focus blur or linear motion blur, and either way, it requires a high level of user interaction to direct the deblurring. In effect, it is "non-blind" deconvolution. The Adobe approach, on the other hand, handles complex (non-linear) motion blur and does so in a so-called "blind" way.
- snikolov 15y agoIf you are wondering how they might be doing it, here is one approach that I saw in a computer vision class (no idea if they are doing anything similar to this) (slides: http://cs.nyu.edu/~fergus/presentations/fergus_deblurring.pdf http://cs.nyu.edu/~fergus/presentations/fergus_deblurring.pd... (~60 MB ppt) paper: http://cs.nyu.edu/~fergus/papers/deblur_fergus.pdf http://cs.nyu.edu/~fergus/papers/deblur_fergus.pdf (~10 MB pdf) ) The basic idea is that you have an unknown original image and it is convolved with an unknown blurring kernel to produce the observed image. It turns out that problem is ill-posed. You could have a bizzare original image blurred with just the right bizzare blurring kernel to produce the observed image. So to estimate both the original image and the kernel, you have to minimize the reconstruction error with respect to the observed image, while penalizing unlikely blurring kernels or original images. If one extracts enough statistics from a dataset of natural images, one can tell whether an image is likely or not by comparing that image's statistics to the corresponding statistics of your dataset of natural images. Similarly, simple blurring kernels are favored over complex ones (think "short arc of motion" vs. "tracing the letters of a word with your camera")
- chopsueyar 15y agoI assume the RedLaser barcode app uses something similar from captured video frames combined with data from the accelerometer, no?
- HardyLeung 15y agoI doubt it. Blind deconvolution is hard. I think they may use something like super resolution where multiple blurred pictures can be used to recreate a less blury picture (and this is significantly easier to do than blind deconvolution).
- Mvandenbergh 15y agoBarcodes are designed to be easy to decode. Even blurred, the 1 or 2 dimensional frequency information is pretty well preserved. If I needed to combine data from several blurred images of a barcode I'd extract the likely values of the barcode from each image separately and then combine those.
- aguynamedben 15y agoENHANCE http://www.youtube.com/watch?v=Vxq9yj2pVWk http://www.youtube.com/watch?v=Vxq9yj2pVWk
- beej71 15y agoIt leaves out a couple of my favs: Super Troopers: http://www.youtube.com/watch?v=KiqkclCJsZs http://www.youtube.com/watch?v=KiqkclCJsZs Red Dwarf: http://www.youtube.com/watch?v=KUFkb0d1kbU http://www.youtube.com/watch?v=KUFkb0d1kbU
- georgieporgie 15y agoSo, why did the whole video look really fake? It seemed to bob around in a very predictable manner. When the first sharpening took effect, it panned and zoomed exactly in time with the appearance of the second image. I'm not claiming the demo is fake, I'm just wondering why the video looks so strange.
- deleted 15y ago[deleted]
- bajsejohannes 15y agoI'm guessing it's a hand held camera with some stabilization algorithm in post processing.
- regularfry 15y agoThat plus the kinda fakey-looking screen made me think the whole thing was staged. Then again, if you were going to stage it, you could do a lot better than that camera motion without trying very hard.
- epo 15y agoAll first-look demos like this are staged/faked whatever, why would you think otherwise?
- georgieporgie 15y agoThe demo doesn't necessarily look fake, the video looks fake.
- StavrosK 15y agoI think it would be unreasonable to claim that it is fake anyway, since relevant research has been public for some time. If there's an amount of trickery involved, it's in the parameter files he loaded (they might be nontrivial/nonautomatic to produce).
- georgieporgie 15y ago
- goodweeds 15y agoIs this much different than the Lytro "focus later" camera? http://www.lytro.com/ http://www.lytro.com/. I don't know much about imaging, but I've been drooling over the demos I've seen online.
- joshu 15y agoCompletely different. This is a way to remove motion blur. Lytro is a way to move some of the focusing elements from physical to computational.
- shazam 15y agoWish they applied that algorithm on the video...
- hopeless 15y agoIndeed. And held the camera steady. I have motion sickness now, and I'm sitting at my desk :(
- TelmoMenezes 15y agoSo now we need blurring algorithms that cause actual information loss (I'm sure they already exist, but now there's suddenly a bigger market for them).
- hopeless 15y agoWe always did need those algorithms. There was a paper a few years ago on decoding gaussian-blurred documents to reveal the redacted passages. The only safe way is to completely remove the original pixels, e.g. by drawing a black box over the text.
- shabble 15y agoand not by just drawing that box over them in the original (textual content) pdf. You need to squash it down to images, then destroy the bits you need to. (There might be other ways, but enough Big Government Agencies have screwed it up to make it worth noting) Is the 'decoding' paper this one: http://dheera.net/projects/blur.php http://dheera.net/projects/blur.php ? The blur function is just smearing pixel values across their neighbours in blocks, so you can treat it as a hashing function, and then generate enough candidates that eventually you get something that hashes correct (or close enough) I'm not sure how practical it'd be on data longer than a credit-card number, but it's an interesting hack nevertheless.
- splicer 15y agoJust use content-aware fill in CS5.
- kondro 15y agoNow all Photoshop needs is an unCrash feature.
- daimyoyo 15y agoWhat they should do is partner with a DSLR maker and put this in the firmware of the camera itself. Imagine one button blur correction. That'd be amazing.
- MichaelApproved 15y agoI don't think DSLRs have the processing power needed to unblur the images. It took several seconds to unblur a section of an image. It'd probably be faster to upload the image to a cloud processor and have it unblur it for you.
- r00fus 15y agoThen perhaps it can be implemented on a newer Android version or in the (soon to arrive) big 10" Kindle Fire. This is a perfect "web service" where you can profit from selling the device to have free access to the service.
- slantyyz 15y agoIt'd probably be faster to upload the image to a cloud processor and have it unblur it for you. It might be more efficient to just teach people how to use their cameras so that they can minimize unwanted blurring.
- ibuildthings 15y agoOne of the tricky thrones in this method is extracting/guessing the camera motion path purely from image measurements. Better the estimate, the better the deblur kernel will be. What might be cool is if they can extract out meta-information using some kind of inertial and gyroscope-akin sensors (which are fast becoming standard in phones and cameras), which can supplement the motion path computation algorithm.
- kr1shna 15y agoNeed this for my wife. Will that camera's RAW format need to be compatible with this algorithm?
- pixcavator 15y agoHere's a relevant image: http://inperc.com/wiki/index.php?title=A_common_view_of_digital_imaging http://inperc.com/wiki/index.php?title=A_common_view_of_digi....
- dextorious 15y agoMy version of Photoshop had that feature for years. 1) Load image. 2) Filter -> Gaussian Blur 3) Undo ;-)
- natex 15y agoImagemagick can already use this "algorithm". See "fourier transform" applications such as: http://www.fmwconcepts.com/imagemagick/fourier_transforms/fourier.html#convolution_deconvolution http://www.fmwconcepts.com/imagemagick/fourier_transforms/fo...)
- teuobk 15y agoDeconvolution is easy. The main difficulty is estimating the blur kernel in a so-called "blind" manner. That's the advance being shown off by Adobe.
- natex 15y agoRight. It wasn't clear to me in the video that Adobe is using "blind" deconvolution. I did catch a glimpse of a small black square within a pure white field (on the right side palette in the video). I'm assuming that it is a spatial domain motion blur filter - variable by which the interface tweaks the effect.
- regularfry 15y agoI thought that was an output field, showing what motion it had estimated. Guess we'll have to wait until it's released to find out :-)
- Aloisius 15y agoCan this overcome some of the soft blurring media companies/journalists use to hide naughty bits and to protect identities?
- deleted 15y ago[deleted]
- KevinEldon 15y agoI've read a few of the very technical responses and they are great, but, for me, the take away was the audience response. It's exactly what I look for when I write software. I want that gasp, that moment where someone realizes they can do a hard thing much easier. Where they realize that they just got a few moments of time back.
- kstenerud 15y agoWell, it KINDA looked like stuff was being unblurred, but it's really hard to tell with the camera panning around out of focus. The only part I could really be sure was actually unblurred was the phone number.
- ooooooooooooo 15y ago