10 ms·
Adobe Photoshop's 'super resolution' made my jaw hit the floor
- afavour 6y agoI guess professional photos have always been touched up and this is just an automation of this process. But I've long felt a little odd about the use of machine learning in photography this way. How long before Google Photos recreates dark photos by just reassembling all of the items it believes are there from machine learning?
- medlyyy 6y agoI mean, the only reason things like this aren't commonplace is cost (of the skillset, tools). Basically anything's possible these days with CGI, everything is purely a matter of the amount of effort you want to put in. And for artistic purposes, why does it matter how the final result was arrived at? If we have powerful and easy techniques for realising an artistic vision, that doesn't seem like a bad thing?
- kristiandupont 6y ago...I guess the enhance jokes were just rendered void! https://www.youtube.com/watch?v=Vxq9yj2pVWk https://www.youtube.com/watch?v=Vxq9yj2pVWk
- qayxc 6y agoAbsolutely not. If there's not enough information available, there's not enough information, full stop. Plausible (i.e. "good looking" or "believable") results are not the same as actual data, which is why enhance wouldn't work on vehicle licence plates or faces for example. Sure, the result might be a plausible looking face or text, but it's still not a valid representation of what was originally captured. That's the danger with using such methods for extracting actual information - it looks fine and is suitable for decorative purposes, but nothing else.
- nwienert 6y agoNo there certainly is a chance for ML to improve here. Let’s take the classic example of enhancing a blurry photo to get a license plate. Humans may not be able to see much in the blur, but an AI trained on many different highly down-res’d images could at least give you plausible outcomes using far less data than a human brain would be able to say anything with confidence. You wouldn’t hold it up as the absolute truth, but you’d run the potential plate and see if it matched some other data you have. So yes, it wouldn’t magically add any more information to the image, but it could be far better at taking low information and giving plausible outcomes that are then necessary to verify.
- hojjat12000 6y agoActually the opposite. These algorithms are more susceptible to noise, they may generate sharp perfect license plate numbers (that are totally fabricated and completely wrong) from a blurry image. But by no means should you even consider the results to have hints of truth. GAN produces totally different results if you slightly change the input. So, as others are also saying, these "enhances" are great for decoration and absolutely should be ignored as facts or truth (specially when it comes to face and license plate and others used by the law enforcement).
- nwienert 6y agoNo. If their training set isn’t too far off what you use it for, it is valid. Just because it’s not guaranteed, doesn’t mean it’s not more accurate than hitting sharpen and squinting. You’re fighting against “would it be reliable” but that isn’t the claim. The claim is could it be better than human, and the answer is yes, it just depends on how well trained it is and the dataset. But this is also entirely testable. I guarantee much like Go, if we set up a “human vs AI guess the blurry image” competition that AI will blow us out of the water. It’s simply a data * training issue, and humans don’t spend hours on end practicing enhancing images like they do playing Chess. Again - it won’t be perfect, obviously. It will have false positives, of course. Doesn’t mean it can’t be better than human. Also GANs are pretty irrelevant, the model structure has nothing to do with the theory.
- deleted 6y ago[deleted]
- endisneigh 6y agoI’m not sure if that’s the case with this tech. I could see in the near future a scenario in which many, many individuals (thousands) are photographing the same things in the same area and you can intelligently superimpose things to “enhance”, tho.
- benlumen 6y agoCSI and the like were just ahead of their time edit: I was joking, but people pointing out that you still can't create something out of nothing etc might not be thinking big enough. I think this technology absolutely has the potential to help. police are literally still using artists impressions - photofits, to find perpetrators
- hojjat12000 6y agoI think the artist impression has a lot more value than a highly realistic generated face. If you see an artistic impression, you will see the facial features that were noticeable. Such as a mole, the shape of the nose, or the thickness of the eyebrows. Then you have a template that your brain uses to match those features with any face that you see. However, if I show you a highly realistic face, your brain will take a different impression. Your brain is trained on faces for thousands of years. It will try to match the face perfectly. An artist impression tells the audience that it is inaccurate. A realistic photo tells the audience that this is _exactly_ who we are looking for.
- medlyyy 6y agoYep. To be useful for exploring potential "true" values, a system would probably need some way of showing you the distribution of its guesses, so you can get an idea of whether there is any significant information there. That aside, you'd still probably need a ML PhD to have a chance of correctly interpreting the results, given the myriad potential issues with current systems.
- tyingq 6y agoThis demo is fun to see the ML take different guesses at how to inpaint the missing data: http://www.chuanxiaz.com/project/pluralistic/ http://www.chuanxiaz.com/project/pluralistic/
- anonymfus 6y agoBy the way here is an enhanced version of the same supercut from the same person: https://www.youtube.com/watch?v=LhF_56SxrGk https://www.youtube.com/watch?v=LhF_56SxrGk
- qayxc 6y agoHm. So I took the example image, upscaled by 200%, applied a sharpen filter (all in Paint.NET) and compared the result to the AI upscaled image. TBH, I couldn't see a difference. 2x upscaling isn't all that impressive to begin with (e.g. produce 4 pixels from 1) and can be done in fairly high quality using traditional non-learning algorithms. I'm much more impressed by 4x and 8x super-resolution. I'm really not sure what the big deal is with 2x.
- formerly_proven 6y agoI concur, the second surfer example - the enhanced image looks a lot like blowing up the original 2x and applying careful sharpening.
- afavour 6y ago> and applying careful sharpening That alone makes it worthwhile, surely? No careful application required.
- ShamelessC 6y agoFor sure, but the title is quite an overstatement and reads like it's from someone who haven't really been paying attention to the many existing open source super resolution offerings.
- FpUser 6y agoSame here. Playing a bit with CLAHE, microcontrast and sharpening gives visually the same if not even better results.
- crazygringo 6y agoCame here to say the same thing. I was expecting the upscaled image to have extra "invented" detail from the supposed ML, as I've seen elsewhere. But looking at these upscaled images, there isn't any at all. There's no extra texture, nothing. I can't find any difference at all, like you say, from just some bicubic interpolation with sharpening. No jaw dropping here, unfortunately.
- deleted 6y ago[deleted]
- ISL 6y agoAs a metrologist (and photographer), the difficulty with these techniques is that they can over-represent the information contained within an image; they present an image of what was "probably" there, rather than representing what was. These aren't so different from our own brains, which remember what we thought we saw, rather than the light that reached our retinas. These methods are already in extensive use (most smartphone images use extensive noise-reduction techniques), but we must be ever-cognizant that image-processing techniques can add yet another layer of nuance and uncertainty when we try to understand an image.
- prox 6y agoWould it be right to say it is an synthesis on top of a analysis? It wasn’t what was observed. For some things it might not matter, but “it looks shopped” isn’t really a positive in my book. Although the use case in the article is pretty handy, to print stuff a lot larger.
- deleted 6y ago[deleted]
- social_quotient 6y agoIt’s like systems that add zeros (floating point math) to the right of the decimal. 18.00000 is not the same as 18. Thoughts?
- etrautmann 6y agoNo - 000000 is not based on a statistical model of what’s most likely to be right if the decimal place. In natural images - the statistical structure allows for this image upscaling but without revealing any previously hidden detail - just using know statistics of the world to show what might be there.
- gmfawcett 6y agoThat's not how floating point math works? At least not for standard floats (IEEE 754), and except for very large integers (near 2^m, where m is the mantissa of the FP type). Floats have an exact representation for integers within their mantissa range -- i.e., '18' is exactly the same as '18.0000'.
- oblio 6y agoI like how he realizes the impact for pro cameras but doesn't highlight the elephant in the room: phone cameras. This means that soon for many people digital cameras outside of their smartphone will become an even more niche product.
- t-writescode 6y agoThis is already the case. I rarely have a need to take out my SLR - it's just too bulky to have a reason for it, unless I'm going on an adventure where photography is one of the or the main purpose. I've gone on hiking trips where my "challenge" was to only use my phone camera. It wasn't much of a challenge for landscapes.
- HenryBemis 6y agoMost people are like that. But when I go for a 'photowalk' I cannot imagine not using my (#1) DSLR (or my (#2) super-duper zoom point-and-shoot camera). Phone (imho) is for quick and dirty, not for a 'it's time to do proper photography'.
- dingaling 6y ago> It wasn't much of a challenge for landscapes. Well if you collapse the problem space to a single point that corresponds to a phone's standard field of view, then it won't be a problem... But what if you wanted to catch a photo of a rare bird in flight at 500mm equivalent, or a surfer caught at 1/4000th of a second?
- t-writescode 6y agoBird-watching and action photography are both niche activities, I think, yes? :)
- agumonkey 6y agoI always expected smartphones already used approximation of SR to compensate for lower quality optics / ICs.
- lprd 6y agoNeat. If only Adobe would do away with their absurd pricing models. I'll never use an adobe product again after trying to end my subscription with them.
- Grakel 6y agoAbsolutely. And the results shown in this article aren't particularly impressive. I'll be sticking with photopea.com, even though I get free CC through work.
- lprd 6y agoCreative Cloud was the point I noticed a shift in Adobe's priorities. I don't know if they switched CEO's at the time, but I starting disliking Adobe more and more from that point forward. I couldn't believe the amount of crud Creative Cloud puts on your system, not the mention all of the tracking and phoning home their software does.
- wlesieutre 6y agoIf you're on a Mac, Pixelmator Pro is a $40 purchase with a similar feature https://www.pixelmator.com/blog/2019/12/17/all-about-the-new-ml-super-resolution-feature-in-pixelmator-pro/ https://www.pixelmator.com/blog/2019/12/17/all-about-the-new...
- Black101 6y agoYeah, a monthly payment forever is ridiculous...
- marcodiego 6y agoHow about a less clickbaity title?
- zarmin 6y agooh man, wait til you see the rest of the internet.
- frereubu 6y agoPixelmator did this a while ago (December 2019): https://www.pixelmator.com/blog/2019/12/17/all-about-the-new-ml-super-resolution-feature-in-pixelmator-pro/ https://www.pixelmator.com/blog/2019/12/17/all-about-the-new... Can't comment on the quality of either because I haven't used them, but they both seem to go to 200%.
- nerbert 6y agoI use it regularly (for my personal photos) and it depends on the photo. My observation is that pictures of natural elements (clouds, water, stars) tend to yield better results than for example a family picture in my house.
- marcodiego 6y agoThe samples on the article show very good results at preserving details so that curves do not get blurry when scaled up but are not particularly impressive. Off topic: I remember a few years ago, some students got very impressed by GIMP's Lanczos-3 upscaling that was much better than the photoshop version they had access at the time.
- PaulHoule 6y agoThe 'Preserve Details 2.0' upscaler from photoshop does an amazingly good job, in particularly I started with a 500x500 square image of a gundam sketch illustration which showed scan lines when printed directly on a 8 inch square but with 4x4 scaling the image was close to perfect.
- uniqueid 6y agoI thought it was interesting when Pixelmator introduced 'ML Super Resolution' but that was in 2019. https://9to5mac.com/2019/12/17/pixelmator-pro-super-resolution https://9to5mac.com/2019/12/17/pixelmator-pro-super-resoluti...
- deleted 6y ago[deleted]
- robk 6y agoTopaz Gigapixel is quite good though despite the negative comments in the article. I'd rather give my money to Topaz for this one feature than keep paying Adobe subscription fees indefinitely
- AmVess 6y agoI've been using it for a year. Each version gets a little better than the last and it gets regular updates. It also has the rather large benefit of not being tied to Creative Cloud, which is one of the worst bits of software ever made.
- kristianp 6y ago"worst bits of software ever made"? That's harsh. But didn't they lose people's data recently? My memory is blurry.
- ValentineC 6y agoI don't know about losing data, but if you install the base Creative Cloud application on Mac, a whole bunch of processes run in the background that can't be easily terminated.
- CharlesW 6y agoBecause they're David to Adobe's Goliath, I also feel compelled to mention that I've just recently discovered/purchased this and am incredibly impressed with it.
- covercash 6y agoLook into Cupscale, it’s a GUI for all of the ESRGAN models. https://github.com/n00mkrad/cupscale https://github.com/n00mkrad/cupscale
- noahatwi 6y agoAlso Waifu2X (don't be put off by the name). https://github.com/AaronFeng753/Waifu2x-Extension-GUI https://github.com/AaronFeng753/Waifu2x-Extension-GUI It has several models for processing animation and photos. If you want it even tediously automates the scaling of videos and GIFs by exporting each frame, upscaling, and joining back together.
- tomc1985 6y agoA non-tech muggle's jaw hitting the floor is practically par for course. I'm so tired of reading these breathless assessments from people who don't know any better
- spion 6y agoSo what happens if you take a tiny 10x10 image and run it through "super resolution" about 8-10 times?
- sweezyjeezy 6y agohttps://openaccess.thecvf.com/content_cvpr_2018/papers/Yu_Super-Resolving_Very_Low-Resolution_CVPR_2018_paper.pdf https://openaccess.thecvf.com/content_cvpr_2018/papers/Yu_Su... stuff like this - assuming it's a GAN under the hood it just tries to guess a 'plausible' possible interpolation, but if you're giving it very little information about what's in the original image, there will be a wide range of plausible images it could have arisen from, so the output can be very far from the truth.
- intricatedetail 6y agoI hate articles where author shows an option but won't actually tell where it is located in the application. I spent 10 minutes looking for it in latest PS and couldn't find it. Then I clicked at link to related article about "Enhance Details" and it seems like the option could be in Lightroom instead? I tried to use it myself because the illustrations in the article don't look to impressive, but authors enthusiasm got me to look for it.
- perl4ever 6y agoWhat I felt was missing was a "conventional" enlargement with the previous best algorithm side by side with the AI one.
- hansoolo 6y agoIt said "Adobe camera raw" from the beginning and even had a screenie showing the menu point...
- intricatedetail 6y agoBut how do I open this Adobe camera raw thing?
- jl6 6y agoIs it just me who found the article immensely frustrating for lacking same-region-crop side by side comparisons?
- 0-_-0 6y agoExactly, an article about superresolution doesn't atually show what superresolution is doing, despite the terribly clickbait-y headline.
- ghshephard 6y agoI don't even understand why someone would use a headline like "Jaw hit the floor" without even bothering to share the two images. It's not like Adobe Photoshop doesn't have the ability to export images...
- lupire 6y agoAdobe paid to promote their stuff
- rrrrrrrrrrrryan 6y agoNo, it's not just you. An overlay with a slider is pretty much the standard way of comparing two near-identical images these days, but this article not even having a side-by-side is just downright lazy.
- xeromal 6y agoLeave it to hackernews to shit on an article a person wrote for free and call them lazy.
- mcrutcher 6y agoThis seems highly relevant as to what is actually going on: http://www.johncostella.com/magic/ http://www.johncostella.com/magic/
- zamadatix 6y agoFor the tl;dr the bit at the end (and linked paper) can cover the topic without the backstory if that's not your sort of thing: "As noted above, in 2021 I analytically derived the Fourier transform of the Magic Kernel in closed form, and found, incredulously, that it is simply the cube of the sinc function. This implies that the Magic Kernel is just the rectangular window function convolved with itself twice—which, in retrospect, is completely obvious. This observation, together with a precise definition of the requirement of the Sharp kernel, allowed me to obtain an analytical expression for the exact Sharp kernel, and hence also for the exact Magic Kernel Sharp kernel, which I recognized is just the third in a sequence of fundamental resizing kernels. These findings allowed me to explicitly show why Magic Kernel Sharp is superior to any of the Lanczos kernels. It also allowed me to derive further members of this fundamental sequence of kernels, in particular the sixth member, which has the same computational efficiency as Lanczos-3, but has far superior properties."
- dimatura 6y agoThat's an interesting article, and relevant in the sense that the "magic kernel" can be used for purposes of super-resolution, but Adobe is using a fairly different approach. Instead of using analytically-derived functions Adobe is using a deep learning model trained on a large dataset of Low resolution-High resolution image pairs. The details are propietary, obviously, but it's likely similar to various deep learning superresolution algorithms in the academic literature. (Some more info here https://blog.adobe.com/en/publish/2021/03/10/from-the-acr-team-super-resolution.html#gs.w4oanu https://blog.adobe.com/en/publish/2021/03/10/from-the-acr-te...)
- NoPicklez 6y agoThis sounds like how nvidia have implemented Deep Learning Super Sampling (DLSS) into computer graphics cards for gaming. Allowing people to run at higher resolutions (e.g. 4k) and in some cases the image looks better than native.
- Causality1 6y agoAny chance of Adobe licensing this tech out? I can see it making one hell of a difference when it comes to zoomed-in pictures on phones.
- deleted 6y ago[deleted]
- Waterluvian 6y agoDoes the ML algorithm with the millions of images training set get run locally or as a remote service? When I use ML these days is it all hardcore data crunching by remote servers or is some of it running on my phone/laptop?
- Invictus0 6y agoThe model is trained on Adobe servers and run locally on your device. The training of the model is much more processor intensive than actually utilizing the trained model, usually by multiple orders of magnitude.
- Waterluvian 6y agoAh okay thanks. So any model that can be trained for generic use (Eg. Not trying to deep fake my specific face) can presumably be run on local machines. Thanks!
- megablast 6y agoDoes it run on the device, or adobe servers???
- Someone 6y agoI suggest to read the Adobe blogpost at https://blog.adobe.com/en/publish/2021/03/10/from-the-acr-team-super-resolution.html#gs.w3bka2 https://blog.adobe.com/en/publish/2021/03/10/from-the-acr-te... instead. It has sample images side-by-side with bicubic upsampling. Even better comparisons are in the blog post for a competing product: https://www.pixelmator.com/blog/2019/12/17/all-about-the-new-ml-super-resolution-feature-in-pixelmator-pro/ https://www.pixelmator.com/blog/2019/12/17/all-about-the-new... (likely the same algorithm, but using a different training set, so results will be different from what Adobes product does). It has comparisons with nearest neighbor, bilinear and Lanczos filters and uses a slider to make it easier to see the difference. Papers on this task: https://paperswithcode.com/task/image-super-resolution https://paperswithcode.com/task/image-super-resolution
- Judgmentality 6y ago> Even better comparisons are in the blog post for a competing product: https://www.pixelmator.com/blog/2019/12/17/all-about-the-new https://www.pixelmator.com/blog/2019/12/17/all-about-the-new... (likely the same algorithm, but using a different training set, so results will be different from what Adobes product does). Aren't those comparisons misleading though? The ML sample is 4x the resolution, comparing to something that is supposed to be used to also make something 4x the resolution. They aren't upsampling the comparison, so I don't know what they're actually doing with it. It's so misleading I just assume the company is scummy.
- TimBurr 6y agoA few examples in the blog post stood out to me - both refine ambigous letters to clear ones, which demonstrates concerns other folks were talking about in this thread. In this one, i's are nearly illegible, but are corrected. It appears to be correct here, but could be wrong. https://blog.adobe.com/hlx_fa6ee5a67d3f2e5b1945745176e6b955efc20a47.jpeg https://blog.adobe.com/hlx_fa6ee5a67d3f2e5b1945745176e6b955e... This one's weirder. In the original, I can't read many of the letters. It looks like G[EL]NE[XR]A[IL]DI[ER][XK][IT]OR - lots of guesses. The enhanced version is GENERALDIREKTOR (I think). At any rate, it's much more confident in the spelling than I am, as a human. https://blog.adobe.com/hlx_a337d462293b51f4d91182cbbebf68cf4fd7613e.jpeg https://blog.adobe.com/hlx_a337d462293b51f4d91182cbbebf68cf4...
- signal11 6y agoFor those wanting to try this out without paying for Creative Suite, Pixelmator Pro on the Mac ($40) has something similar[1]. The iPad version ($8) also has this feature now[2]. [1] https://www.pixelmator.com/blog/2019/12/17/all-about-the-new-ml-super-resolution-feature-in-pixelmator-pro/ https://www.pixelmator.com/blog/2019/12/17/all-about-the-new... [2] https://www.pixelmator.com/blog/2020/09/15/pixelmator-photo-major-update-brings-ml-super-resolution-to-ipad/ https://www.pixelmator.com/blog/2020/09/15/pixelmator-photo-...
- bscphil 6y agoIf it's just a matter of "trying it out", Adobe's Creative Cloud has a free trial already, is only $10 a month normally (IIRC), and has the advantage of running on Windows. Edit: the photography bundle, which includes Photoshop, is $10/month, not Creative Cloud as a whole. Thanks to mastazi.
- mastazi 6y agoJust to clarify, the one that costs US $9.99 per month is the Photography bundle (Lightroom and Photoshop). Just adding this as there are many different Creative Cloud subscriptions with monthly prices as high as US $52 https://www.adobe.com/creativecloud/plans.html https://www.adobe.com/creativecloud/plans.html
- bscphil 6y agoThat's right, thanks, I'd forgotten. That said, Photoshop is all you'd need to test the feature here.
- mastazi 6y agoYes, and confusingly, as I just discovered from the link I posted above, Photoshop alone costs more than the Photography bundle! I didn't realise that at first, I was planning to get Photoshop by itself, so it seems that your comment just ended up saving me a few bucks :-D
- 6y ago
- peterfield 6y agoIs that Ryan Gosling? There's definitely something going on there. https://petapixel.com/2020/08/17/gigapixel-ai-accidentally-added-ryan-goslings-face-to-this-photo/ https://petapixel.com/2020/08/17/gigapixel-ai-accidentally-a...
- aimor 6y agoI wanted a different comparison, so I upscaled without filtering and cropped one of the images to show the same region. https://ibb.co/8gTKTtr https://ibb.co/8gTKTtr https://ibb.co/9H0TCwh https://ibb.co/9H0TCwh
- deleted 6y ago[deleted]
- mm983 6y agothis isn’t anything new, there have been projects like this, including guis, around for years in github
- tambourine_man 6y agoIt's a very hit and miss feature, like most AI enhance stuff. Often enough it will look worse than bicubic interpolation. Chroma noise get's crazy most of the time. What's remarkable at: pattern cloth, straight lines with large contrast (electric wires against a blue sky, dark glasses mount against white skin, etc). Nice to have option all in all.
- serguzest 6y agoPossible practical application: can anyone upscale the beatles abbey road cover to a higher resolution please? I've always wanted it as a wall poster.
- de6u99er 6y agoIsn't this all (NVifia DLSS, Adobe SuperResolution) based on Google's Android camera work?
- smcleod 6y agoSounds pretty similar to the ML Super Resolution feature Pixelmator Pro has had for some time (highly recommend the product too).
- globular-toast 6y agoWhat's the point? If display resolution was way higher than capture resolution than I could see it. But the opposite is true. A 4K display is 10 megapixels, most entry level cameras are in the region of 20-25mp.
- aembleton 6y agoYou might want to crop in to that photo.
- npteljes 6y agoIn case anyone interested, vas3k has a very good quality write-up of similar advances made in the field: https://vas3k.com/blog/computational_photography/ https://vas3k.com/blog/computational_photography/ (even mentioning the related ML Enhance feature in Pixelmator)
- Klwohu 6y agoI was BLOWN AWAY by this STUNNING new technology! The brilliant minds at Adobe have done it yet again. This changes everything. Adobe (ticker symbol ADBE) keeps innovating and defining the very future of creative imaging software. From the widely loved Cloud platform and easy rental model to the file formats which are so packed with features that each new update takes the competition possibly years to fully reverse engineer for interoperability. Probably because their engineers aren’t nearly as good! Yes my jaw hit the floor when I saw this headline.
- noahatwi 6y agoMany github projects do this for free: https://github.com/AaronFeng753/Waifu2x-Extension-GUI https://github.com/AaronFeng753/Waifu2x-Extension-GUI
- mikewarot 6y agoThe other path to SuperResolution is to take multiple images, keeping track of camera orientation and rotation to stabilize and then merge the multiple exposures into a single image with much more real image information than the native resolution of the imager. The only physical requirements are that the camera not be absolutely stable, and the subject not be moving. You can get on the order of sqrt(N) improvement in resolution from N images with an optimized system. I've done it in the past, with a hand held DSLR and Hugin (the panorama stitching program, in this case used to align the stack of almost identical pictures with subpixel accuracy)
- HotVector 6y agoIs it common practice now to take open source techniques and ship them as proprietary software? I'm seeing a lot of Photoshop tools which I just saw in Two Minute Papers a couple months ago...