8 ms·
Super resolution is adding data that is not here, it's asking an algo to produce part of the art. At one point do we go from picture to painting ?
by BiteCode_dev 6y ago
Super resolution is adding data that is not here, it's asking an algo to produce part of the art.
At one point do we go from picture to painting ?
- alextheparrot 6y agoAnyone who edits photos in Lightroom understands that modern photography captures both reality and artistry. My goal when editing photos is to make them more clearly express how I felt or how I saw. This is often quite divorced from what shows up on the back screen of my camera.
- l-lousy 6y agoIt’s also very useful to be able to take a tighter crop of an existing image without worrying about pixelation
- perardi 6y agoI think photography has always been about artistry. You are taking a 2-dimensional slice of a 4-dimensional world. Interpretation is inherent. Ansel Adams was surely no stranger to post-processing. https://photofocus.com/photography/a-look-inside-ansel-adams-darkroom-magic/ https://photofocus.com/photography/a-look-inside-ansel-adams...
- wlesieutre 6y agoThat was true of film photography and development too
- pfortuny 6y agoA good experiment (which I cannot do): take a true photo of somebody, reduce it in size and then “super-resolve” it. How much must you reduce in order to produce a different “person” (or a non-person).
- Kelteseth 6y agoYou can also repeat this indefinitely: 1. Scale down 2. "Super-resolve" 3. Goto 1 In Germany, we have this game "stille post" where you whisper someone a phrase and then multiple children try to whisper the exact phrase to the next. Most of the time a completely different phrase comes out in the end.
- PEJOE 6y agoIn the US we also play the same game, but call it "Telephone."
- GeneralTspoon 6y agoAnd we call it "Chinese Whispers" in the UK
- dbrueck 6y agoYeah, although technically the goal behind things like 'super resolution' is to add data that is not there anymore but probably was there before. i.e. you had the original scene that was captured by a digital camera (a lossy operation) and then saved as an image file (often also a lossy operation), and then a tool like this makes an educated guess as to what information was lost in the 1st and 2nd steps.
- roywiggins 6y agoYou have to demosaic things anyway (as mentioned in the article) so it's not like you can escape algorithmic fudging of details.
- 6gvONxR4sf7o 6y agoPresumably you go from picture to painting the moment you open photoshop. Photography for record keeping/science and photography for aesthetics/artistry diverge long before you get to techniques like super resolution. Which is still a fuzzy boundary because anyone who has taken a picture including a sunny sky can tell you raw photos generally don't capture how it looks to your eyeballs.
- TehCorwiz 6y agoPhotography has always been more artistry than reality. From film selection, lens choice, framing and cropping, color correction, "dodging" and "burning" details in our out it's always been a compromise between what the camera sees (which itself may not be reality if you're using physical in-front-of-the-lens filters or other distortive techniques) and what the photographer want to convey. That's one of the reasons that Photographic Journalism often holds itself to ethical limitations with regards to how a photo can be manipulated after it's been taken. I'm almost certain that this kind of thing would at least walk that fine line if not fall right on over it.
- falcrist 6y agoOn one hand as Ansel Adams said "You don't take a photograph, you make it.", and that still holds true today. You're influencing the picture when you choose your camera, your lens, your sensitivity (ISO), your color mode, your white balance, your raw software, your editor, etc etc all the way from the camera to the print. On the other hand, there certainly is a difference between working with the information (pixels) you've captured, and inventing information by either drawing on the image or creating new data. This methodology falls squarely into the gray area between those two.
- diarrhea 6y agoExactly. That is also why claiming to only do "SOOC JPG" is ill-informed. People claiming it don't understand the process: to even produce a JPG, there has to be a process of interpretation already taking place. When I volunteered for a small newspaper I did my usual, sometimes significant, editing (Lightroom-level, not PS) and didn't see anything wrong with it (neither did they; of course edits look better than plain JPGs). But I can appreciate how this becomes much more important with increasing range. Imagine if Pete Souza spoiled 8 years' worth of Obama presidency imagery just because he edited them in some obscure way.
- mikepurvis 6y agoNever mind the manipulations which can occur even before the shutter is ever pressed— I'm having troubling finding links now, but I know there was controversy a few years ago about a picture of bombing rubble with an object (a dress form or something, maybe?) standing up in the middle of it, apparently by chance which created a very artistic contrast, but there were later questions about whether the photographer had in fact arranged the object in that way rather than discovering and capturing a preexisting scene. EDIT: Still can't find it, but here's a list which includes a number of other wartime photos which are proven or suspected to have been staged in various ways: https://militaryhistorynow.com/2015/09/25/famous-fakes-10-celebrated-wartime-photos-that-were-staged-edited-or-fabricated/ https://militaryhistorynow.com/2015/09/25/famous-fakes-10-ce...
- sweetheart 6y agoIt's always been more like painting than anything resembling a truthful representation of reality. In fact, the first ever photograph [1] much more closely resembles a painting than anything else, which is ironic. All advancements have simply given us more control in how painterly we render our photographs, but have never _really_ brought us closer to the truth. [1]: https://en.wikipedia.org/wiki/History_of_photography#/media/File:View_from_the_Window_at_Le_Gras_colorized_2020_new.png https://en.wikipedia.org/wiki/History_of_photography#/media/...
- sosuke 6y agoAnsel Adams answering a similar question https://youtu.be/Ml__B0l9GIs?t=1514 https://youtu.be/Ml__B0l9GIs?t=1514
- advisedwang 6y agoMy worry is when this gets used for something forensic. I imagine this will tend to reproduce things in the dataset, e.g. up-scaling blurry text may look like fonts that it has memorized more than the original. Or upscaling a feather will provide details like the feather of more common birds. Or upscaling blured out numbers will pick some numbers at random [1]. We need to make sure people don't rely on these details, e.g. in courts, HR reviews, when reddit sleuths try and investigate an incident, when someone looks for cheating partners etc. [1] https://www.theregister.com/2013/08/06/xerox_copier_flaw_means_dodgy_numbers_and_dangerous_designs/ https://www.theregister.com/2013/08/06/xerox_copier_flaw_mea...
- xadhominemx 6y agoI am not worried about it this at all. It’s not likely to randomly incriminate a real person.
- fsflover 6y agoYes, it is: https://news.ycombinator.com/item?id=26504144 https://news.ycombinator.com/item?id=26504144
- xadhominemx 6y agoOk that’s pretty far removed from a prosecutor relying on that image though
- sennight 6y agoI remember that time a guy's life got turned upside down because his fingerprints "matched" those found on a bomb. Despite the fact that he had no motive, disposition, or access - they were determined to convict, because to do otherwise would be to admit that fingerprint analysis doesn't enjoy the scientific foundation that DNA evidence does. So yeah, real people have been harmed by bad matching algorithms.
- speeder 6y agoI am from Brazil. On the first day of trials of deep-learning based facial recognition here, a random person was arrested because the algorithm confused that person with another one. Even more stupid, is that the person with "outstanding warrant" was actually ALREADY in prison. So yes, AI managed to arrest the same person, twice, one time the real person, one time a random look-alike.
- kevincox 6y agoIt is sort of. Their argument is that by taking into account the subpixel pattern of the sensor they can actually extract more detail than is readily visible in the picture. Basically you can imagine that the blue subpixel is always to the top-left of the pixel. If you shifted the blue down and right one half a pixel you would have a more "accurate" production. In this way you can add a new pixel with the blue value closer to the right spot, then interpolate the blue of the original. Of course you can also do logic such as detecting lines of different lightness and applying those on top. So yes, especially with their machine learning they are adding new detail, but that is also likely some detail that was already there, but could not be conveyed with with the lower resolution. I wonder how different this would be from the simple approach of realigning the subpixels on a higher-resolution image and interpolating the "missing" subpixels. This approach may look better but wouldn't add any data.
- ramraj07 6y agoNot true, there’s always far more data in an image than a quick glance can decipher, astronomy field obviously pioneered numerous methodologies over the decades (many of which biologists do more shittyly, speaking as a biologist who did so). In the end I’d argue a lot of these “super resolution” methods are just glorified dexonvolution, not that it’s a knock on these methods but apparently it’s not cool to call deconvolution deconvolution anymore.
- xadhominemx 6y agoNo these are not deconvolution at all. They are using AI to add information to the image.
- ramraj07 6y agoSuperresolution in consumer cameras is definitely adding detail to the image using prior information about the universe (hair has a pattern, what looks like an edge has to be sharper than what the image says, etc). This is definitely the questionable artsy aspect of this superres boom. But the question I answered was more specific which supposed how you can make up info which doesn’t exist, but that’s definitely not true either. Modern superres tech (especially the non-deep learning kind) can extract more info from the image if you systematically account for the psf of the camera, distortions etc.
- lmm 6y agoWhat makes you so confident that this "AI" doesn't end up just doing deconvolution?
- xadhominemx 6y agoBecause the images being enhanced are in focus
- nnmg 6y agoMost are, but in biology/physics STED[1] and STORM are physics based methods for overcoming the diffraction limit[2]. STED is pure physics, no math/deconvolution/AI tricks. [1] https://en.wikipedia.org/wiki/STED_microscopy https://en.wikipedia.org/wiki/STED_microscopy [2] https://en.wikipedia.org/wiki/Super-resolution_microscopy https://en.wikipedia.org/wiki/Super-resolution_microscopy