14 ms·
As a metrologist (and photographer), the difficulty with these techniques is that they can over-represent the information contained within an image; they presen
by ISL 6y ago
As a metrologist (and photographer), the difficulty with these techniques is that they can over-represent the information contained within an image; they present an image of what was "probably" there, rather than representing what was. These aren't so different from our own brains, which remember what we thought we saw, rather than the light that reached our retinas.
These methods are already in extensive use (most smartphone images use extensive noise-reduction techniques), but we must be ever-cognizant that image-processing techniques can add yet another layer of nuance and uncertainty when we try to understand an image.
- prox 6y agoWould it be right to say it is an synthesis on top of a analysis? It wasn’t what was observed. For some things it might not matter, but “it looks shopped” isn’t really a positive in my book. Although the use case in the article is pretty handy, to print stuff a lot larger.
- deleted 6y ago[deleted]
- social_quotient 6y agoIt’s like systems that add zeros (floating point math) to the right of the decimal. 18.00000 is not the same as 18. Thoughts?
- etrautmann 6y agoNo - 000000 is not based on a statistical model of what’s most likely to be right if the decimal place. In natural images - the statistical structure allows for this image upscaling but without revealing any previously hidden detail - just using know statistics of the world to show what might be there.
- gmfawcett 6y agoThat's not how floating point math works? At least not for standard floats (IEEE 754), and except for very large integers (near 2^m, where m is the mantissa of the FP type). Floats have an exact representation for integers within their mantissa range -- i.e., '18' is exactly the same as '18.0000'.
- dejj 6y agoYou probably want go with 0.1+0.2 != 0.3
- HenryBemis 6y agoI would say that it's like pixel's RGB at address 1x1 is 0-0-0 and pixel at address 1x2 is 0-0-2 and squeezing between them a pixel with color 0-0-1 (averaging the two values near it)(assuming doing this on a image that has 1 pixel height and e.g. 2 pixes width; so that the new image would be would be: 1x1 0-0-0 (original) 1x2 0-0-1 (made up) 1x3 0-0-2 (original)
- webmaven 6y agoWhat you're describing is relatively straightforward (bi)linear interpolation. It is worth noting that even at this relatively simple level, going with bicubic interpolation instead will usually give you nicer results, except in cases where the hard edges in the image are only horizontal or vertical.
- genericone 6y agoI dont think you mean for floating point, but for mechanical tolerances. Many times, you dont want to pay an extra $50,000 for the 5 digits of precision... but sometimes you do. Shitty system if it automatically messed up all your part tolerances.
- roughly 6y agoI think this point is worth pushing on a bit harder, which is to say that the "additional details" in the picture are guesses by the software, not actual additional details. The data present in the picture is fixed, the software uses that data to build educated guesses on what was actually there. If the photo doesn't contain enough data to actually determine what a given piece of text in the image says, the software can provide a guess, but it's just that, a guess. Similarly, if the photo doesn't provide enough detail to positively identify a person, the "super resolution" one cannot be used to positively identify them either, as it's a guess made from incomplete data, not genuinely new data. The point is worth belaboring because people have a tendency to take the output from these systems as Truth, and while they can be interesting and useful, they should not be used for things for which the truth has consequences without understanding their limitations. You're right to compare this to how our brains reconstruct our own memories, and the implications that has for eyewitness testimony should inform how we consider the outputs from these systems.
- TimTheTinker 6y agoThis “guessing” is nice for the sake of artistry, but we’ve got to be careful when knowing what actually was there is important—like when photos are submitted as evidence in court cases, or when determining the identity of a person from a photo as part of an investigation.
- smnrchrds 6y agoI hope such photos are submitted as camera takes them. With our without this new feature, photoshopping a photo before presenting it to court must be illegal.
- roughly 6y agoIf you consider photos taken by cell phones, it's hard to really say what "as the camera takes them" means - a lot of ML-driven retouching happens "automagically" with most modern cell phones already and I'd expect more in the future.
- sweezyjeezy 6y agoIf you're using this to try and enhance super grainy CCTV footage to get a face or license plate I'd agree. Purely in the context of this article, the author is just upscaling an already high-definition image 2x. There's very little artifice that can be really added at this level that a human could perceive IMO.
- jrockway 6y agoI think this is probably good for what people use photos for; it lets them show a crop without the image looking pixelated. That means if they just want a photo to draw you in to their blog post, they don't have to take a perfect photograph with the right lens and right composition at the right time. And I think that's fine. No new information is created by ML upscaling, but it will look just good enough to fade into the background. I personally take a lot of high resolution art photos. One that is deeply in my memory is a picture I took of the Manhattan bridge from the Brooklyn side with a 4x5 camera. I can get out the negative and view it under magnification and read the street signs across the river. (I would link you, but Google downrez'd all my photos, so the negatives are all I have.) ML upscaling probably won't let you do that, but on the other hand, it's probably pointless. It's not something that has a commercial use, it's just neat. If you want to know what the street signs on the FDR say, you can just look at Google Street View. (OK, maybe it does have some value. I used to work in an office that had pictures blown up to room-size used as wallpaper in conference rooms. It looked great, and satisfied my desire to get close and see every detail. But, you know you're taking that kind of picture in advance, and you use the right tools. You can rent a digital medium format camera. You can use film and get it drum scanned. But, for people that just need a picture for an article, fake upscaling is probably good enough. The picture isn't an art exhibit, or an attempt to collect visual data. It's just something to draw you into the article in the 3 milliseconds before you see a wall of text and bounce.)
- bscphil 6y ago> Google downrez'd all my photos, so the negatives are all I have Wow, Google ate your one digital copy? That's tragic. What's the approximate resolution you could get out of a scan from these labs? I was interested in getting into film cameras at one point, and I was disappointed with how low the scanning resolution is from most labs. For example mpix only advertises 18MB, which they say is good enough for a 12in by 18in print. North Coast Photo (what Ken Rockwell recommends) is even worse! What if you want something to put on the wall? Granted, if the original film you shoot is perfect you can have a print done from the negatives, but that kind of defeats the point of having a high quality scan as a backup.
- newobj 6y agoThis is perceptual/creative enhance. Not Blade Runner enhance.
- danaliv 6y ago> These aren't so different from our own brains, which remember what we thought we saw, rather than the light that reached our retinas. Never mind memories; there are parts of our eyes that aren’t responsive to light at all. We’re always hallucinating.
- deleted 6y ago[deleted]
- webmaven 6y ago> Never mind memories; there are parts of our eyes that aren’t responsive to light at all. We’re always hallucinating. Are you referring to the blind spot, or something else? Interestingly, the blind spot turns out not to be a design requirement, it is a contingent feature that cephalopods like octopuses (whose eyes evolved independently from vertebrates') don't have.
- dorkwood 6y agoAn example I saw getting traction on Twitter a few months ago was a photo of Melania Trump that was purported to be a body double. Since the original image was blurry, someone used an AI upscaler to "enhance" the photograph and increase the resolution. Then the comments started to roll in: the teeth are different! The tip of her nose doesn't match! It's not her! Technically, they were correct -- it wasn't her. It was an algorithm's best-guess reconstruction based on training data of other people's faces. Unfortunately, neither the original poster or anyone else in the thread seemed to grasp this concept.
- TeMPOraL 6y agoWhat worries me is that COTS photo equipment increasingly comes with these algorithmic retouches that "over-represent" the data - or, put another way, bake its own interpretation into image, in a way that cannot be distinguished from source data. It's nice for a casual Instagrammer, but then a lot of science and engineering also gets done using COTS equipment. I worry that at some point, a lot of money will be burned, a lot of time wasted, or even lives lost, because someone didn't notice they've based the conclusions of their scientific experiment/engineering analysis on such "computer best guesses". As a researcher, you'll see a weird pattern on some of the photos and will be left wondering, is that a real phenomenon, or is it just one of the black box, trade secret neural networks in the camera choking on input data it wasn't trained for?
- kamel3d 6y agoWell just don't use it
- neogodless 6y agoI did a web search for "cots" and learned that a cot is... > a small usually collapsible bed often of fabric stretched on a frame But in this case, COTS is apparently... > commercial, off-the-shelf In other words "photo equipment" or "consumer/retail photo equipment." Upon further reading[0] it seems odd to use the term here, but maybe I'm misunderstanding something. It's often used for software and has a key phrase... > packaged solutions which are then adapted to satisfy the needs of the purchasing organization But it's possible the term has been co-opted to mean something else now. [0] https://en.wikipedia.org/wiki/Commercial_off-the-shelf https://en.wikipedia.org/wiki/Commercial_off-the-shelf
- TeMPOraL 6y agoI use it in a way it's used in disciplines that also work with specialty-built, or even custom-built equipment. Such as science, military and some types of engineering (e.g. aerospace). The first sentence of the linked article describes it: "Commercial off-the-shelf or commercially available off-the-shelf[1] (COTS) products are packaged solutions[buzzword] which are then adapted to satisfy the needs of the purchasing organization, rather than the commissioning of custom-made, or bespoke, solutions." So for example, a research team may decide to not spend money on expensive scientific cameras for monitoring experiment, and instead opt to buy an expensive - but still much cheaper - DSLR sold to photographers, or strap a couple of iPhones 15 they found in the drawer (it's the future, they're all using iPhones 17, which is two generations behind the newest one). That's using COTS equipment. COTS is typically sold to less sophisticated users, but is often useful for less sophisticated needs of more sophisticated users too. But if COTS cameras start to accrue built-in algorithms that literally fake data, it may be a while before such researchers realize they're looking at photos where most of the pixels don't correspond to observable reality, in a complicated way they didn't expect.
- lumost 6y agoI take a lot of pictures of mountain scenery. I blew up one of these that had an interesting composition consisting of rocks/fields/mountain peaks. On inspection the resulting image had substantially changed the composition by increasing the size of a field relative to all other objects. It’s much easier for the model to blow up uninteresting pieces of the photograph than interesting pieces.
- atlantis_ 6y agoI have been using neural-enhance (gh:alexj) and Topaz tools to upscale PAL/NTSC artworks the last three years and would not be so judicious in describing these tools. They are hallucinating what the model assumes an upscaled image should look like and not enhancing in any way as the word is understood. The original image ceases to exist. A more honest term might be “Render As Upscaled” or “Generate Higher Resolution Image” (likewise “ML”, not “AI”). When playing around funny things happen too: recursive upscale/sharpen and analogue artifacts begin resembling topography, molten metal etc. Now imagine this being fitted into military drones, which it almost certainly is.