3 ms·

It used to be that (rational) people would be fairly in agreement that one cannot extract information which is not there. That's why we laugh at the "zoom -> 9

by dusted 2y ago

It used to be that (rational) people would be fairly in agreement that one cannot extract information which is not there. That's why we laugh at the "zoom -> 9 pixels -> enhance -> clear picture -> zoom ->.." trop in movies.. AI does not change this.. It adds stuff that's not there, sure, it adds stuff that might be there, or could have been there, but it literally cannot know, so it just paints in plausibility.. Which is horrible and gross already in principle. I imagine we're not far from the first conviction based on AI enhanced images.. give the model a blurry CCTV frame and a list of suspects and it will enhance the image until it looks like one of them, and since people are apparently this stupid, someone is going to be locked up for something they didn't do. Back to movies, just, fuck no! There's no enhancements to make, the movie is _DONE_, it looks like it does, leave it.

dml2135 2y ago
It's frustrating that the article doesn't mention this relatively simple, well understood, and very relevant principle. (Unless I just missed it somewhere, someone please correct me if I'm wrong).
Mahnahnohnah 2y ago
Thats not true. When you have moving film, you have 24fps from the same scene, a lens and a compression algorithm. There is a higher chance for pixel a to be color y if over a span of x frames and compression artifact this pixel shows value z. You will also have the chance to track details visible from frames/stills/pictures from seconds or even minutes ago if the actor just has the same cloth. And you can estimate the face of the actor across the movie and create an inherant face. Nonetheless, besides this type of upscaling, if an AI is trained to upscale based on probability of the real world and the movie is from the real world, it is still more than just random. Btw. there have been plenty of movie makers making movies the way they did because thats the only way they were able to make it. Low light high end camera equipment is expensive. And if you watch The Matrix without editing on a oled 60" and higher, it looks off because greenscreen details are more noticable than it has been before. Its aboslutly valid to revisit original material to adjust it to today (at least for me).
pimlottc 2y ago
There’s also the laws of large numbers here. When you make hundreds and thousands of guesses, even with really high accuracy, you’re going to get some wrong. For a 4K movie at 24 frames a second, you’re talking millions of guesses per minute, perhaps billions or more over the length of an entire film. It’s inevitable you’re going to get weird glitches, artifacts, or just parts that look “off”. It’s better to use a light touch for techniques like this rather just blindly applying it to the entire film.
- antifa 2y ago
  I wonder how effective some laser focused AI would be for fixing really bad CGI.
shepherdjerred 2y ago
> AI does not change this.. It adds stuff that's not there, sure, it adds stuff that might be there, or could have been there, but it literally cannot know, so it just paints in plausibility.. Which is horrible and gross already in principle. That's essentially how the brain works, too.
- Free-is-Freedom 2y ago
  The difference is that people can express why they made a decision
yreg 2y ago
> That's why we laugh at the "zoom -> 9 pixels -> enhance -> clear picture -> zoom ->.." trope in movies.. > AI does not change this.. It kind of changes it in some cases. The information might be there, but superhuman pattern recognition might be needed to extract it. And of course, in case factuality doesn't matter, the missing information can be generated and filled in. This obviously doesn't work when you are looking for a terrorist's license plate or when you want to watch an original performance in a movie.
- dml2135 2y ago
  "Information" in terms of, what does this thing look like -- could maybe be determined from other shots -- yes, sure. But I think "information" in the context of film here refers to the indexical mark of light upon the image sensor, and in that case no. If it's not recorded, you can't extract it. And whatever you do put there is of little interest to the film buff to whom "image quality" means a more faithful reproduction of the negative that was seen in theaters.
  - yreg 2y ago
    I'm talking in the context of the quoted statement >"zoom -> 9 pixels -> enhance -> clear picture -> zoom ->.." trope in movies You can have e.g. a picture of a blurry piece of paper, that humans can't read, but I imagine software might be able to read it (with reasonable accuracy and consistency). The information might be recorded, but hidden.
    - pimlottc 2y ago
      That’s not really true though. Unless you’re talking about a trivial distortion, like inverting the colors, there’s always some loss of information. In the case of blurry text, we’re still making an assumption that the paper holds some form of human writing, and not literally a blurry pattern. Maybe there’s external context that confirms this. But solely based on the image itself, you can’t know this. It’s basically a hash function; there are multiple possible “source” images of what’s on the paper that may end up looking exactly the same on the blurry/low-res/degraded etc output video. Human readable text is likely the most plausible but it’s not 100%. You can’t reverse an operation that loses information with absolute certainty unless you are using other factors to constrain the possible inputs.
r_c_a_d_t_s 2y ago
I agree. "AI-enhanced" is just the latest badge to put on content in the hope that some suckers will pay more money for the "same" content they already have. There's no striving for a better version here, just a drive for more profit.

by dusted 2y ago

dml2135 2y ago

It's frustrating that the article doesn't mention this relatively simple, well understood, and very relevant principle. (Unless I just missed it somewhere, someone please correct me if I'm wrong).

Mahnahnohnah 2y ago

Thats not true. When you have moving film, you have 24fps from the same scene, a lens and a compression algorithm. There is a higher chance for pixel a to be color y if over a span of x frames and compression artifact this pixel shows value z. You will also have the chance to track details visible from frames/stills/pictures from seconds or even minutes ago if the actor just has the same cloth. And you can estimate the face of the actor across the movie and create an inherant face. Nonetheless, besides this type of upscaling, if an AI is trained to upscale based on probability of the real world and the movie is from the real world, it is still more than just random. Btw. there have been plenty of movie makers making movies the way they did because thats the only way they were able to make it. Low light high end camera equipment is expensive. And if you watch The Matrix without editing on a oled 60" and higher, it looks off because greenscreen details are more noticable than it has been before. Its aboslutly valid to revisit original material to adjust it to today (at least for me).

pimlottc 2y ago

There’s also the laws of large numbers here. When you make hundreds and thousands of guesses, even with really high accuracy, you’re going to get some wrong. For a 4K movie at 24 frames a second, you’re talking millions of guesses per minute, perhaps billions or more over the length of an entire film. It’s inevitable you’re going to get weird glitches, artifacts, or just parts that look “off”. It’s better to use a light touch for techniques like this rather just blindly applying it to the entire film.

antifa 2y ago
I wonder how effective some laser focused AI would be for fixing really bad CGI.

shepherdjerred 2y ago

> AI does not change this.. It adds stuff that's not there, sure, it adds stuff that might be there, or could have been there, but it literally cannot know, so it just paints in plausibility.. Which is horrible and gross already in principle. That's essentially how the brain works, too.

Free-is-Freedom 2y ago
The difference is that people can express why they made a decision

yreg 2y ago

> That's why we laugh at the "zoom -> 9 pixels -> enhance -> clear picture -> zoom ->.." trope in movies.. > AI does not change this.. It kind of changes it in some cases. The information might be there, but superhuman pattern recognition might be needed to extract it. And of course, in case factuality doesn't matter, the missing information can be generated and filled in. This obviously doesn't work when you are looking for a terrorist's license plate or when you want to watch an original performance in a movie.

dml2135 2y ago
"Information" in terms of, what does this thing look like -- could maybe be determined from other shots -- yes, sure. But I think "information" in the context of film here refers to the indexical mark of light upon the image sensor, and in that case no. If it's not recorded, you can't extract it. And whatever you do put there is of little interest to the film buff to whom "image quality" means a more faithful reproduction of the negative that was seen in theaters.
- yreg 2y ago
  I'm talking in the context of the quoted statement >"zoom -> 9 pixels -> enhance -> clear picture -> zoom ->.." trope in movies You can have e.g. a picture of a blurry piece of paper, that humans can't read, but I imagine software might be able to read it (with reasonable accuracy and consistency). The information might be recorded, but hidden.
  - pimlottc 2y ago
    That’s not really true though. Unless you’re talking about a trivial distortion, like inverting the colors, there’s always some loss of information. In the case of blurry text, we’re still making an assumption that the paper holds some form of human writing, and not literally a blurry pattern. Maybe there’s external context that confirms this. But solely based on the image itself, you can’t know this. It’s basically a hash function; there are multiple possible “source” images of what’s on the paper that may end up looking exactly the same on the blurry/low-res/degraded etc output video. Human readable text is likely the most plausible but it’s not 100%. You can’t reverse an operation that loses information with absolute certainty unless you are using other factors to constrain the possible inputs.

r_c_a_d_t_s 2y ago

I agree. "AI-enhanced" is just the latest badge to put on content in the hope that some suckers will pay more money for the "same" content they already have. There's no striving for a better version here, just a drive for more profit.