4 ms·

Before anyone asks, I do think there is some value to upscaling of old photos and videos, like the popular HD/4k 60fps upscaling of videos from the 1800s: It gi

by anyfoo 4y ago

Before anyone asks, I do think there is some value to upscaling of old photos and videos, like the popular HD/4k 60fps upscaling of videos from the 1800s: It gives even casual viewers a better sense of what the actual scene, and the life depicted within, might have been at that period. It's easier to accept this as "more real" (even though it's ironically less real). But I really wish any such upscaling was very clearly marked as such, and pointed out to even casual viewers with disclaimers: Unlike the old choppy, blurry, low-res video, this is not a sample of reality back then anymore, but a speculative reconstruction.

FinnKuhn 4y ago
To be fair upscaling with videos isn't necessarily making information up, but depending on the method used may use information from other frames, wich I think isn't the same as speculative reconstruction for photos. For increasing the frame rate for videos this depends on the model used, but in general most of them don't add any new information, but instead calculate how something looked in-between two frames. https://medium.com/axinc-ai/flavr-a-machine-learning-model-to-increase-video-frame-rate-758fe8132818 https://medium.com/axinc-ai/flavr-a-machine-learning-model-t...
- anyfoo 4y ago
  Definitely! I was explicitly talking about things like "AI upscaling". Not simple interpolation (which keeps the image spectrum the same and is effectively just anti-aliasing), for example. To be honest though, the method you linked does seem to fall in the bucket I am warning against: It does make information up, it does add that made up information. Those frames in between are not simply interpolated, or you would not need an ML model. Case in point, individual droplets in that example animation may not have existed in reality, or moved the way they did here. But hopefully that's fixed by just dropping the interpolated frames, at least.

by anyfoo 4y ago

FinnKuhn 4y ago

To be fair upscaling with videos isn't necessarily making information up, but depending on the method used may use information from other frames, wich I think isn't the same as speculative reconstruction for photos. For increasing the frame rate for videos this depends on the model used, but in general most of them don't add any new information, but instead calculate how something looked in-between two frames. https://medium.com/axinc-ai/flavr-a-machine-learning-model-to-increase-video-frame-rate-758fe8132818 https://medium.com/axinc-ai/flavr-a-machine-learning-model-t...

anyfoo 4y ago
Definitely! I was explicitly talking about things like "AI upscaling". Not simple interpolation (which keeps the image spectrum the same and is effectively just anti-aliasing), for example. To be honest though, the method you linked does seem to fall in the bucket I am warning against: It does make information up, it does add that made up information. Those frames in between are not simply interpolated, or you would not need an ML model. Case in point, individual droplets in that example animation may not have existed in reality, or moved the way they did here. But hopefully that's fixed by just dropping the interpolated frames, at least.