14 ms·

AI based interpolation will soon make these mathematical interpolation methods obsolete right?

NayamAmarshe 3y ago
Yes, with Upscayl, we’re already doing that (https://upscayl.org https://upscayl.org). It’s not as fast yet but it’s extremely good. Good enough that you could print a low resolution image after upscaling.
- deadbabe 3y ago
  Thank you for everything you do!!
amelius 3y ago
If you prefer hands with six fingers ...
deleted 3y ago
[deleted]
edflsafoiewq 3y ago
In quality, yes, but they're only feasible for stuff like batch processing because they're so huge and slow. You can't use them for stuff like scaling an <img> in a browser. Incidentally, the way I always thought it would work is the huge and slow AI would be used to discover a tiny algorithm that just so happens to be an amazing image rescaler. Instead we just ship the whole AI. Anybody know of any work on the former strategy?
- eru 3y ago
  > In quality, yes, but they're only feasible for stuff like batch processing because they're so huge and slow. Aren't games using these for real time upscaling many dozen times a second already?
krapp 3y ago
No, why would they? What would you expect an "AI" to do besides somehow implement an interpolation algorithm? The math is unavoidable.
- wmf 3y ago
  AI algorithms encode domain knowledge in their weights that isn't present in simple interpolation functions. This can lead to better perceptual quality.
- deadbabe 3y ago
  With AI based interpolation, I should be able to zoom in forever without seeing pixelation at any level of zoom.
  - krapp 3y ago
    Beyond a certain point that ceases to be interpolation and just becomes simulation. An image only has so much information in it, and AI isn't magic.
    - giobox 3y ago
      For lots of uses cases, if the end output meets user expectations, does it really matter if it was interpolated from simulated data? For many use cases (editing old family photos etc), I'm increasingly not caring. I don't need per-pixel accuracy, I need a nice artifact that reminds me of a memory. If it looks 100 percent accurate to my eye, I'm not sure I care how much "fake" data it took to generate it. The important thing is it records the scene as I remember it. This isn't hypothetical either; Photoshop already shipped their first iteration of the idea, for sure won't be the last: > https://www.digitaltrends.com/computing/photoshop-generative-expand-ai/ https://www.digitaltrends.com/computing/photoshop-generative... > https://www.reddit.com/r/photoshop/comments/13s6tp9/using_the_new_generative_fill_to_produce_infinite/ https://www.reddit.com/r/photoshop/comments/13s6tp9/using_th...
      - krapp 3y ago
egorfine 3y ago
For practical reasons mostly yes. For scientifically correct scaling (think forensics) you will of course need to interpolate the source data instead of generating synthetic data which is done by AI upscaling.
- ur-whale 3y ago
  Not entirely sure the AI scaling is more or less "correct" than the "interpolate the source data" method. Both use priors, the former a large and complex one, the latter a simplistic one, but both "invent data based on assumptions".
  - Dylan16807 3y ago
    You can completely understand and intuit what interpolation does, and it doesn't drag in extra detailing from anywhere.
  - amelius 3y ago
    That's correct but probably also practically irrelevant.
Dwedit 3y ago
AI-based image upscaling happened many years ago. Waifu2x first released back in 2015, and ESRGAN first released in 2018.
- waffletower 3y ago
  SwinIR hails from 2021: https://arxiv.org/abs/2108.10257 https://arxiv.org/abs/2108.10257

Yes, with Upscayl, we’re already doing that (https://upscayl.org https://upscayl.org). It’s not as fast yet but it’s extremely good. Good enough that you could print a low resolution image after upscaling.

deadbabe 3y ago
Thank you for everything you do!!

amelius 3y ago

If you prefer hands with six fingers ...

deleted 3y ago

[deleted]

edflsafoiewq 3y ago

In quality, yes, but they're only feasible for stuff like batch processing because they're so huge and slow. You can't use them for stuff like scaling an <img> in a browser. Incidentally, the way I always thought it would work is the huge and slow AI would be used to discover a tiny algorithm that just so happens to be an amazing image rescaler. Instead we just ship the whole AI. Anybody know of any work on the former strategy?

eru 3y ago
> In quality, yes, but they're only feasible for stuff like batch processing because they're so huge and slow. Aren't games using these for real time upscaling many dozen times a second already?

krapp 3y ago

No, why would they? What would you expect an "AI" to do besides somehow implement an interpolation algorithm? The math is unavoidable.

wmf 3y ago
AI algorithms encode domain knowledge in their weights that isn't present in simple interpolation functions. This can lead to better perceptual quality.
deadbabe 3y ago
With AI based interpolation, I should be able to zoom in forever without seeing pixelation at any level of zoom.
- krapp 3y ago
  Beyond a certain point that ceases to be interpolation and just becomes simulation. An image only has so much information in it, and AI isn't magic.
  - giobox 3y ago
    For lots of uses cases, if the end output meets user expectations, does it really matter if it was interpolated from simulated data? For many use cases (editing old family photos etc), I'm increasingly not caring. I don't need per-pixel accuracy, I need a nice artifact that reminds me of a memory. If it looks 100 percent accurate to my eye, I'm not sure I care how much "fake" data it took to generate it. The important thing is it records the scene as I remember it. This isn't hypothetical either; Photoshop already shipped their first iteration of the idea, for sure won't be the last: > https://www.digitaltrends.com/computing/photoshop-generative-expand-ai/ https://www.digitaltrends.com/computing/photoshop-generative... > https://www.reddit.com/r/photoshop/comments/13s6tp9/using_the_new_generative_fill_to_produce_infinite/ https://www.reddit.com/r/photoshop/comments/13s6tp9/using_th...
    - krapp 3y ago

egorfine 3y ago

For practical reasons mostly yes. For scientifically correct scaling (think forensics) you will of course need to interpolate the source data instead of generating synthetic data which is done by AI upscaling.

ur-whale 3y ago
Not entirely sure the AI scaling is more or less "correct" than the "interpolate the source data" method. Both use priors, the former a large and complex one, the latter a simplistic one, but both "invent data based on assumptions".
- Dylan16807 3y ago
  You can completely understand and intuit what interpolation does, and it doesn't drag in extra detailing from anywhere.
- amelius 3y ago
  That's correct but probably also practically irrelevant.

Dwedit 3y ago

AI-based image upscaling happened many years ago. Waifu2x first released back in 2015, and ESRGAN first released in 2018.

waffletower 3y ago
SwinIR hails from 2021: https://arxiv.org/abs/2108.10257 https://arxiv.org/abs/2108.10257