7 ms·

Definitely a breath of fresh air that someone's still trying to do super-resolution without neural networks. This example shows that at the moment, it can still

Definitely a breath of fresh air that someone's still trying to do super-resolution without neural networks. This example shows that at the moment, it can still be better and MUCH faster to use classical CV techniques for certain applications.

gruez 7y ago
A similar thing happened with upscaling algorithms for video games. AMD's Contrast Adaptive Sharpening was shown to have superior image quality than Nvidia's Deep Learning Super Sampling[1]. Plus the former algorithm works on every game and doesn't need a training set unlike the deep learning algorithm. [1] https://www.techspot.com/article/1873-radeon-image-sharpening-vs-nvidia-dlss/ https://www.techspot.com/article/1873-radeon-image-sharpenin...
- kevin_thibedeau 7y ago
  ML implementations can insert detail that was never present in the original image. You can't get that with other methods. That may or not be a good thing depending on the source material and your desired result.
  - 2bitencryption 7y ago
    "detail that was never present" doesn't exist. ML can insert "its best guess based on a training set". A human-tuned algo can insert "its output as defined by the handwritten aglo", which presumably is based on the human's own "training set" of personal experience. but the truth of any lossy encoding is that... information is lost, period. best you can do is guess as to what was there.
    - IfOnlyYouKnew 7y ago
      This is one of those old talking points people for some reason love... "Information is lost" is too vague. You're counting bits on disk, but fewer bits does not always mean "less information" when your algorithm gets smarter. Compression is the obvious, classical example. Even for lossy compression, information loss is << change in size. ML offers the promise to take this to extreme levels: give it a picture of (part of) the NY skyline, and it adds the rest from memory, adjusting weather and time of day to your sample. Is that new information "real"? That's really up to your definition. The best example of this idea is those CSI-Style "Enhance" effects: It used to be true that people on Slashdot and later HN would outrank each other with the superior smartitude of saying "That's impossible! Information was lost!". Funny story: that effect now exists. It's quite obvious that, for example, a low-res image of a license plate still contains some data, and that an algorithm can find a license plate number that maximizes the probability of that specific low-res image. With a bit of ML, those algorithms have become better than the human brain in almost zero time flat. Turns out the information was still there.
      - AstralStorm 7y ago
        Maximizing probability naively sometimes works, but of course it can produce misleading garbage. And then you can get fooled instead of actually correctly believing the image was unreadable. There is no free lunch, even with robust estimators. They will make mistakes. For image quality, it is ok to make a mistake here or there. For actual recognition? Terrible. Better than human brain? Show it. People are pretty good at reading blurry text when trained, but I'm not aware of a test pitting trained people against a machine. (No, Mechanical Turk does not count as trained at a specific task.)
        floriol 7y ago
        Human brain can just as easily predict erroneously, we just seldom happen to have only a single shot at it. For visual recognition we usually look at it for an extended amount of time, waiting with "judgement" until the probability that what we see is indeed what we think it is is sufficiently high. Neural networks also output a probability (when trained in a problem that require it), that can signal their confidence in their answer.
      - boomlinde 7y ago
        The license plate may as well be printed with the exact pattern you see on screen, and the assumption that it’s a low resolution sampling of some higher density information would be false. Any additional information derived from it is conjecture, however based on reasonable assumptions. By ”enhancing” the image you may gain information, but that information doesn’t inherently relate to the information you lost.
      - pjc50 7y ago
        This is quite capable of producing a high-res image of some license plate, yes. But it's only probabilistic: there's no proof that the license plate definitely has the exact same number as the one in the low-res photo. You have to allow for the possibility of the system hallucinating the wrong result and enhancing the certainty of it. While you could use it as input to a police search it would be grossly unjust to show such an enhanced image to a jury.
ackbar03 7y ago
I think it only works for this particular use case though. Anime visuals are much less complicated then real world pictures. The fact that they are (mostly) created digitally in the first place means a algorithmic solution is likely to be available
- dylan604 7y ago
  Um, that's not really accurate. There is a huuuuuge library of anime that was created by hand, recorded to film, and then edited. It wasn't until the mid '00s that digital was becoming a thing. It was cheaper to do it by hand than have to render out digitally.
  - AstralStorm 7y ago
    It is still a simple case, because it is animation with sharp continuous lines, gradients or relatively flat inpainting. And mostly temporally stable, which is not even getting exploited by this cheap but effective superresolution filter.

gruez 7y ago

A similar thing happened with upscaling algorithms for video games. AMD's Contrast Adaptive Sharpening was shown to have superior image quality than Nvidia's Deep Learning Super Sampling[1]. Plus the former algorithm works on every game and doesn't need a training set unlike the deep learning algorithm. [1] https://www.techspot.com/article/1873-radeon-image-sharpening-vs-nvidia-dlss/ https://www.techspot.com/article/1873-radeon-image-sharpenin...

kevin_thibedeau 7y ago
ML implementations can insert detail that was never present in the original image. You can't get that with other methods. That may or not be a good thing depending on the source material and your desired result.
- 2bitencryption 7y ago
  "detail that was never present" doesn't exist. ML can insert "its best guess based on a training set". A human-tuned algo can insert "its output as defined by the handwritten aglo", which presumably is based on the human's own "training set" of personal experience. but the truth of any lossy encoding is that... information is lost, period. best you can do is guess as to what was there.
  - IfOnlyYouKnew 7y ago
    This is one of those old talking points people for some reason love... "Information is lost" is too vague. You're counting bits on disk, but fewer bits does not always mean "less information" when your algorithm gets smarter. Compression is the obvious, classical example. Even for lossy compression, information loss is << change in size. ML offers the promise to take this to extreme levels: give it a picture of (part of) the NY skyline, and it adds the rest from memory, adjusting weather and time of day to your sample. Is that new information "real"? That's really up to your definition. The best example of this idea is those CSI-Style "Enhance" effects: It used to be true that people on Slashdot and later HN would outrank each other with the superior smartitude of saying "That's impossible! Information was lost!". Funny story: that effect now exists. It's quite obvious that, for example, a low-res image of a license plate still contains some data, and that an algorithm can find a license plate number that maximizes the probability of that specific low-res image. With a bit of ML, those algorithms have become better than the human brain in almost zero time flat. Turns out the information was still there.
    - AstralStorm 7y ago
      Maximizing probability naively sometimes works, but of course it can produce misleading garbage. And then you can get fooled instead of actually correctly believing the image was unreadable. There is no free lunch, even with robust estimators. They will make mistakes. For image quality, it is ok to make a mistake here or there. For actual recognition? Terrible. Better than human brain? Show it. People are pretty good at reading blurry text when trained, but I'm not aware of a test pitting trained people against a machine. (No, Mechanical Turk does not count as trained at a specific task.)
      - floriol 7y ago
        Human brain can just as easily predict erroneously, we just seldom happen to have only a single shot at it. For visual recognition we usually look at it for an extended amount of time, waiting with "judgement" until the probability that what we see is indeed what we think it is is sufficiently high. Neural networks also output a probability (when trained in a problem that require it), that can signal their confidence in their answer.
    - boomlinde 7y ago
      The license plate may as well be printed with the exact pattern you see on screen, and the assumption that it’s a low resolution sampling of some higher density information would be false. Any additional information derived from it is conjecture, however based on reasonable assumptions. By ”enhancing” the image you may gain information, but that information doesn’t inherently relate to the information you lost.
    - pjc50 7y ago
      This is quite capable of producing a high-res image of some license plate, yes. But it's only probabilistic: there's no proof that the license plate definitely has the exact same number as the one in the low-res photo. You have to allow for the possibility of the system hallucinating the wrong result and enhancing the certainty of it. While you could use it as input to a police search it would be grossly unjust to show such an enhanced image to a jury.

ackbar03 7y ago

I think it only works for this particular use case though. Anime visuals are much less complicated then real world pictures. The fact that they are (mostly) created digitally in the first place means a algorithmic solution is likely to be available

dylan604 7y ago
Um, that's not really accurate. There is a huuuuuge library of anime that was created by hand, recorded to film, and then edited. It wasn't until the mid '00s that digital was becoming a thing. It was cheaper to do it by hand than have to render out digitally.
- AstralStorm 7y ago
  It is still a simple case, because it is animation with sharp continuous lines, gradients or relatively flat inpainting. And mostly temporally stable, which is not even getting exploited by this cheap but effective superresolution filter.