9 ms·

So this means that CSI's "enhance" is actually more possible than reputed.

jhdevos 14y ago
Not directly (but of course you were not entirely serious). Look at, for instance, the letter chart. The letters at the bottom are turned into very sharp forms, but not all look like letters, and those that do might not be turned into the correct ones... For instance, if they used this to "enhance" a picture with a piece of paper in someone's pocket, these letters would be enhanced based on other forms somewhere else in the picture -- so some vague letter shapes might become 'Burger King' just because the photo also contains a Burger King logo somewhere :)
ynniv 14y ago
This doesn't actually add information, it just creates a magnified image that looks nicer than bicubic.
- hamburglar 14y ago
  Yeah, and in fact the fact that it looks crisper can be misleading. You can't rely on a detail that a method like this infers from the low resolution image. Look at the green tiles on the kitchen floor in the 6th example. The "Our SR Result" image looks fantastic, but it's obvious that it's false, because the tiles end up misshapen and unevenly spaced. You really wouldn't want to convict someone of murder based on this type of enhancement.
- jessriedel 14y ago
  Actually, it does add information. It takes information from elsewhere in the picture and uses it to interpolate/guess what high-resolution image produced the low-resolution pixels.
  - daenz 14y ago
    "Adding information" in this context means providing accurate data that was not previously there. The guesses that the algorithm makes are approximations, but there are many original signals that could produce those same approximations, so the information is not accurate.
- wmf 14y ago
  It's a shame that hamburglar is hellbanned because his comment is insightful. In some sense, isn't this algorithm "inventing" detail? If you're lucky the detail corresponds closely to what should be there, but sometimes it doesn't and thus it can't be relied upon.
  - ehsanu1 14y ago
    For those without showdead one, hamburglar's comment: Yeah, and in fact the fact that it looks crisper can be misleading. You can't rely on a detail that a method like this infers from the low resolution image. Look at the green tiles on the kitchen floor in the 6th example. The "Our SR Result" image looks fantastic, but it's obvious that it's false, because the tiles end up misshapen and unevenly spaced. You really wouldn't want to convict someone of murder based on this type of enhancement.
    - kemiller 14y ago
      I can't reply to him directly, so I'll do it here. You wouldn't want to convict someone of murder, no. But you could certainly use it to generate leads.
      - sukuriant 14y ago
        The scary thing comes when people stare using it to convict someone of murder. CSI has trained us to do things like that :/
    - kintamanimatt 14y ago
      Looking through hamburglar's comment history, I can't work out why he'd be shadowbanned. He doesn't seem to have said anything particularly egregious or spammed.
      - hamburglar 14y ago
        I'm shadowbanned/hellbanned? News to me. I'm pretty new here and have no idea how that even happens. Perhaps I should go away.
- JackC 14y ago
  It may not add totally new information, but it does more than create an image that looks nicer. It's not just inventing plausible information, it's providing the most likely new information given the available data. For example, "given what we can see of that eye chart, it's most likely the little blob is a 'G'". So it's adding new interpretive statistical information, which could be pretty helpful in some situations.
  - ef4 14y ago
    The problem is that the algorithm is using a definition of "most likely" that doesn't help you answer questions about the real scene captured in the image. For example, if what you're really trying to do is decode text, this algorithm will lose every time to one that knows the prior probabilities of natural human languages. ("ONLY" is much more probable than "QNLY", but this algorithm does't know that).
    - ansgri 14y ago
      It doesn't add any information to a human eye, but may be very useful preprocessing for algorithms that do not know about the quasi-fractal structure of natural images, and that's most of CV.
- EastCoastLA 14y ago
  Could you use AI to find a near match for the image? Using the eye chart as the example: Since this now becomes a image matching problem once Super Resolution is used, search an image database and zoom enhance.

Not directly (but of course you were not entirely serious). Look at, for instance, the letter chart. The letters at the bottom are turned into very sharp forms, but not all look like letters, and those that do might not be turned into the correct ones... For instance, if they used this to "enhance" a picture with a piece of paper in someone's pocket, these letters would be enhanced based on other forms somewhere else in the picture -- so some vague letter shapes might become 'Burger King' just because the photo also contains a Burger King logo somewhere :)

ynniv 14y ago

This doesn't actually add information, it just creates a magnified image that looks nicer than bicubic.

hamburglar 14y ago
Yeah, and in fact the fact that it looks crisper can be misleading. You can't rely on a detail that a method like this infers from the low resolution image. Look at the green tiles on the kitchen floor in the 6th example. The "Our SR Result" image looks fantastic, but it's obvious that it's false, because the tiles end up misshapen and unevenly spaced. You really wouldn't want to convict someone of murder based on this type of enhancement.
jessriedel 14y ago
Actually, it does add information. It takes information from elsewhere in the picture and uses it to interpolate/guess what high-resolution image produced the low-resolution pixels.
- daenz 14y ago
  "Adding information" in this context means providing accurate data that was not previously there. The guesses that the algorithm makes are approximations, but there are many original signals that could produce those same approximations, so the information is not accurate.
wmf 14y ago
It's a shame that hamburglar is hellbanned because his comment is insightful. In some sense, isn't this algorithm "inventing" detail? If you're lucky the detail corresponds closely to what should be there, but sometimes it doesn't and thus it can't be relied upon.
- ehsanu1 14y ago
  For those without showdead one, hamburglar's comment: Yeah, and in fact the fact that it looks crisper can be misleading. You can't rely on a detail that a method like this infers from the low resolution image. Look at the green tiles on the kitchen floor in the 6th example. The "Our SR Result" image looks fantastic, but it's obvious that it's false, because the tiles end up misshapen and unevenly spaced. You really wouldn't want to convict someone of murder based on this type of enhancement.
  - kemiller 14y ago
    I can't reply to him directly, so I'll do it here. You wouldn't want to convict someone of murder, no. But you could certainly use it to generate leads.
    - sukuriant 14y ago
      The scary thing comes when people stare using it to convict someone of murder. CSI has trained us to do things like that :/
  - kintamanimatt 14y ago
    Looking through hamburglar's comment history, I can't work out why he'd be shadowbanned. He doesn't seem to have said anything particularly egregious or spammed.
    - hamburglar 14y ago
      I'm shadowbanned/hellbanned? News to me. I'm pretty new here and have no idea how that even happens. Perhaps I should go away.
JackC 14y ago
It may not add totally new information, but it does more than create an image that looks nicer. It's not just inventing plausible information, it's providing the most likely new information given the available data. For example, "given what we can see of that eye chart, it's most likely the little blob is a 'G'". So it's adding new interpretive statistical information, which could be pretty helpful in some situations.
- ef4 14y ago
  The problem is that the algorithm is using a definition of "most likely" that doesn't help you answer questions about the real scene captured in the image. For example, if what you're really trying to do is decode text, this algorithm will lose every time to one that knows the prior probabilities of natural human languages. ("ONLY" is much more probable than "QNLY", but this algorithm does't know that).
  - ansgri 14y ago
    It doesn't add any information to a human eye, but may be very useful preprocessing for algorithms that do not know about the quasi-fractal structure of natural images, and that's most of CV.
EastCoastLA 14y ago
Could you use AI to find a near match for the image? Using the eye chart as the example: Since this now becomes a image matching problem once Super Resolution is used, search an image database and zoom enhance.