3 ms·

So we finally have the magical "Enhance" button from sci-fi detective movies and shows, nice!

by dartharva 2y ago

scotty79 2y ago
Except that it's useless for any kind of investigative work where truth matters.
- xhkkffbf 2y ago
  It's true. And for other kinds of more common uses like watching for flaws. Imagine a system like this watching for small flaws in something with a repeated texture. The algorithm will assume the flaw is a mistake make by the camera because of poor resolution and cover up the flaw.
  - tliltocatl 2y ago
    "A camera that produces a nice picture" and "a camera that tries reproduce reality as faithfully as possible" are going to become two different products. That's also a problem because the second one is not going to benefit from the economy of scales of the first one…
- montroser 2y ago
  No, that appears to be incorrect. It is the case that with completely generative models, you will get hallucinated details very likely to be untruthful. But with this approach, you can see blurry input images of license plates that with our naked eye we could not possibly decipher the characters, then put through this model where the output is very close to the actual ground truth. https://dachunkai.github.io/evtexture.github.io/static/images/quanlitative_reds.png https://dachunkai.github.io/evtexture.github.io/static/image... Where does this information come from? It seems they are generating synthetic "event camera-like" events just from diffs between still frames? So maybe they trained a model based on real events from a real event camera? It's hard to tell from their write-up. But these results are very impressive.
  - spookie 2y ago
    You don't need these kinds of solutions for license plates, take a look at this: https://www.youtube.com/watch?v=19wgu5GZDhk https://www.youtube.com/watch?v=19wgu5GZDhk
    - the8472 2y ago
      Normal upscaling is picking pixel values in the higher-resolution target image that when run through an equivalent downscaling function would fit the smaller image. There are lots of degrees of freedom to make things up. Then there is undoing reversible transforms, such as some blurs. That makes information that was there all along more legible. Such as the example you have there. This paper is a case of both. It does upscaling, but it uses temporal information to find additional constraints that can be used to restrict the degrees of freedom of the "making values up" part. So it's part information recovery, part hallucination.
  - vessenes 2y ago
    In the GitHub comments they say what they are doing to infer events is significantly different than per frame diffs. I have no idea what they are actually doing though.
Dwedit 2y ago
ESRGAN already comes very close to being that magical Enhance button.
deleted 2y ago
[deleted]

So we finally have the magical "Enhance" button from sci-fi detective movies and shows, nice!

scotty79 2y ago

Except that it's useless for any kind of investigative work where truth matters.

xhkkffbf 2y ago
It's true. And for other kinds of more common uses like watching for flaws. Imagine a system like this watching for small flaws in something with a repeated texture. The algorithm will assume the flaw is a mistake make by the camera because of poor resolution and cover up the flaw.
- tliltocatl 2y ago
  "A camera that produces a nice picture" and "a camera that tries reproduce reality as faithfully as possible" are going to become two different products. That's also a problem because the second one is not going to benefit from the economy of scales of the first one…
montroser 2y ago
No, that appears to be incorrect. It is the case that with completely generative models, you will get hallucinated details very likely to be untruthful. But with this approach, you can see blurry input images of license plates that with our naked eye we could not possibly decipher the characters, then put through this model where the output is very close to the actual ground truth. https://dachunkai.github.io/evtexture.github.io/static/images/quanlitative_reds.png https://dachunkai.github.io/evtexture.github.io/static/image... Where does this information come from? It seems they are generating synthetic "event camera-like" events just from diffs between still frames? So maybe they trained a model based on real events from a real event camera? It's hard to tell from their write-up. But these results are very impressive.
- spookie 2y ago
  You don't need these kinds of solutions for license plates, take a look at this: https://www.youtube.com/watch?v=19wgu5GZDhk https://www.youtube.com/watch?v=19wgu5GZDhk
  - the8472 2y ago
    Normal upscaling is picking pixel values in the higher-resolution target image that when run through an equivalent downscaling function would fit the smaller image. There are lots of degrees of freedom to make things up. Then there is undoing reversible transforms, such as some blurs. That makes information that was there all along more legible. Such as the example you have there. This paper is a case of both. It does upscaling, but it uses temporal information to find additional constraints that can be used to restrict the degrees of freedom of the "making values up" part. So it's part information recovery, part hallucination.
- vessenes 2y ago
  In the GitHub comments they say what they are doing to infer events is significantly different than per frame diffs. I have no idea what they are actually doing though.

Dwedit 2y ago

ESRGAN already comes very close to being that magical Enhance button.

deleted 2y ago

[deleted]