5 ms·
The fine article shows the low-res input and high-res output photos, but conspicuously does not show a high-res original from whence the low-res input was deriv
by dotancohen 5y ago
The fine article shows the low-res input and high-res output photos, but conspicuously does not show a high-res original from whence the low-res input was derived.
Without comparing a high-res original photograph to the high-res output photograph, we do not know if this fine technique is capable of producing nice-looking high-res imagery, or if it is capable of reproducing how an image of the subject would have looked like had it been taken in higher resolution.
In other words, does the output of the technique match the actual object in the photograph?
- choeger 5y agoOf course it does not. The model generates "believable" images not exact ones.
- savolai 5y agoThe point is, it's helpful if the reader can evaluate just how faithful the transformation is vis-a-vis the original.
- amitport 5y agoHelpful, interesting, yes. BUT not a real critique on this paper which specifically does not claim anything regarding similarity with the original. This happens a lot with junior peer reviewers, they focus more on what the paper isn't or could be instead of what's actually there.
- savolai 5y agoSure, if you are evaluating at whether the claims are accurate on that level of precision, similarity may not be relevant. However for the rest of us, evaluating whether the work is relevant in the surrounding world really is important. And for that, resemblance to originals is critical in some use cases. I.e. we're evaluating a fit for specific use cases. Do you believe that is not a useful discussion?
- dorkwood 5y agoThis isn't obvious to regular people, though. I can recall seeing a conspiracy get traction on Twitter a few months back, where it was claimed that a photograph of a famous person was actually a body double. Someone used an ML upscaler to "enhance" the image, and their followers began scrutinizing the result: "The teeth are different!", "The nose shape is wrong!", "It's not the same person!"
- thrdbndndn 5y agoThere is an app called "face app" or whatever that already did pretty good job upscaling people's face using state-of-the-art AI upscaling. The result is impressive, but the moment you started to use it on someone you're actually familiar, it becomes weird very quickly for the obvious reasons. The teech, for example, are never right. This kind of "believable but not truthful" results are rampant in all these machine-learning based tools. It's not very harmful in case of upscaling a few photos I guess, but I've been bitten by it in an acclaimed translation service called DeepL. I use it to translate Japanese to English frequently, and have found that it often (nontrivially) made up sentences that don't exist in the original paragraphs, sometimes have the opposite meanings, or totally ignore part of the text to make the result "more fluent". And unlike traditional translation tools, they are very hard to notice if you know nothing about the original language. I have to from time to time use some more "primitive" translation tools, and compare the results side by side, to avoid such issues. It's frustrating.
- dotancohen 5y agoThen why is it mentioned that one of the potential uses are in medical imagery? Not to mention what happens when some good-intending but ill-informed agency release an enhanced photo of a suspect on the run.
- dpwm 5y ago> Not to mention what happens when some good-intending but ill-informed agency release an enhanced photo of a suspect on the run. The more interesting situation is when such "enhanced photos" are used in prosecutions. I suspect that ML "forensics" techniques and questionable expert witnesses will be used to elevate hunches to guilty verdicts and raise (false) conviction rates.
- stjohnswarts 5y agoI think it would be possible by any defense attorney to show how bad a job it does at reproducing the original by showing hi rez images (actual) vs AI enhance upscales. Shouldn't be hard to demonstrate "reasonable doubt"
- actually_a_dog 5y agoHow do you expect an algorithm to create information out of nowhere to fill in these details exactly as they were in the source photo?
- nayaketo 5y agoIf this AI can't produce upscaled photos that are close to original photos, the application of this tech is severely limited. There's not going to be CSI like "enhance" moment in real life like the article claims.
- actually_a_dog 5y agoYou really don't see any use for this if it can't create information? CSI "enhance" is, and always has been impossible.
- nayaketo 5y agoIt does create information though, just imaginary/wrong one.
- actually_a_dog 5y agoNo, it doesn't. A piece of code n bits long can only have a maximum of n bits of information in it; you can't get more than n bits of information out of it. It can't create information out of nothing.
- ObscureScience 5y agoI'm not sure what you mean, but that is pretty much what generative neural systems do. If by information you mean data that correspond to reality? Than sure. I guess you can say that the data is a correlated to reality by a certain probability, and the better the model the higher the probability?
- 5y ago
- Ballas 5y agoThat is indeed a shortcoming of this article in my opinion as well. If you want a comparison to the original high res photos, there are some examples in the original paper for SR3: https://arxiv.org/pdf/2104.07636.pdf https://arxiv.org/pdf/2104.07636.pdf Have not had a look at the CDM paper.
- sandos 5y agoWhat strikes me is that the jaguar (?) got "kind" eyes with SR3, and "mean" eyes with the other algoritms, including the original!
- tcpekin 5y agoFig. 9 of this paper is really interesting if you zoom in. It looks like if the model was not trained on the appropriate class label, it just goes completely off the rails. As previous commenters have noted, I would be very hesitant to use this for anything analytical, or where you are looking for something unexpected. For faces though, this is amazing.
- zepn 5y ago> where you are looking for something unexpected I'd be extremely sceptical of its use on medical images for this reason.
- helsinkiandrew 5y ago> In other words, does the output of the technique match the actual object in the photograph? Probably a lot in some cases and a little bit in most others. I wonder how long before this gets used in court by an incompetent prosecutor.
- addandsubtract 5y agoIf you follow the referenced page[0], you'll find examples[1] showing the high-res originals. [0] https://iterative-refinement.github.io/ https://iterative-refinement.github.io/ [1] https://iterative-refinement.github.io/images/super_res_examples.png https://iterative-refinement.github.io/images/super_res_exam...