3 ms·

I'm not an expert on this algorithm, but I'm quite sure that the algorithm does not recover information which exists in the image; rather, it invents informatio

by nkoren 14y ago

I'm not an expert on this algorithm, but I'm quite sure that the algorithm does not recover information which exists in the image; rather, it invents information which categorically does not exist. Since both the natural world and many built objects are quasi-fractal in nature (repeating windows in a building, leaves on a plant, etc.), the information which it invents often happens to be correct. For non-fractal items like license plates, however, there's no chance that it would produce a useful result. I'm pretty sure that if you seeded the algorithm with images of many other Texas license plates, and then asked it to sharpen this image, what you'd get would simply be the super-resolution sum of all the other Texas license plates. It would show you something that looked intelligible, but actually had absolutely nothing to do with the original image. That would do more to confuse the issue than to shed light on it. The other point is that I'm not sure how well this would actually work on blurry images, or whether (as a super-resolution technique) it's very specialised for working on pixelated images. That said, if it can work on blurry images too, then it would be a fun and interesting exercise to see what other historical images could yield insight (or confusion) with this kind of enhancement.

regularfry 14y ago
There was a hack a little while ago where someone demonstrated reading digits from a pixelated image, with no more information than the font used. That's precisely what would happen here.
- ef4 14y ago
  The hack you're referring to works because the input space is very small and well-defined, and the blur filter that's being applied is perfectly predictable. There's no truly random noise involved. If instead you printed the document out and blurred it with a wet sponge, it would be much harder to reverse, because the real world is much noisier. Similarly, an analog photo at the limit of its resolution is full of real noise that's hard to remove.
Someone 14y ago
8"what you'd get would simply be the super-resolution sum of all the other Texas license plates."* I do not think so. That sum would not look like any other Texas license plate. Instead, it would locally pick the best fit parts to what data it can find in the image. Chances are their algorithm works at different scales; the better they can mix the information at different scales correctly, the smaller the risk that those local best fits are inconsistent with each other (for example, the left of a letter might look like a b, while the right looks like a q. If the algorithm does not look at scales of the width of a character, it could easily produce something that combines the two characters into one. Looking at that Snellen chart example, I think I see something like that in the top/bottom of some of the characters (last line should probably read D K N T W U L J S P X V M R A H C F O Y Z G; see http://guereros.info/img.php?fl=p5r4p4m4d4n2t24606s4t2m416d4n5l5q5i594h4q4z5t5v5h4d464y5v2c4j4g4n2p4c4j4k4l4p4j5m5q4s2m4x5f4a426g5p4u2r2c4t2b45444v5h484r4z5v5k5r2e4h4m5 http://guereros.info/img.php?fl=p5r4p4m4d4n2t24606s4t2m416d4...) I think that, if there is sufficient data in the image, chances are good (but not necessarily 100%) that the result will be a sharp representation of the real license plate. Chances are also good that it would remove that barely visible dead fly from the plate. If there is insufficient information, it would pick a valid license plate that fit the data best. It would be nice if the algorithm also computed some validity estimates (a bit like an alpha mask 'my confidence in predicting this pixel is x%')

by nkoren 14y ago

regularfry 14y ago

There was a hack a little while ago where someone demonstrated reading digits from a pixelated image, with no more information than the font used. That's precisely what would happen here.

ef4 14y ago
The hack you're referring to works because the input space is very small and well-defined, and the blur filter that's being applied is perfectly predictable. There's no truly random noise involved. If instead you printed the document out and blurred it with a wet sponge, it would be much harder to reverse, because the real world is much noisier. Similarly, an analog photo at the limit of its resolution is full of real noise that's hard to remove.

Someone 14y ago

8"what you'd get would simply be the super-resolution sum of all the other Texas license plates."* I do not think so. That sum would not look like any other Texas license plate. Instead, it would locally pick the best fit parts to what data it can find in the image. Chances are their algorithm works at different scales; the better they can mix the information at different scales correctly, the smaller the risk that those local best fits are inconsistent with each other (for example, the left of a letter might look like a b, while the right looks like a q. If the algorithm does not look at scales of the width of a character, it could easily produce something that combines the two characters into one. Looking at that Snellen chart example, I think I see something like that in the top/bottom of some of the characters (last line should probably read D K N T W U L J S P X V M R A H C F O Y Z G; see http://guereros.info/img.php?fl=p5r4p4m4d4n2t24606s4t2m416d4n5l5q5i594h4q4z5t5v5h4d464y5v2c4j4g4n2p4c4j4k4l4p4j5m5q4s2m4x5f4a426g5p4u2r2c4t2b45444v5h484r4z5v5k5r2e4h4m5 http://guereros.info/img.php?fl=p5r4p4m4d4n2t24606s4t2m416d4...) I think that, if there is sufficient data in the image, chances are good (but not necessarily 100%) that the result will be a sharp representation of the real license plate. Chances are also good that it would remove that barely visible dead fly from the plate. If there is insufficient information, it would pick a valid license plate that fit the data best. It would be nice if the algorithm also computed some validity estimates (a bit like an alpha mask 'my confidence in predicting this pixel is x%')