5 ms·

Dear Mr. Glasner, Bagon, and Irani (authors of the super-resolution paper): Please apply this technique to resolve a historically important question about the

by alister 14y ago

Dear Mr. Glasner, Bagon, and Irani (authors of the super-resolution paper): Please apply this technique to resolve a historically important question about the assassination of President John F. Kennedy by identifying the license plate number of the car immediately behind the bus: http://i754.photobucket.com/albums/xx183/twsxpress/Forum2/Yellow_Transit_Van.png http://i754.photobucket.com/albums/xx183/twsxpress/Forum2/Ye... http://i754.photobucket.com/albums/xx183/twsxpress/Forum2/murry1240.jpg http://i754.photobucket.com/albums/xx183/twsxpress/Forum2/mu... Better resolution photos should be available since the ones I've seen printed in books are better than these. And there are plenty of photos of Texas license plates from 1963 with which to seed your algorithm. The reason this is interesting is because Roger Craig, a Deputy Sheriff in Dallas, after he witnessed the assassination said that he saw Lee Harvey Oswald run from the Texas School Book Depository and get into Nash Rambler station wagon driven by a another man. These photos show a Nash Rambler station wagon that was indeed passing the Depository just after the assassination. If motor vehicle records still exist for 1963, then its owner can be identified. The key point is that the existence of a person picking up Oswald after the assassination would strongly indicate the possibility that he was not acting alone. I'll readily grant that this isn't likely to settle the issue, but I think it still would be amazing if the super-resolution approach was able to generate new information about a topic that no one expects to ever see new information about.

jacobolus 14y ago
That’s not how this technique works. There’s no magic here, just some algorithms. It doesn’t use time and space travel to reach back into the original scene and pull data that wasn’t captured by the camera. Instead, it guesses at what the blown up image should look like based on an examination of other features already in the image, with some expectation that shapes and textures will be similar in different parts of the image and at different scales – will have a sort of fractal nature – and by aiming to make shapes with edges instead of fuzzy boundaries. Using these algorithms can’t reveal the precise content of a license plate that was too small in the original image to see the letters of.
- alister 14y ago
  > license plate that was too small in the original image to see the letters of I did mention that better quality photos are available. And I did show that there are at least 2 photos of the same license plate -- there are probably more. In the photos printed in books, the license plate looks like the last couple lines of the Snellen eye chart in the original article -- i.e. it's quite blurry, but the super-resolution nethod was able to resolve it. > it guesses at what the blown up image should look like based on an examination of other features already in the image, with some expectation that shapes and textures will be similar in different parts of the image and at different scales It is true that there are no license plates at different scales in the same image. But are you quite certain that the technique cannot be applied across a range of photographs? That is, couldn't photos of license plates of the same type (and hence with exactly the same font) at different scales taken by the same photographic process on the same day in the same conditions have the same "fractal nature" that could be exploited to improve the resolution of the target photo? Let me put it another way: Suppose I cut up the Snellen eye chart into 6 equal pieces. Now I have 6 photos. Clearly, 5 of those photos can be used to improve the resolution of the 6th. Is there some principle at work here that says that for super-resolution to work, all data must come from a single original photo?
  - jacobolus 14y ago
    If you had several pictures of the same license plate, each of which was big enough to get meaningful information from but just barely too small to make out the letters when you blew it up with a regular bicubic interpolation + gaussian sharpening step, then you might be able to figure out more than you previously could. It seems quite unlikely to me, but go ahead and try it. If you look carefully, it’s actually possible to make meaningful guesses at the letters in the eye chart just as well in the bicubic version as in the “SR” version.
    - ricardobeat 14y ago
      Did we look at the same chart? The characters in that last line are 3 or 4 pixels tall. I just did an experiment and 80% of guesses were wrong. There is just not enough information, but this method can still decode them.
nkoren 14y ago
I'm not an expert on this algorithm, but I'm quite sure that the algorithm does not recover information which exists in the image; rather, it invents information which categorically does not exist. Since both the natural world and many built objects are quasi-fractal in nature (repeating windows in a building, leaves on a plant, etc.), the information which it invents often happens to be correct. For non-fractal items like license plates, however, there's no chance that it would produce a useful result. I'm pretty sure that if you seeded the algorithm with images of many other Texas license plates, and then asked it to sharpen this image, what you'd get would simply be the super-resolution sum of all the other Texas license plates. It would show you something that looked intelligible, but actually had absolutely nothing to do with the original image. That would do more to confuse the issue than to shed light on it. The other point is that I'm not sure how well this would actually work on blurry images, or whether (as a super-resolution technique) it's very specialised for working on pixelated images. That said, if it can work on blurry images too, then it would be a fun and interesting exercise to see what other historical images could yield insight (or confusion) with this kind of enhancement.
- regularfry 14y ago
  There was a hack a little while ago where someone demonstrated reading digits from a pixelated image, with no more information than the font used. That's precisely what would happen here.
  - ef4 14y ago
    The hack you're referring to works because the input space is very small and well-defined, and the blur filter that's being applied is perfectly predictable. There's no truly random noise involved. If instead you printed the document out and blurred it with a wet sponge, it would be much harder to reverse, because the real world is much noisier. Similarly, an analog photo at the limit of its resolution is full of real noise that's hard to remove.
- Someone 14y ago
  8"what you'd get would simply be the super-resolution sum of all the other Texas license plates."* I do not think so. That sum would not look like any other Texas license plate. Instead, it would locally pick the best fit parts to what data it can find in the image. Chances are their algorithm works at different scales; the better they can mix the information at different scales correctly, the smaller the risk that those local best fits are inconsistent with each other (for example, the left of a letter might look like a b, while the right looks like a q. If the algorithm does not look at scales of the width of a character, it could easily produce something that combines the two characters into one. Looking at that Snellen chart example, I think I see something like that in the top/bottom of some of the characters (last line should probably read D K N T W U L J S P X V M R A H C F O Y Z G; see http://guereros.info/img.php?fl=p5r4p4m4d4n2t24606s4t2m416d4n5l5q5i594h4q4z5t5v5h4d464y5v2c4j4g4n2p4c4j4k4l4p4j5m5q4s2m4x5f4a426g5p4u2r2c4t2b45444v5h484r4z5v5k5r2e4h4m5 http://guereros.info/img.php?fl=p5r4p4m4d4n2t24606s4t2m416d4...) I think that, if there is sufficient data in the image, chances are good (but not necessarily 100%) that the result will be a sharp representation of the real license plate. Chances are also good that it would remove that barely visible dead fly from the plate. If there is insufficient information, it would pick a valid license plate that fit the data best. It would be nice if the algorithm also computed some validity estimates (a bit like an alpha mask 'my confidence in predicting this pixel is x%')
tqkxzugoaupvwqr 14y ago
I highly recommend "Family of Secrets" by Russ Baker. It gives great insight into the history of the Bush family and their intelligence network. A large part of the book is devoted to their involvement in the JFK assassination. http://www.amazon.com/s/ref=nb_sb_ss_i_0_19/183-0647002-3212717?url=search-alias%3Daps&field-keywords=russ+baker+family+of+secrets&sprefix=russ+baker+family+o%2Caps%2C224 http://www.amazon.com/s/ref=nb_sb_ss_i_0_19/183-0647002-3212...

by alister 14y ago

jacobolus 14y ago

That’s not how this technique works. There’s no magic here, just some algorithms. It doesn’t use time and space travel to reach back into the original scene and pull data that wasn’t captured by the camera. Instead, it guesses at what the blown up image should look like based on an examination of other features already in the image, with some expectation that shapes and textures will be similar in different parts of the image and at different scales – will have a sort of fractal nature – and by aiming to make shapes with edges instead of fuzzy boundaries. Using these algorithms can’t reveal the precise content of a license plate that was too small in the original image to see the letters of.

alister 14y ago
> license plate that was too small in the original image to see the letters of I did mention that better quality photos are available. And I did show that there are at least 2 photos of the same license plate -- there are probably more. In the photos printed in books, the license plate looks like the last couple lines of the Snellen eye chart in the original article -- i.e. it's quite blurry, but the super-resolution nethod was able to resolve it. > it guesses at what the blown up image should look like based on an examination of other features already in the image, with some expectation that shapes and textures will be similar in different parts of the image and at different scales It is true that there are no license plates at different scales in the same image. But are you quite certain that the technique cannot be applied across a range of photographs? That is, couldn't photos of license plates of the same type (and hence with exactly the same font) at different scales taken by the same photographic process on the same day in the same conditions have the same "fractal nature" that could be exploited to improve the resolution of the target photo? Let me put it another way: Suppose I cut up the Snellen eye chart into 6 equal pieces. Now I have 6 photos. Clearly, 5 of those photos can be used to improve the resolution of the 6th. Is there some principle at work here that says that for super-resolution to work, all data must come from a single original photo?
- jacobolus 14y ago
  If you had several pictures of the same license plate, each of which was big enough to get meaningful information from but just barely too small to make out the letters when you blew it up with a regular bicubic interpolation + gaussian sharpening step, then you might be able to figure out more than you previously could. It seems quite unlikely to me, but go ahead and try it. If you look carefully, it’s actually possible to make meaningful guesses at the letters in the eye chart just as well in the bicubic version as in the “SR” version.
  - ricardobeat 14y ago
    Did we look at the same chart? The characters in that last line are 3 or 4 pixels tall. I just did an experiment and 80% of guesses were wrong. There is just not enough information, but this method can still decode them.

nkoren 14y ago

I'm not an expert on this algorithm, but I'm quite sure that the algorithm does not recover information which exists in the image; rather, it invents information which categorically does not exist. Since both the natural world and many built objects are quasi-fractal in nature (repeating windows in a building, leaves on a plant, etc.), the information which it invents often happens to be correct. For non-fractal items like license plates, however, there's no chance that it would produce a useful result. I'm pretty sure that if you seeded the algorithm with images of many other Texas license plates, and then asked it to sharpen this image, what you'd get would simply be the super-resolution sum of all the other Texas license plates. It would show you something that looked intelligible, but actually had absolutely nothing to do with the original image. That would do more to confuse the issue than to shed light on it. The other point is that I'm not sure how well this would actually work on blurry images, or whether (as a super-resolution technique) it's very specialised for working on pixelated images. That said, if it can work on blurry images too, then it would be a fun and interesting exercise to see what other historical images could yield insight (or confusion) with this kind of enhancement.

regularfry 14y ago
There was a hack a little while ago where someone demonstrated reading digits from a pixelated image, with no more information than the font used. That's precisely what would happen here.
- ef4 14y ago
  The hack you're referring to works because the input space is very small and well-defined, and the blur filter that's being applied is perfectly predictable. There's no truly random noise involved. If instead you printed the document out and blurred it with a wet sponge, it would be much harder to reverse, because the real world is much noisier. Similarly, an analog photo at the limit of its resolution is full of real noise that's hard to remove.
Someone 14y ago
8"what you'd get would simply be the super-resolution sum of all the other Texas license plates."* I do not think so. That sum would not look like any other Texas license plate. Instead, it would locally pick the best fit parts to what data it can find in the image. Chances are their algorithm works at different scales; the better they can mix the information at different scales correctly, the smaller the risk that those local best fits are inconsistent with each other (for example, the left of a letter might look like a b, while the right looks like a q. If the algorithm does not look at scales of the width of a character, it could easily produce something that combines the two characters into one. Looking at that Snellen chart example, I think I see something like that in the top/bottom of some of the characters (last line should probably read D K N T W U L J S P X V M R A H C F O Y Z G; see http://guereros.info/img.php?fl=p5r4p4m4d4n2t24606s4t2m416d4n5l5q5i594h4q4z5t5v5h4d464y5v2c4j4g4n2p4c4j4k4l4p4j5m5q4s2m4x5f4a426g5p4u2r2c4t2b45444v5h484r4z5v5k5r2e4h4m5 http://guereros.info/img.php?fl=p5r4p4m4d4n2t24606s4t2m416d4...) I think that, if there is sufficient data in the image, chances are good (but not necessarily 100%) that the result will be a sharp representation of the real license plate. Chances are also good that it would remove that barely visible dead fly from the plate. If there is insufficient information, it would pick a valid license plate that fit the data best. It would be nice if the algorithm also computed some validity estimates (a bit like an alpha mask 'my confidence in predicting this pixel is x%')

tqkxzugoaupvwqr 14y ago

I highly recommend "Family of Secrets" by Russ Baker. It gives great insight into the history of the Bush family and their intelligence network. A large part of the book is devoted to their involvement in the JFK assassination. http://www.amazon.com/s/ref=nb_sb_ss_i_0_19/183-0647002-3212717?url=search-alias%3Daps&field-keywords=russ+baker+family+of+secrets&sprefix=russ+baker+family+o%2Caps%2C224 http://www.amazon.com/s/ref=nb_sb_ss_i_0_19/183-0647002-3212...