5 ms·

Any video filter experts here? Request to any video filter expert ------------------------------------ I started watching this. The slides are unreadable bu

Any video filter experts here? Request to any video filter expert ------------------------------------ I started watching this. The slides are unreadable but the camera is perfectly still and the slides are for several "key frames" where the compression algorithm decides to replace one set of compression artifacts for another. For example try to read the first keyword under "Translates into:": https://www.youtube.com/watch?v=Z_cX3bzkExE&t=2m14s https://www.youtube.com/watch?v=Z_cX3bzkExE&t=2m14s The keyword is unreadable at the start but as you keep looking at it over 50 keyframes it becomes readable to me. Since the camera is in a fixed position it should be possible to combine the data from those artifacts into a single superresolution video with very small assumptions. (i.e. the assumption that the image is the same image until at least 5% change or something). There's not even anyone moving in front of it. -> Can someone who actually knows this stuff apply a superresolution interlacing filter to this video and post the superresolution version somewhere? I hope this is not too much work, and I am sure we would all appreciate the results since the slides are not human-readable before applying some kind of superresolution!

coleifer 8y ago
How precious that you think someone cares.
- logicallee 8y ago
  I think there's an Adobe After Effects plugin. Granted the people who are set up for heavy video editing are probably a very small percentage of HN readers :) There's probably more HN readers who would take is as a challenge to do an unsupervised convolutional neural net :D and come back with a research paper. I myself don't use Adobe After Effects.
peterwwillis 8y ago
I'm sorry you got downvoted for this comment. HN voting is the worst.
- logicallee 8y ago
  Perhaps I was naive about the state of the art. After the now-dead reply I received, I searched, and found a couple of papers like this https://arxiv.org/abs/1801.04590 https://arxiv.org/abs/1801.04590 - "Frame-Recurrent Video Super-Resolution" but if you look at p. 8, I think many of the algorithms still wouldn't end up with readable text. This paper is from this year, so it is an area of active research. I wrote a quick mail to the authors to see if they would put the video through their setup (since the last paper update was just 3 months ago) and share their results. 2. Trying it myself... After my downvotes I tried this small piece of software: http://www.infognition.com/VideoEnhancer/ http://www.infognition.com/VideoEnhancer/ Which shows a before/after. Here is their page on their super-resolution algorithm: http://www.infognition.com/articles/what_is_super_resolution.html http://www.infognition.com/articles/what_is_super_resolution... I used their plugin on virtualdub on a sample of the video. The results weren't useable. Here is a picture which shows the before and after: https://imgur.com/a/0rhy7q7 https://imgur.com/a/0rhy7q7 (The diagonal lines are a watermark because I didn't pay to register video enhancer.) Also note that though it might look like a sharpen mask was applied, in fact it was not: this is just the superresolution that video enhancer came up with. Now granted I don't think that this particular site uses state of the art algorithms (its references on the page I linked are decades old) but it's the first one I found. The site also has a page explaining when it doesn't work: http://www.infognition.com/articles/when_super_resolution_doesnt_work.html http://www.infognition.com/articles/when_super_resolution_do... It specifically calls out "If your video is compressed to a low bitrate, in many cases this is very bad for super-resolution." This certainly seems to be the case here. On my comparison picture above you can see that it certainly is an improvement, it is just not enough. I still can't read most of the lines. I think this also doesn't use as many keyframes as it could. (Which makes sense - it is rare that a rare static image is up for, in this case, 25 full seconds!) There are at least 14 full keyframes there so I think there is more detail to be extracted, but it would, obviously, take longer analysis. I'll let you know if I find anything better or get an answer from the paper authors.
  - acqq 8y ago
    It's surely interesting that you try to figure out the "state of art" but as far as I see the amount of information is too low to recover more readability, and that was my estimate even before your experiments. The way I saw it, the algorithm would have to model the "compression interference patterns" to do that, and even if something like that existed the amount of information seems to be far too low. If it's really about the given talk and not exploring state of art in algorithms for recovering of the (textual?) information lost due to video compression, you'd be better off to communicate 1) with Dr Hipp who gave the talk -- he published the more recent original slides on the sqlite site: https://www.sqlite.org/talks/ https://www.sqlite.org/talks/ so it could be reasonable he'd be willing to publish these older slides (which he probably considers in some aspects outdated). Then only if that fails: 2) with the author of the video who possibly still has a higher quality version of the video, if the quality was dropped during the video compression or preparation for youtube, e.g. while trying to reduce bandwidth or reencode from the native recording format.
reilly3000 8y ago
There was a similar issue with the detection of text at angles faced when trying to decipher if the first lady was wearing jacket with a dog whistle insensitive message on it. The original source image was at an angle, so determining its authenticity was challenging until other images emgerged.

coleifer 8y ago

How precious that you think someone cares.

logicallee 8y ago
I think there's an Adobe After Effects plugin. Granted the people who are set up for heavy video editing are probably a very small percentage of HN readers :) There's probably more HN readers who would take is as a challenge to do an unsupervised convolutional neural net :D and come back with a research paper. I myself don't use Adobe After Effects.

peterwwillis 8y ago

I'm sorry you got downvoted for this comment. HN voting is the worst.

logicallee 8y ago
Perhaps I was naive about the state of the art. After the now-dead reply I received, I searched, and found a couple of papers like this https://arxiv.org/abs/1801.04590 https://arxiv.org/abs/1801.04590 - "Frame-Recurrent Video Super-Resolution" but if you look at p. 8, I think many of the algorithms still wouldn't end up with readable text. This paper is from this year, so it is an area of active research. I wrote a quick mail to the authors to see if they would put the video through their setup (since the last paper update was just 3 months ago) and share their results. 2. Trying it myself... After my downvotes I tried this small piece of software: http://www.infognition.com/VideoEnhancer/ http://www.infognition.com/VideoEnhancer/ Which shows a before/after. Here is their page on their super-resolution algorithm: http://www.infognition.com/articles/what_is_super_resolution.html http://www.infognition.com/articles/what_is_super_resolution... I used their plugin on virtualdub on a sample of the video. The results weren't useable. Here is a picture which shows the before and after: https://imgur.com/a/0rhy7q7 https://imgur.com/a/0rhy7q7 (The diagonal lines are a watermark because I didn't pay to register video enhancer.) Also note that though it might look like a sharpen mask was applied, in fact it was not: this is just the superresolution that video enhancer came up with. Now granted I don't think that this particular site uses state of the art algorithms (its references on the page I linked are decades old) but it's the first one I found. The site also has a page explaining when it doesn't work: http://www.infognition.com/articles/when_super_resolution_doesnt_work.html http://www.infognition.com/articles/when_super_resolution_do... It specifically calls out "If your video is compressed to a low bitrate, in many cases this is very bad for super-resolution." This certainly seems to be the case here. On my comparison picture above you can see that it certainly is an improvement, it is just not enough. I still can't read most of the lines. I think this also doesn't use as many keyframes as it could. (Which makes sense - it is rare that a rare static image is up for, in this case, 25 full seconds!) There are at least 14 full keyframes there so I think there is more detail to be extracted, but it would, obviously, take longer analysis. I'll let you know if I find anything better or get an answer from the paper authors.
- acqq 8y ago
  It's surely interesting that you try to figure out the "state of art" but as far as I see the amount of information is too low to recover more readability, and that was my estimate even before your experiments. The way I saw it, the algorithm would have to model the "compression interference patterns" to do that, and even if something like that existed the amount of information seems to be far too low. If it's really about the given talk and not exploring state of art in algorithms for recovering of the (textual?) information lost due to video compression, you'd be better off to communicate 1) with Dr Hipp who gave the talk -- he published the more recent original slides on the sqlite site: https://www.sqlite.org/talks/ https://www.sqlite.org/talks/ so it could be reasonable he'd be willing to publish these older slides (which he probably considers in some aspects outdated). Then only if that fails: 2) with the author of the video who possibly still has a higher quality version of the video, if the quality was dropped during the video compression or preparation for youtube, e.g. while trying to reduce bandwidth or reencode from the native recording format.

reilly3000 8y ago

There was a similar issue with the detection of text at angles faced when trying to decipher if the first lady was wearing jacket with a dog whistle insensitive message on it. The original source image was at an angle, so determining its authenticity was challenging until other images emgerged.