5 ms·

Video deinterlacing is one of those topics that I always felt would be a good way to teach myself deep learning techniques. The idea that you have a previous fr

Video deinterlacing is one of those topics that I always felt would be a good way to teach myself deep learning techniques. The idea that you have a previous frame and a next frame with information to help you fill in the gaps of the current frame is really intuitive and easy to visualize. I'm sure this has been solved really well by now, but it seems like a good problem to learn with.

IshKebab 4y ago
Definitely already been done, but yeah it's pretty much ideal for ML.
NonNefarious 4y ago
I wouldn't say it has been solved all that well. The problem is the temporal difference between the two fields, so moving objects have a "comb" distortion while more static parts of the image mostly don't. Interlacing should never have been a part of a digital-TV standard, but broadcasters pushed a bunch of FUD and lies during the progressive-vs.-interlaced debate. They whined that going progressive "would break their pipeline," when that's obviously BS because they'd been playing film-originated material for decades. Acquisition should always have taken place in a progressive format, even if transmission continued to be interlaced for a while. Now we're stuck with a bunch of historical footage that looks like shit. And then there's the asinine perpetuation of non-integer frame rates; those too should have been abolished with digital TV.
- Melatonic 4y ago
  Yea seriously - interlacing made sense only for when it was actually required by the technology. We should have dropped it at the first chance we got
- babypuncher 4y ago
  Interlacing should never have been part of any digital video standard, in my opinion. All it has done is cause problems. Early digital productions shot in 720p look better than their 1080i peers.
  - rasz 4y ago
    The problem is first digital video was sill targeting CRTs.
    - babypuncher 4y ago
      Interlacing could have been handled by the playback hardware. At the end of the day, 60 interlaced fields or 30 full frames per second still require the same amount of bandwidth.
    - NonNefarious 4y ago
      That doesn't matter. There is absolutely no work required to show a progressive image on an interlaced device. The opposite is not true; you can never recover the information you've lost by acquiring motion pictures with an interlacing device. Historical note: DVI (which was HDMI but without audio) was invented for CRTs.
      - rasz 4y ago
        Storing progressive would mean either ~2x more data or losing 1/2 of temporal information. For the longest time we settled at half temporal with stuttering 25-30 fps, ewww. Interlacing is a simpler case of https://en.wikipedia.org/wiki/Checkerboard_rendering https://en.wikipedia.org/wiki/Checkerboard_rendering, and we seem to have developed quite good algorithms for that. Are there any deinterlacing algorithms reusing motion vectors? seems like a low hanging fruit. >DVI (which was HDMI but without audio) was invented for CRTs. nah, DVI came out of Silicon Image 1995 work on PanelLink (TMDS), intended application right in the name :) DVI is pretty lame tho, its just digitized VGA with all the stupid quirks (realtime speed, blanking) useless in the age of LCDs. Afaik even the newest DisplayPort still retains all the lameness despite being packetized, still sends VGA formatted video data with embedded clock, syncs, porches etc divided into neat little packets instead of transmitting only the picture. MIPI DSI Burst mode is close to ideal, dump raw screen worth of video data at max speed and turn off the link until another screen is ready. SDI 30 year old SDI is a digital video aimed at CRTs.
brigade 4y ago
It's one of the oldest widespread applications of neural networks for video, with NNEDI at least dating to 15 years ago. That said, I don't think there's been much work incorporating new research from the current resurgence of deep learning.
- babypuncher 4y ago
  I wish Handbrake would incorporate some of these. Its built-in deinterlacing options leave me wanting, particularly with older animated content.
Melatonic 4y ago
Thats not necessarily only deinterlacing - lots of modern codecs are doing compression (pretty much all of them) based on how much change there is frame to frame. But almost nothing is interlaced these days. Although of course interlacing itself was a simple form of what you are describing above in a way.

IshKebab 4y ago

Definitely already been done, but yeah it's pretty much ideal for ML.

NonNefarious 4y ago

I wouldn't say it has been solved all that well. The problem is the temporal difference between the two fields, so moving objects have a "comb" distortion while more static parts of the image mostly don't. Interlacing should never have been a part of a digital-TV standard, but broadcasters pushed a bunch of FUD and lies during the progressive-vs.-interlaced debate. They whined that going progressive "would break their pipeline," when that's obviously BS because they'd been playing film-originated material for decades. Acquisition should always have taken place in a progressive format, even if transmission continued to be interlaced for a while. Now we're stuck with a bunch of historical footage that looks like shit. And then there's the asinine perpetuation of non-integer frame rates; those too should have been abolished with digital TV.

Melatonic 4y ago
Yea seriously - interlacing made sense only for when it was actually required by the technology. We should have dropped it at the first chance we got
babypuncher 4y ago
Interlacing should never have been part of any digital video standard, in my opinion. All it has done is cause problems. Early digital productions shot in 720p look better than their 1080i peers.
- rasz 4y ago
  The problem is first digital video was sill targeting CRTs.
  - babypuncher 4y ago
    Interlacing could have been handled by the playback hardware. At the end of the day, 60 interlaced fields or 30 full frames per second still require the same amount of bandwidth.
  - NonNefarious 4y ago
    That doesn't matter. There is absolutely no work required to show a progressive image on an interlaced device. The opposite is not true; you can never recover the information you've lost by acquiring motion pictures with an interlacing device. Historical note: DVI (which was HDMI but without audio) was invented for CRTs.
    - rasz 4y ago
      Storing progressive would mean either ~2x more data or losing 1/2 of temporal information. For the longest time we settled at half temporal with stuttering 25-30 fps, ewww. Interlacing is a simpler case of https://en.wikipedia.org/wiki/Checkerboard_rendering https://en.wikipedia.org/wiki/Checkerboard_rendering, and we seem to have developed quite good algorithms for that. Are there any deinterlacing algorithms reusing motion vectors? seems like a low hanging fruit. >DVI (which was HDMI but without audio) was invented for CRTs. nah, DVI came out of Silicon Image 1995 work on PanelLink (TMDS), intended application right in the name :) DVI is pretty lame tho, its just digitized VGA with all the stupid quirks (realtime speed, blanking) useless in the age of LCDs. Afaik even the newest DisplayPort still retains all the lameness despite being packetized, still sends VGA formatted video data with embedded clock, syncs, porches etc divided into neat little packets instead of transmitting only the picture. MIPI DSI Burst mode is close to ideal, dump raw screen worth of video data at max speed and turn off the link until another screen is ready. SDI 30 year old SDI is a digital video aimed at CRTs.

brigade 4y ago

It's one of the oldest widespread applications of neural networks for video, with NNEDI at least dating to 15 years ago. That said, I don't think there's been much work incorporating new research from the current resurgence of deep learning.

babypuncher 4y ago
I wish Handbrake would incorporate some of these. Its built-in deinterlacing options leave me wanting, particularly with older animated content.

Melatonic 4y ago

Thats not necessarily only deinterlacing - lots of modern codecs are doing compression (pretty much all of them) based on how much change there is frame to frame. But almost nothing is interlaced these days. Although of course interlacing itself was a simple form of what you are describing above in a way.