3 ms·

Previously discussed: [HN 2016] - https://news.ycombinator.com/item?id=12871403 https://news.ycombinator.com/item?id=12871403

by dpeck 7y ago

userbinator 7y ago
I wrote a comment there about frame differencing and wanting to try writing a video decoder after having written a JPEG one. Now over 2.5 years later, I can say that I did write a toy H.261 decoder, along with one for MPEG-1 and the beginnings of another for H.262/MPEG-2 in my spare time, and it wasn't all that difficult. In fact, because of the need to not only decode an image for each frame, but to do it quickly enough to keep up with the framerate on the limited hardware of the time, these early video codecs are in some ways simpler than JPEG --- e.g. all the Huffman tables are static, the colourspace and subsampling are either fixed or have only a small number of variations, etc. As evidence in support of this, my JPEG decoder is around 750 LoC (in C), while the H.261 one is slightly smaller at just under 700 LoC. The MPEG-1 decoder is a bit more complex, at close to 1kLoC. I haven't finished the H.262 one and it is more complex (interlacing is a pain...), but it's probably doable in less than 2kLoC total for a decoder that understands both MPEG-1 and 2/H.262 --- the latter is somewhat a superset of the former. I wasn't really optimising for size, nor did I have a good idea of how much code it'd take before I finished, so these should be quite "realistic" estimates of complexity. I've seen and given recommendations on here to write a JPEG codec as a learning exercise, and a search of GitHub reveals plenty of others who've done it; but the same situation doesn't seem to hold for video. Nonetheless, having done it and realised it's not that much work (i.e. should be doable in a weekend or two), I now recommend trying to write codecs for H.261 and MPEG-1 too. There's not that much media now in those two codecs, but if you get to MPEG-2, you can experience watching DVDs using your own code.
- jwr 7y ago
  The complexity in H.264 gets you later on, when you try to build a complete (e.g. fully compliant) decoder. There are some killer features which seemed like a good idea at the time, but which make the complexity staggering. MBAFF is a good example (Macroblock-Adaptive Frame/Field encoding). What this means is that the encoder can choose for every block whether it wants to encode it as a single frame or two interlaced fields. Combined with B-frames (and remember that B-frames reference frames both in the past and in the future) this makes for really interesting memory access patterns, which then kills your cache locality and murders performance.

Previously discussed: [HN 2016] - https://news.ycombinator.com/item?id=12871403 https://news.ycombinator.com/item?id=12871403

by dpeck 7y ago

Previously discussed: [HN 2016] - https://news.ycombinator.com/item?id=12871403 https://news.ycombinator.com/item?id=12871403

userbinator 7y ago

I wrote a comment there about frame differencing and wanting to try writing a video decoder after having written a JPEG one. Now over 2.5 years later, I can say that I did write a toy H.261 decoder, along with one for MPEG-1 and the beginnings of another for H.262/MPEG-2 in my spare time, and it wasn't all that difficult. In fact, because of the need to not only decode an image for each frame, but to do it quickly enough to keep up with the framerate on the limited hardware of the time, these early video codecs are in some ways simpler than JPEG --- e.g. all the Huffman tables are static, the colourspace and subsampling are either fixed or have only a small number of variations, etc. As evidence in support of this, my JPEG decoder is around 750 LoC (in C), while the H.261 one is slightly smaller at just under 700 LoC. The MPEG-1 decoder is a bit more complex, at close to 1kLoC. I haven't finished the H.262 one and it is more complex (interlacing is a pain...), but it's probably doable in less than 2kLoC total for a decoder that understands both MPEG-1 and 2/H.262 --- the latter is somewhat a superset of the former. I wasn't really optimising for size, nor did I have a good idea of how much code it'd take before I finished, so these should be quite "realistic" estimates of complexity. I've seen and given recommendations on here to write a JPEG codec as a learning exercise, and a search of GitHub reveals plenty of others who've done it; but the same situation doesn't seem to hold for video. Nonetheless, having done it and realised it's not that much work (i.e. should be doable in a weekend or two), I now recommend trying to write codecs for H.261 and MPEG-1 too. There's not that much media now in those two codecs, but if you get to MPEG-2, you can experience watching DVDs using your own code.

jwr 7y ago
The complexity in H.264 gets you later on, when you try to build a complete (e.g. fully compliant) decoder. There are some killer features which seemed like a good idea at the time, but which make the complexity staggering. MBAFF is a good example (Macroblock-Adaptive Frame/Field encoding). What this means is that the encoder can choose for every block whether it wants to encode it as a single frame or two interlaced fields. Combined with B-frames (and remember that B-frames reference frames both in the past and in the future) this makes for really interesting memory access patterns, which then kills your cache locality and murders performance.