4 ms·

I had a really good go at reading this and trying to understand it but I feel I don't understand a whole lot more about image decoding/encoding than before I st

I had a really good go at reading this and trying to understand it but I feel I don't understand a whole lot more about image decoding/encoding than before I started. Like I get the core concepts of key frames, motion vectors and such on a high level but if you asked me to actually create a decoder I wouldn't have a clue where to start. I feel like I would need a full hour long video on each paragraph of this post to really understand it.

magicalhippo 5y ago
We made a mpeg-ish encoder and decoder at uni, and it was surprisingly simple. Keyframes were just jpegs. Then for intraframes, you first found the motion vectors for each 8x8 block, then generated the predicted intraframe from the keyframe and motion vector. Then you subtracted the prediction off the keyframe. The resulting "prediction error image" was then simply jpeg encoded as the intraframe, and appended to the output after the motion vectors. Decoding was reverse, reconstruct the predicted intraframe from the previous keyframe and motion vectors, and add back the prediction error image. Might be glossing over something as it was over a decade ago but should be about the gist of it. We played with different algorithms for finding motion vectors and such, including accelerating it with GPGPU. Really fun project, and assuming you use an existing jpeg library, not at all big or difficult.
- magicalhippo 5y ago
  > simply jpeg encoded as the intraframe In case it wasn't obvious, the error image can of course have negative values which jpeg can't handle. So you add a bias of +128 and clamp the biased error to [0, 255]. During decoding you simply subtract the bias when adding back the error image.
a_e_k 5y ago
Last year, I did a deep dive into Fabrice Bellard's obfuscated image decoder (http://www.ioccc.org/2018/bellard/hint.html http://www.ioccc.org/2018/bellard/hint.html) entry for the 2018 IOCCC. His code implements of a lot of these decoding techniques in just 4KB of source, including a stringified 128x128 test image. You can find my deobfuscation and detailed explanation of his program here: http://eastfarthing.com/blog/2020-09-14-decoder/ http://eastfarthing.com/blog/2020-09-14-decoder/

magicalhippo 5y ago

We made a mpeg-ish encoder and decoder at uni, and it was surprisingly simple. Keyframes were just jpegs. Then for intraframes, you first found the motion vectors for each 8x8 block, then generated the predicted intraframe from the keyframe and motion vector. Then you subtracted the prediction off the keyframe. The resulting "prediction error image" was then simply jpeg encoded as the intraframe, and appended to the output after the motion vectors. Decoding was reverse, reconstruct the predicted intraframe from the previous keyframe and motion vectors, and add back the prediction error image. Might be glossing over something as it was over a decade ago but should be about the gist of it. We played with different algorithms for finding motion vectors and such, including accelerating it with GPGPU. Really fun project, and assuming you use an existing jpeg library, not at all big or difficult.

magicalhippo 5y ago
> simply jpeg encoded as the intraframe In case it wasn't obvious, the error image can of course have negative values which jpeg can't handle. So you add a bias of +128 and clamp the biased error to [0, 255]. During decoding you simply subtract the bias when adding back the error image.

a_e_k 5y ago

Last year, I did a deep dive into Fabrice Bellard's obfuscated image decoder (http://www.ioccc.org/2018/bellard/hint.html http://www.ioccc.org/2018/bellard/hint.html) entry for the 2018 IOCCC. His code implements of a lot of these decoding techniques in just 4KB of source, including a stringified 128x128 test image. You can find my deobfuscation and detailed explanation of his program here: http://eastfarthing.com/blog/2020-09-14-decoder/ http://eastfarthing.com/blog/2020-09-14-decoder/