5 ms·

Understanding and writing a JPEG decoder in Python

263 points by sfpoet 6y ago

subins2000 6y ago
This was a really good read to understand JPEG with the right videos and images included. Thanks!
- yasoob 6y ago
  Glad to hear that. :) IMO some content is just better suited as visualizations in the form of videos and images.
iAmAPencilYo 6y ago
Thank you for this; lots of details, references and detailed diagrams! One small note - the first image you mention that we'll need to reference is a bit small to read. Zooming in on the page does not zoom it in due to format of the page. I had to right click -> View image, in a separate tab in order to be able to zoom in enough to read the righthand side of it. edit: this is on the latest version of Firefox
pratio 6y ago
This is a fantastic writeup, love the attention to detail and thank you so much for not posting it on medium.
jonny789 6y ago
Thanks a lot for your article. I think,for me it will be a perfect start for understanding JPEG decoders & making one too. :)
greggman3 6y ago
This is great! I first learned the general ideas from this Computerphile video https://www.youtube.com/watch?v=n_uNPbdenRs https://www.youtube.com/watch?v=n_uNPbdenRs While we're on it does anyone know of articles/videos that go into similar detail for HEVC?
Ahmd72 6y ago
Thanks for making this, a couple of months ago I was working on a jpeg project and needed to understand how it works. I was searching wildly for a tutorial with a python implementation, unfortunately never found one. I found this one in C++ though, it goes a lot more in depth https://www.youtube.com/playlist?list=PLpsTn9TA_Q8VMDyOPrDKmSJYt1DLgDZU4 https://www.youtube.com/playlist?list=PLpsTn9TA_Q8VMDyOPrDKm...
vmilner 6y ago
It always puzzles me that the quantisation matrices don't have symmetry along the diagonal. I guess it suggests the distribution of vertical and horizontal lines in typical pictures differs.
yasoob 6y ago
Hi everyone! OP here. Why write another article on JPEG when there are already hundreds of articles on the internet? Well, normally when you read articles on JPEG, the author just gives you details about what the format looks like. You don’t implement any code to do the actual decompression and decoding. Even if you do write code, it is in C/C++ and not accessible to a wide group of people. I tried to change that through this article. I give you a guided tour of JPEG encoding/decoding process and show you how it can be implemented in Python3. I mainly focus on decoding baseline encoded JPEG images.
- voltagex_ 6y ago
  I really really like this - especially with all the references you've listed. I think more programmers should have a go at working with various file formats. Well done on this article.
  - yasoob 6y ago
    Thanks! And I completely agree with the idea that more programmers should have a go at working with various file formats. I learned so much about binary file layouts through this one project. Before writing this article I had no idea that JPEGs use Huffman tables. Now all those Huffman table lectures from college algos classes make sense. I was never told in that class that this is a major use case for Huffman tables. Maybe I would have shown more interest then if I knew this fact.
- enriquto 6y ago
  > Even if you do write code, it is in C/C++ and not accessible to a wide group of people. To be honest, writing a jpeg decoder in C seems easier and more natural than doing it in python.
  - userbinator 6y ago
    As someone who has done it in C, I would both agree and disagree --- reading the file format will definitely be easier since C lets you pick bits/bytes/words/etc. off the stream directly, but on the other hand the high-level structures (looping, etc.) would probably be easier with Python. Overall, seeing as OP's Python implementation is less than 300 LoC while my C implementation was closer to 750, the Python might be easier. Then again, I followed the algorithms/flowcharts in the spec, which do a faster table-based Huffman decoding than the bit-by-bit of this Python version, and also implemented arbitrary(!) chroma subsampling as the spec allows, so it's not a direct comparison.

263 points by sfpoet 6y ago

subins2000 6y ago

This was a really good read to understand JPEG with the right videos and images included. Thanks!

yasoob 6y ago
Glad to hear that. :) IMO some content is just better suited as visualizations in the form of videos and images.

iAmAPencilYo 6y ago

Thank you for this; lots of details, references and detailed diagrams! One small note - the first image you mention that we'll need to reference is a bit small to read. Zooming in on the page does not zoom it in due to format of the page. I had to right click -> View image, in a separate tab in order to be able to zoom in enough to read the righthand side of it. edit: this is on the latest version of Firefox

pratio 6y ago

This is a fantastic writeup, love the attention to detail and thank you so much for not posting it on medium.

jonny789 6y ago

Thanks a lot for your article. I think,for me it will be a perfect start for understanding JPEG decoders & making one too. :)

greggman3 6y ago

This is great! I first learned the general ideas from this Computerphile video https://www.youtube.com/watch?v=n_uNPbdenRs https://www.youtube.com/watch?v=n_uNPbdenRs While we're on it does anyone know of articles/videos that go into similar detail for HEVC?

Ahmd72 6y ago

Thanks for making this, a couple of months ago I was working on a jpeg project and needed to understand how it works. I was searching wildly for a tutorial with a python implementation, unfortunately never found one. I found this one in C++ though, it goes a lot more in depth https://www.youtube.com/playlist?list=PLpsTn9TA_Q8VMDyOPrDKmSJYt1DLgDZU4 https://www.youtube.com/playlist?list=PLpsTn9TA_Q8VMDyOPrDKm...

vmilner 6y ago

It always puzzles me that the quantisation matrices don't have symmetry along the diagonal. I guess it suggests the distribution of vertical and horizontal lines in typical pictures differs.

yasoob 6y ago

Hi everyone! OP here. Why write another article on JPEG when there are already hundreds of articles on the internet? Well, normally when you read articles on JPEG, the author just gives you details about what the format looks like. You don’t implement any code to do the actual decompression and decoding. Even if you do write code, it is in C/C++ and not accessible to a wide group of people. I tried to change that through this article. I give you a guided tour of JPEG encoding/decoding process and show you how it can be implemented in Python3. I mainly focus on decoding baseline encoded JPEG images.

voltagex_ 6y ago
I really really like this - especially with all the references you've listed. I think more programmers should have a go at working with various file formats. Well done on this article.
- yasoob 6y ago
  Thanks! And I completely agree with the idea that more programmers should have a go at working with various file formats. I learned so much about binary file layouts through this one project. Before writing this article I had no idea that JPEGs use Huffman tables. Now all those Huffman table lectures from college algos classes make sense. I was never told in that class that this is a major use case for Huffman tables. Maybe I would have shown more interest then if I knew this fact.
enriquto 6y ago
> Even if you do write code, it is in C/C++ and not accessible to a wide group of people. To be honest, writing a jpeg decoder in C seems easier and more natural than doing it in python.
- userbinator 6y ago
  As someone who has done it in C, I would both agree and disagree --- reading the file format will definitely be easier since C lets you pick bits/bytes/words/etc. off the stream directly, but on the other hand the high-level structures (looping, etc.) would probably be easier with Python. Overall, seeing as OP's Python implementation is less than 300 LoC while my C implementation was closer to 750, the Python might be easier. Then again, I followed the algorithms/flowcharts in the spec, which do a faster table-based Huffman decoding than the bit-by-bit of this Python version, and also implemented arbitrary(!) chroma subsampling as the spec allows, so it's not a direct comparison.