6 ms·
Unraveling the JPEG
- dragontamer 7y agoNot much to say aside from a really good demonstration! It must have taken a good chunk of Javascript to convert all of the files in just the right way, as well as a lot of thinking to isolate the various parts of the JPEG format to make it easier to understand. Good job!
- onemoresoop 7y agoDamn, this is the most granular JPEG editor I've seen so far. Leaving the joke aside, that's a really fun research/demo.
- sprash 7y agoThis shows again that JPEG is apparently some alien technology from the future. People always say that nothing good can come out of a committee. JPEG proves the opposite.
- cmurf 7y agoWhat is the relevance of JFIF in this? That's the file format, JPEG is the compression scheme. Yet we refer to JPEG files not JFIF. How did it happen we care more, or give more credit to the compression part?
- dmitrygr 7y agoBecause the whole point is making images smaller. The header bytes you append up front aren't particularly clever or worth a discussion.
- nneonneo 7y agoHonestly, I don’t consider JFIF to be a good format. It’s poorly extensible and requires hacks like byte stuffing to work properly. I much prefer PNG personally - it’s highly extensible via fourcc codes and doesn’t require weird hacks.
- userbinator 7y agoI'm not sure why you consider byte stuffing a "hack", because it's a great way to make the codestream self-synchronising and easily decoded by hardware. It's also used a lot in video formats, because it enables easy seeking.
- userbinator 7y agoIf you want to know all the details, the spec itself isn't too hard to understand either, and comes with a bunch of flowcharts on how to decode the Huffman tables and the run-length codes for each DCT block: https://www.w3.org/Graphics/JPEG/itu-t81.pdf https://www.w3.org/Graphics/JPEG/itu-t81.pdf I highly recommend writing a JPEG en/decoder as an exercise in coding; getting to the point of being able to decode almost all the images you probably have or can find on the Internet shouldn't take more than a few hours and <1k lines of (C) code. Once you've done that, you'll be ready to take on a simple video codec, like H.261. I find it self-motivational because the results are very visible (and if you have a bug, all sorts of fun effects can appear too.) (As you may have guessed, I've written a JPEG decoder, as well as JBIG2 (partially), H.261, and MPEG-1, each of which is roughly 1kLoC. Currently thinking of trying JPEG2000 too, which is over an order of magnitude more complex.)
- cedex12 7y agoany book recommendation on the basics of compression?
- grenoire 7y agoWhat's always daunting to me in projects like these is the time it would take me to dig and understand the spec itself...
- userbinator 7y agoThe JPEG spec is a bit on the verbose side, and most of it can be skimmed easily; for example, one of the sentences near the beginning is "An encoder is an embodiment of an encoding process." There's also specifications of lossless modes and arithmetic compression, which you probably don't care about for a first exercise; the "baseline DCT" is the only variant that really matters. The spec is nearly 200 pages, but the diagrams and flowcharts take up a significant amount of that space. Interestingly enough, the JPEG2000 spec is at first glance not that much longer, but it is far more terse, so the size of the spec is only a rough estimate of the complexity and work it'd take to implement.
- 1wd 7y ago
- xchip 7y agoRelated: This a JPEG visualizer in just 250 lines in easy to read Python 3.0 code. https://github.com/aguaviva/micro-jpeg-visualizer https://github.com/aguaviva/micro-jpeg-visualizer
- deleted 7y ago[deleted]
- herpderperator 7y agoThis is actually fascinating. I had always imagined (and thought I experienced) that JPGs get corrupted by practically any modification, but it seems most modifications don't make the image undisplayable, rather only those that mess with the header (for obvious reasons!)
- nneonneo 7y agoThe key is that almost any sequence of bits can be parsed as valid Huffman codes (since unused codes would be a waste of the code space). JPEG doesn’t use any length field for image data, nor does it have any integrity verification. This means you can edit or delete almost any byte in the image data segment and a decoder will still interpret it. A few blocks near the edit site will be corrupted, but it’s very likely that the bitstream will resynchronize some blocks later, so the corruption tends to be highly localized (modulo block/colour component shifting). By contrast, PNG, which also uses Huffman codes for image compression via DEFLATE, has both length fields and integrity checks (CRC32), which the majority of decoders will check. Hence, almost any corruption to a PNG will render it invalid. If you fix up the length and CRC fields after tampering with the file, you can glitch PNGs too, which can produce some very interesting effects.
- mercer 7y agoIt was incredibly fun to see how some changes had drastic effects, but I was also surprised by the fact that most modifications past the header either had no clear visible effect, or kept the image intact. Somehow I'd come to expect otherwise.
- nneonneo 7y agoWhat’s remarkable is, like MP3, JPEG is a perceptual format designed decades ago but still so widely used today. We’ve actually gotten much, much better at image and audio compression in the intervening years, but both JPEG and MP3 hang on because they’re “good enough” for the majority of uses. I do hope that new alternatives like BPG, HEIC or WebP displace JPEG someday - but that day promises to be very far into the future.
- userbinator 7y agoOver 2 decades of software and hardware understand JPEG, its decoding/encoding complexity is low, and it's been around long enough that any patents on the original standard have expired. The same can't be said of the newer formats, so if you want to distribute images widely then you must use a highly compatible format. This is why I believe the other formats will always have only niche use --- remember JPEG2000? It's still around in things like PDFs and is popular for geographical applications (where the scalability and tiling features find practical use), but the immense complexity and somewhat unclear patent situation meant it never gained any widespread acceptance.
- tambourine_man 7y agoWe are also not using your 90s encoder. The amount of research done in this area is remarkable. Check Mozjpeg and Guetzli if you haven’t yet. On the MP3 side, current LAME is a lot better than the encoders we used back then.
- spilk 7y agoI believe that iPhones for the past generation or two have defaulted to taking pictures in HEIC.
- bArray 7y agoJust as a side note: I found a WebP file on a website I wanted to share in Slack. I downloaded it and tried to convert it to JPEG to put in Slack (as it didn't seem to support it). It was much harder than it should have been to convert the image type. Never had such an issue with a JPEG, PNG, GIF, etc.
- _bxg1 7y agoHow does this compare to PNG? I believe the latter is lossless, so presumably it's just the plain pixels with Huffman or something similar. But I'd be curious to know more details.
- microcolonel 7y agoIt's a totally different approach, a bit too much to explain in a HN comment; but generally it involves rearranging the data with a set of filters, and then compressing with what is essentially gzip.
- zamalek 7y agoYup. PNG is 1, if JPEG were a 10 on the complexity scale (which it isn't, it is pretty simple on the absolute scale). You can understand PNG if you know a bit about algorithms, but you need a tiny bit more than high-school math to really grok JPEG. Lossless encoding can be far simpler to understand (in some cases it's not, see FLAC).
- userbinator 7y agoPNG filters basically difference each byte of each line of image data with a choice of [1] nothing, [2] the pixel preceding the current one, [3] the pixel immediately above on the previous line, [4] an average of the latter two, or [5] "Paeth" (a dynamic choice between the left, up, and upper-left pixels), the goal being to increase redundancy for areas of smooth gradients. Each line has its own choice of filter indicated by a prefix byte. Then it compresses the differenced data using gzip/flate/zlib (a general-purpose LZ+Huffman algorithm).
- tomduncalf 7y agoThis is an amazingly well done article! The writing is very clear and the interactive elements are fantastic in explaining the concepts. Really impressive and it was great to demystify how JPEGs work, I had a vague idea but the details are fascinating. Would love to see another one of these about MP3!