13 ms·

Mozilla Advances JPEG Encoding with Mozjpeg 2.0

309 points by dochtman 12y ago

RDeckard 12y ago
This is all fine and dandy, but whatever happened to JPEG-2000 and wavelet-based compression? Still licensing/patent issues blocking wide adoption?
- acdha 12y ago
  It's still competitive performanc-wise but adoption has been lacking. Patents were an issue but there were other problems, too: uncertainty about rights and the high complexity of the spec (which for awhile was, IIRC, $1500 simply to get a copy) meant that there wasn't much traction in the open-source world, which in turn meant that many image processing apps either didn't support it at all or used a slow library with limited compatibility and incomplete spec support. There are several commercial libraries available but they also had compatibility issues (largely resolved by now) and required negotiating licenses. Mike Shaver had a good comment in the Mozilla feature-request ticket which is no doubt representative of many other projects' concerns: https://bugzilla.mozilla.org/show_bug.cgi?id=36351#c120 https://bugzilla.mozilla.org/show_bug.cgi?id=36351#c120 None of this is insurmountable, of course, but it made a lot of people hesitant to invest heavily in it. I wrote about the long-term risks awhile ago: http://blogs.loc.gov/digitalpreservation/2013/01/is-jpeg-2000-a-preservation-risk/ http://blogs.loc.gov/digitalpreservation/2013/01/is-jpeg-200... The situation is getting better – OpenJPEG is now actively developed (see http://www.openjpeg.org http://www.openjpeg.org), supports many of the more valuable features (e.g. tiled decoding so you can avoid decoding an entire large image to extract a fragment), and is becoming an official reference implementation: http://comments.gmane.org/gmane.comp.graphics.openjpeg/773 http://comments.gmane.org/gmane.comp.graphics.openjpeg/773 Recently ImageMagick and Pillow (the maintained fork of the Python Imaging Library) added support for JPEG-2000 using OpenJPEG and the same trend seems to be happening in other places. The big remaining challenge is browser support for non-Apple browsers but that's now possible, if somewhat grotesque, using Emscripten on OpenJPEG to render into a canvas tag.
pekk 12y ago
If Google had done this, people would be in here complaining it wasn't open even though it was released under an open source license.
joedrew 12y ago
(Note: I used to be employed by Mozilla, and in that capacity I was the owner of Mozilla's image decoders. I've been disconnected from all decisions for almost a year, though.) The main take-home here is that while Google's numbers all show WebP as being objectively better, the metrics they chose for comparison were relatively bad (i.e., some of them didn't take into account colour or didn't model colour correctly), and once you accounted for that the numbers were not nearly as good a story for WebP; in some cases, JPEG outperformed it. The facts that (1) WebP was not terribly compelling technically, (2) JPEG is already supported by everything on the web, not to mention devices and mobile phones etc, and (3) there's still headroom to improve JPEG in a backwards-compatible way, meant that WebP was (and, it seems, remains) a non-starter.
- dbcooper 12y ago
  I had read in the past that HEVC was probably the best for still image lossy compression. [1] Does anyone have further information on this? [1] http://en.m.wikipedia.org/wiki/High_Efficiency_Video_Coding#Main_Still_Picture http://en.m.wikipedia.org/wiki/High_Efficiency_Video_Coding#...
  - sp332 12y ago
    According to the study linked in OP, it's still true. https://people.mozilla.org/~josh/lossy_compressed_image_study_july_2014/ https://people.mozilla.org/~josh/lossy_compressed_image_stud...
    - dbcooper 12y ago
      Cheers!
  - derf_ 12y ago
    Best from a compression perspective, perhaps. There's the minor detail of it being patent-encumbered. There's no license pool available yet, so you couldn't pay for it if you wanted to. The proposed terms of the pool that is organizing (with some of the known patent holders refusing to join) would require up to $25mln/year for the video codec. No idea if you'd be able to get a better deal for still-images only. Meanwhile, JPEG is free.
deleted 12y ago
[deleted]
- dictum 12y ago
  My guess is that the $60,000 is for initial exploratory work, and if they find a way to improve it substantially that requires more development effort a bigger donation will come. And the saying about gift horses' mouths applies to FB's horses too ;)
deleted 12y ago
[deleted]
skrowl 12y ago
Seems to be source only. Has anyone built win64 binaries for this that they'd like to share.
- jibsen 12y ago
  On that subject; say what you want about how clunky CMake is, but it's really nice to be able to build projects like this with VC++ or plain MinGW-w64.
paulsmith 12y ago
Not sure how fair a comparison it is but FWIW I built it and ran it (./cjpeg) over a few photos and got ~18% savings in file size vs. imagemagick convert at quality=50%
xvilka 12y ago
It is sad that Mozilla wont implement WebP format in addition to GIF/APNG due to political reasons (I see no valid technical reasons to not include this code, especially after the news about including code for DRM). They've even disabled comments in the WebP bug [1]. And this bug was already second, they've closed first one [2]. You still can write your comments in this bug (about APNG removal) - it is not closed yet [3] [1] https://bugzilla.mozilla.org/show_bug.cgi?id=856375 https://bugzilla.mozilla.org/show_bug.cgi?id=856375 [2] https://bugzilla.mozilla.org/show_bug.cgi?id=600919 https://bugzilla.mozilla.org/show_bug.cgi?id=600919 [3] https://bugzilla.mozilla.org/show_bug.cgi?id=935466 https://bugzilla.mozilla.org/show_bug.cgi?id=935466
- Scaevolus 12y ago
  It's especially amusing since Firefox supports WebM, which is VP8 video in a Matroska container. WebP is a VP8 frame in a simple container.
  - TD-Linux 12y ago
    WebP is not just a VP8 frame. It now has its own separate library to implement decoding, and its animation features are far different from VP8.
- thristian 12y ago
  Did you not read the section in the linked article about how WebP is not measurably better than JPEG?
  - xvilka 12y ago
    I've read it. Yep, looks like they can be compared for the static images. But what about animation? And there are plenty of websites using webp already. It is not like JPEG or WebP - why not just support them both!? (or APNG and WebP).
    - kevingadd 12y ago
      I imagine there's some amount of frustration on the part of people being told they should spend time implementing WebP, when APNG was a real working alternative circa ~2004 and nobody else was willing to implement it. (Admittedly, WebP does things APNG isn't capable of.) Reminds me of MNG, where we had an animation format with support for alpha and either PNG or JPEG style compression of frames, but it never got a pervasive browser implementation so it died. (I get the impression there were strong technical reasons for that, of course.) Coincidentally, MNG also indirectly solved this problem. JNG was a subset-format of MNG that provided JPEG-compressed static images with optional support for alpha, so implementing MNG implied implementing JNG. But we never got it on the web. At the very least, instead of just saying 'WebP is bad', they're putting their money where their mouth is and advancing the state-of-the-art in JPEG encoding so that people won't need WebP (other than the alpha channel use case, that is.)
notlisted 12y ago
How does this MozJPEG 2.0 encoding compare to JPEGMini? I've been using JPEGMini for all client projects, it results in optimized JPEGs that work on all platforms with minute differences, rarely visible to the naked eye, only when I subtract pre/post in Photoshop. Very rarely does it save less than 5% and for larger images (saved with PhotoShop, save for web) often 15-30% or even more. Is the test set of images available?
- muizelaar 12y ago
  jpegmini works by compressing an image as much as possible while keeping the distortion calculated by there particular metric below a threshold. To compare against mozjpeg you'd need to compress a bunch of images with JPEGMini and then compress those images with mozjpeg so that they were the same size. You could then compare the quality of the resulting images. The test set of images is here: https://github.com/bdaehlie/web_image_formats/tree/master/images https://github.com/bdaehlie/web_image_formats/tree/master/im... If you're interested, https://github.com/dwbuiten/smallfry https://github.com/dwbuiten/smallfry is program similar to JPEGMini that uses mozjpeg as its backend.
  - ZeroGravitas 12y ago
    How much extra saving does JPEGMini or smallfry offer over a sanely but conservatively encoded JPEG?
- mzs 12y ago
  http://people.mozilla.org/~josh/lossy_compressed_image_study_july_2014/#image_sets http://people.mozilla.org/~josh/lossy_compressed_image_study... mozjpeg does Trellis quantization, there is no detailed explanation about jpegmini, a guess is it optimizes the DCT similarly to how it is described in this paper: http://vision.arc.nasa.gov/publications/sid93/sid93.pdf http://vision.arc.nasa.gov/publications/sid93/sid93.pdf
  - muizelaar 12y ago
    There are some papers by the JPEGMini people that give a pretty good hint as to how it works: http://spie.org/Publications/Proceedings/Paper/10.1117/12.872231 http://spie.org/Publications/Proceedings/Paper/10.1117/12.87... http://spie.org/Publications/Proceedings/Paper/10.1117/12.872230 http://spie.org/Publications/Proceedings/Paper/10.1117/12.87...
1ris 12y ago
It's very sad Mozilla does not even mention their own codec, Daala, and only compares to HEVC etc. I suspect it's severely lacks funding.
- joshmoz 12y ago
  Daala development is proceeding at a rapid pace. We're just not ready to bring Daala into this conversation quite yet.
- ZeroGravitas 12y ago
  There's a progress report for Daala that makes a similar comparison. They claim to beat JPEG, VP8 and x264 but not HEVC (yet!) on still images.
pdknsk 12y ago
IMO the linked study is pretty much useless, because of enforced 4:2:0 chroma subsampling. I guess this is because some of the newer codecs were originally video codecs where 4:2:0 is standard, but for still images I expect much better. Photoshop, which is obviously widely used on the desktop to make JPEGs, only uses 4:2:0 for quality <= 50 by default. This also makes me question the use of metrics to measure image quality.
- kevingadd 12y ago
  I don't think it would make sense to compare some codecs at 4:2:0 and some at 4:4:4. Unless they can all do 4:4:4, you have to pick a subsampling that they can all do and compare them at that so it's like-for-like. If your goal is to demonstrate 'JPEG can deliver better quality than this format by using 4:4:4, because that format can't do 4:4:4', then that's great, but it's a different observation than the one they're trying to make. (Also, I would be pretty curious about whether the different subsampling produces superior file sizes.)
panzi 12y ago
5%? I don't know but I hoped for significantly more.
NVI 12y ago
Is there a command line tool available? I’d like to include it into my website build process.
- joshmoz 12y ago
  Yes, building the source will produce (among other things) a command line application called "cjpeg", which can encode or re-encode.
vlucas 12y ago
Facebook's $60,000 donation is interesting. The article says that the average file size reduction was 5%. Given that Facebook has built datacenters capable of storing exabytes of photo data in "cold storage", I wonder how much money Mozilla just saved Facebook in storage space costs in relation to how much they donated.
- spacefight 12y ago
  It would be interesting to learn what the cost of CPU overhead vs. additonal cost of 5% cold storage is - but I bet the FB folks did that calculation.
Daemon404 12y ago
It should be noted, since it is nowhere to be seen in this post, it breaks API and ABI while still presenting itself as libjpeg version 6 to the system, which is very evil. Open Issues: https://github.com/mozilla/mozjpeg/issues/67 https://github.com/mozilla/mozjpeg/issues/67 https://github.com/mozilla/mozjpeg/issues/21 https://github.com/mozilla/mozjpeg/issues/21
bastawhiz 12y ago
- Alpha channel - Animation - Proper metadata - File size The list of things that WebP offers that JPEG, PNG, or GIF alone will never address.

309 points by dochtman 12y ago

This is all fine and dandy, but whatever happened to JPEG-2000 and wavelet-based compression? Still licensing/patent issues blocking wide adoption?

acdha 12y ago
It's still competitive performanc-wise but adoption has been lacking. Patents were an issue but there were other problems, too: uncertainty about rights and the high complexity of the spec (which for awhile was, IIRC, $1500 simply to get a copy) meant that there wasn't much traction in the open-source world, which in turn meant that many image processing apps either didn't support it at all or used a slow library with limited compatibility and incomplete spec support. There are several commercial libraries available but they also had compatibility issues (largely resolved by now) and required negotiating licenses. Mike Shaver had a good comment in the Mozilla feature-request ticket which is no doubt representative of many other projects' concerns: https://bugzilla.mozilla.org/show_bug.cgi?id=36351#c120 https://bugzilla.mozilla.org/show_bug.cgi?id=36351#c120 None of this is insurmountable, of course, but it made a lot of people hesitant to invest heavily in it. I wrote about the long-term risks awhile ago: http://blogs.loc.gov/digitalpreservation/2013/01/is-jpeg-2000-a-preservation-risk/ http://blogs.loc.gov/digitalpreservation/2013/01/is-jpeg-200... The situation is getting better – OpenJPEG is now actively developed (see http://www.openjpeg.org http://www.openjpeg.org), supports many of the more valuable features (e.g. tiled decoding so you can avoid decoding an entire large image to extract a fragment), and is becoming an official reference implementation: http://comments.gmane.org/gmane.comp.graphics.openjpeg/773 http://comments.gmane.org/gmane.comp.graphics.openjpeg/773 Recently ImageMagick and Pillow (the maintained fork of the Python Imaging Library) added support for JPEG-2000 using OpenJPEG and the same trend seems to be happening in other places. The big remaining challenge is browser support for non-Apple browsers but that's now possible, if somewhat grotesque, using Emscripten on OpenJPEG to render into a canvas tag.

pekk 12y ago

If Google had done this, people would be in here complaining it wasn't open even though it was released under an open source license.

joedrew 12y ago

(Note: I used to be employed by Mozilla, and in that capacity I was the owner of Mozilla's image decoders. I've been disconnected from all decisions for almost a year, though.) The main take-home here is that while Google's numbers all show WebP as being objectively better, the metrics they chose for comparison were relatively bad (i.e., some of them didn't take into account colour or didn't model colour correctly), and once you accounted for that the numbers were not nearly as good a story for WebP; in some cases, JPEG outperformed it. The facts that (1) WebP was not terribly compelling technically, (2) JPEG is already supported by everything on the web, not to mention devices and mobile phones etc, and (3) there's still headroom to improve JPEG in a backwards-compatible way, meant that WebP was (and, it seems, remains) a non-starter.

dbcooper 12y ago
I had read in the past that HEVC was probably the best for still image lossy compression. [1] Does anyone have further information on this? [1] http://en.m.wikipedia.org/wiki/High_Efficiency_Video_Coding#Main_Still_Picture http://en.m.wikipedia.org/wiki/High_Efficiency_Video_Coding#...
- sp332 12y ago
  According to the study linked in OP, it's still true. https://people.mozilla.org/~josh/lossy_compressed_image_study_july_2014/ https://people.mozilla.org/~josh/lossy_compressed_image_stud...
  - dbcooper 12y ago
    Cheers!
- derf_ 12y ago
  Best from a compression perspective, perhaps. There's the minor detail of it being patent-encumbered. There's no license pool available yet, so you couldn't pay for it if you wanted to. The proposed terms of the pool that is organizing (with some of the known patent holders refusing to join) would require up to $25mln/year for the video codec. No idea if you'd be able to get a better deal for still-images only. Meanwhile, JPEG is free.

deleted 12y ago

[deleted]

dictum 12y ago
My guess is that the $60,000 is for initial exploratory work, and if they find a way to improve it substantially that requires more development effort a bigger donation will come. And the saying about gift horses' mouths applies to FB's horses too ;)

deleted 12y ago

[deleted]

skrowl 12y ago

Seems to be source only. Has anyone built win64 binaries for this that they'd like to share.

jibsen 12y ago
On that subject; say what you want about how clunky CMake is, but it's really nice to be able to build projects like this with VC++ or plain MinGW-w64.

paulsmith 12y ago

Not sure how fair a comparison it is but FWIW I built it and ran it (./cjpeg) over a few photos and got ~18% savings in file size vs. imagemagick convert at quality=50%

xvilka 12y ago

It is sad that Mozilla wont implement WebP format in addition to GIF/APNG due to political reasons (I see no valid technical reasons to not include this code, especially after the news about including code for DRM). They've even disabled comments in the WebP bug [1]. And this bug was already second, they've closed first one [2]. You still can write your comments in this bug (about APNG removal) - it is not closed yet [3] [1] https://bugzilla.mozilla.org/show_bug.cgi?id=856375 https://bugzilla.mozilla.org/show_bug.cgi?id=856375 [2] https://bugzilla.mozilla.org/show_bug.cgi?id=600919 https://bugzilla.mozilla.org/show_bug.cgi?id=600919 [3] https://bugzilla.mozilla.org/show_bug.cgi?id=935466 https://bugzilla.mozilla.org/show_bug.cgi?id=935466

Scaevolus 12y ago
It's especially amusing since Firefox supports WebM, which is VP8 video in a Matroska container. WebP is a VP8 frame in a simple container.
- TD-Linux 12y ago
  WebP is not just a VP8 frame. It now has its own separate library to implement decoding, and its animation features are far different from VP8.
thristian 12y ago
Did you not read the section in the linked article about how WebP is not measurably better than JPEG?
- xvilka 12y ago
  I've read it. Yep, looks like they can be compared for the static images. But what about animation? And there are plenty of websites using webp already. It is not like JPEG or WebP - why not just support them both!? (or APNG and WebP).
  - kevingadd 12y ago
    I imagine there's some amount of frustration on the part of people being told they should spend time implementing WebP, when APNG was a real working alternative circa ~2004 and nobody else was willing to implement it. (Admittedly, WebP does things APNG isn't capable of.) Reminds me of MNG, where we had an animation format with support for alpha and either PNG or JPEG style compression of frames, but it never got a pervasive browser implementation so it died. (I get the impression there were strong technical reasons for that, of course.) Coincidentally, MNG also indirectly solved this problem. JNG was a subset-format of MNG that provided JPEG-compressed static images with optional support for alpha, so implementing MNG implied implementing JNG. But we never got it on the web. At the very least, instead of just saying 'WebP is bad', they're putting their money where their mouth is and advancing the state-of-the-art in JPEG encoding so that people won't need WebP (other than the alpha channel use case, that is.)

notlisted 12y ago

How does this MozJPEG 2.0 encoding compare to JPEGMini? I've been using JPEGMini for all client projects, it results in optimized JPEGs that work on all platforms with minute differences, rarely visible to the naked eye, only when I subtract pre/post in Photoshop. Very rarely does it save less than 5% and for larger images (saved with PhotoShop, save for web) often 15-30% or even more. Is the test set of images available?

muizelaar 12y ago
jpegmini works by compressing an image as much as possible while keeping the distortion calculated by there particular metric below a threshold. To compare against mozjpeg you'd need to compress a bunch of images with JPEGMini and then compress those images with mozjpeg so that they were the same size. You could then compare the quality of the resulting images. The test set of images is here: https://github.com/bdaehlie/web_image_formats/tree/master/images https://github.com/bdaehlie/web_image_formats/tree/master/im... If you're interested, https://github.com/dwbuiten/smallfry https://github.com/dwbuiten/smallfry is program similar to JPEGMini that uses mozjpeg as its backend.
- ZeroGravitas 12y ago
  How much extra saving does JPEGMini or smallfry offer over a sanely but conservatively encoded JPEG?
mzs 12y ago
http://people.mozilla.org/~josh/lossy_compressed_image_study_july_2014/#image_sets http://people.mozilla.org/~josh/lossy_compressed_image_study... mozjpeg does Trellis quantization, there is no detailed explanation about jpegmini, a guess is it optimizes the DCT similarly to how it is described in this paper: http://vision.arc.nasa.gov/publications/sid93/sid93.pdf http://vision.arc.nasa.gov/publications/sid93/sid93.pdf
- muizelaar 12y ago
  There are some papers by the JPEGMini people that give a pretty good hint as to how it works: http://spie.org/Publications/Proceedings/Paper/10.1117/12.872231 http://spie.org/Publications/Proceedings/Paper/10.1117/12.87... http://spie.org/Publications/Proceedings/Paper/10.1117/12.872230 http://spie.org/Publications/Proceedings/Paper/10.1117/12.87...

1ris 12y ago

It's very sad Mozilla does not even mention their own codec, Daala, and only compares to HEVC etc. I suspect it's severely lacks funding.

joshmoz 12y ago
Daala development is proceeding at a rapid pace. We're just not ready to bring Daala into this conversation quite yet.
ZeroGravitas 12y ago
There's a progress report for Daala that makes a similar comparison. They claim to beat JPEG, VP8 and x264 but not HEVC (yet!) on still images.

pdknsk 12y ago

IMO the linked study is pretty much useless, because of enforced 4:2:0 chroma subsampling. I guess this is because some of the newer codecs were originally video codecs where 4:2:0 is standard, but for still images I expect much better. Photoshop, which is obviously widely used on the desktop to make JPEGs, only uses 4:2:0 for quality <= 50 by default. This also makes me question the use of metrics to measure image quality.

kevingadd 12y ago
I don't think it would make sense to compare some codecs at 4:2:0 and some at 4:4:4. Unless they can all do 4:4:4, you have to pick a subsampling that they can all do and compare them at that so it's like-for-like. If your goal is to demonstrate 'JPEG can deliver better quality than this format by using 4:4:4, because that format can't do 4:4:4', then that's great, but it's a different observation than the one they're trying to make. (Also, I would be pretty curious about whether the different subsampling produces superior file sizes.)

panzi 12y ago

5%? I don't know but I hoped for significantly more.

NVI 12y ago

Is there a command line tool available? I’d like to include it into my website build process.

joshmoz 12y ago
Yes, building the source will produce (among other things) a command line application called "cjpeg", which can encode or re-encode.

vlucas 12y ago

Facebook's $60,000 donation is interesting. The article says that the average file size reduction was 5%. Given that Facebook has built datacenters capable of storing exabytes of photo data in "cold storage", I wonder how much money Mozilla just saved Facebook in storage space costs in relation to how much they donated.

spacefight 12y ago
It would be interesting to learn what the cost of CPU overhead vs. additonal cost of 5% cold storage is - but I bet the FB folks did that calculation.

Daemon404 12y ago

It should be noted, since it is nowhere to be seen in this post, it breaks API and ABI while still presenting itself as libjpeg version 6 to the system, which is very evil. Open Issues: https://github.com/mozilla/mozjpeg/issues/67 https://github.com/mozilla/mozjpeg/issues/67 https://github.com/mozilla/mozjpeg/issues/21 https://github.com/mozilla/mozjpeg/issues/21

bastawhiz 12y ago

- Alpha channel - Animation - Proper metadata - File size The list of things that WebP offers that JPEG, PNG, or GIF alone will never address.