4 ms·

Some of it relates to the modeling of spontaneous excitation bias of opsin molecule releasing their energy in the retina. These are modeled in the compression c

Some of it relates to the modeling of spontaneous excitation bias of opsin molecule releasing their energy in the retina. These are modeled in the compression curve in JPEG XL by a bias before the compression non-linearity, whereas other codecs don't have that bias unless they use sRGB -- sRGB has modeled the bias as a short linear slope, but compresses in wrong directions (RGB) in the color space, not LMS. In JPEG XL, the compression curves happen in the direction of LMS reception directions, not in RGB, that makes reds, browns, oranges, purple colors a lot more uniform in quantization. Finally, more emphasis was spend with visual masking in JPEG XL, to avoid blurring in 'fragile' textures even if they are in the same compression block with sharp edges.

pizza 4y ago
Hi Jyrki, this is all very interesting. A couple questions for you, while you’re in this thread: - where is the best documentation for the Butteraugli metric? I’ve found it quite difficult to find a high-level rationale/design notes for the metric. - I also understand that JXL doesn’t try to explicitly optimize against Butteraugli distance unless you invoke the higher effort levels, and then, it uses a slightly different function than the original Butteraugli distance - is that correct? - One q about the distance parameter -d; if I pass -d 1 -E x for some x, I’m asking for “an image with at most Butteraugli distance 1”, and if I pick another effort level higher than x, am I asking it to introduce a little bit more loss, so that the Butteraugli distance is likely to increase? Ie is the distance parameter more of an “upper bound for acceptable loss” - I’ve seen Butteraugli values quoted for different “norms/p-norms” - which one do I use? - How do I learn more about these colorspace transfer functions? Thanks again for your awesome work!
- JyrkiAlakuijala 4y ago
  1) Butteraugli Butteraugli is largerly a 'learned' metric, just very small with its ~200 parameters. I implemented possible components and tuned them and their interactions using methods similar to machine learning. Describing it to humans should be possible, but going all the way is going to be the same as describing DNNs. SSIMULACRA 2 is taking the same approach and getting far better results than a previous effort SSIMULACRA. 2) butteraugli distance invoked in the higher effort levels JXL heuristics are optimized to give good butteraugli scores. These heuristics are mostly used at speeds 6 and 7. Speed 8 and 9 no longer trust solely on the heuristics, but they try to optimize a constant butteraugli score. It is important to note that they don't do RD-optimization, so they are quite bad at lower quality -- they will drive the quality bad no matter what. There is a hard limit for not reducing the quality too much in relation to the heuristics at speeds 6 and 7, but it is not RD optimization. Small savings can lead to large degradations. This is ok with near visually lossless (quality 90+), but not good for low quality (< 50). Butteraugli has slightly evolved through the times, the latest version is available in libjxl and used by cjxl at effort 8 and 9. 3) -d 1 “an image with at most Butteraugli distance 1” ? It used to work that way, but there was some inflation of this concept, and often the maximum score is twice the asked value. Ideally the maximum should be the given value, but it is very expensive to guarantee. In guetzli we have such guarantees, but it runs 1000x slower. 4) p-norm usual metrics (SSIM, PSNR, VMAF) aggregate error over the image to a single number using the 2nd norm we use a higher norm, and actually a combination of norms. If you ask for a 3rd norm, it is an equal combination of 3rd, 6th and 12th norm. Using higher norms puts more emphasis on large errors somewhere than 2nd norm. The better the quality in image compression, best human rater correlations can be found in higher norms. 3rd norm is a good allrounder, I'd usually use 6th norm myself. A lower norm can be better for automated optimization since the cost function will have less wrinkles. 5) colorspace we don't have a good write up other than possibly the JPEG XL spec. there is a javascript implementation of xyb here: https://gist.github.com/mattdesl/c0ad8d54b3491c7fd39e222e698b2a30 https://gist.github.com/mattdesl/c0ad8d54b3491c7fd39e222e698... Please consider joining the JPEG XL discord. I'm available for guiding you to learn more if you like.
  - pizza 4y ago
    Awesome, thank you!

pizza 4y ago

Hi Jyrki, this is all very interesting. A couple questions for you, while you’re in this thread: - where is the best documentation for the Butteraugli metric? I’ve found it quite difficult to find a high-level rationale/design notes for the metric. - I also understand that JXL doesn’t try to explicitly optimize against Butteraugli distance unless you invoke the higher effort levels, and then, it uses a slightly different function than the original Butteraugli distance - is that correct? - One q about the distance parameter -d; if I pass -d 1 -E x for some x, I’m asking for “an image with at most Butteraugli distance 1”, and if I pick another effort level higher than x, am I asking it to introduce a little bit more loss, so that the Butteraugli distance is likely to increase? Ie is the distance parameter more of an “upper bound for acceptable loss” - I’ve seen Butteraugli values quoted for different “norms/p-norms” - which one do I use? - How do I learn more about these colorspace transfer functions? Thanks again for your awesome work!

JyrkiAlakuijala 4y ago
1) Butteraugli Butteraugli is largerly a 'learned' metric, just very small with its ~200 parameters. I implemented possible components and tuned them and their interactions using methods similar to machine learning. Describing it to humans should be possible, but going all the way is going to be the same as describing DNNs. SSIMULACRA 2 is taking the same approach and getting far better results than a previous effort SSIMULACRA. 2) butteraugli distance invoked in the higher effort levels JXL heuristics are optimized to give good butteraugli scores. These heuristics are mostly used at speeds 6 and 7. Speed 8 and 9 no longer trust solely on the heuristics, but they try to optimize a constant butteraugli score. It is important to note that they don't do RD-optimization, so they are quite bad at lower quality -- they will drive the quality bad no matter what. There is a hard limit for not reducing the quality too much in relation to the heuristics at speeds 6 and 7, but it is not RD optimization. Small savings can lead to large degradations. This is ok with near visually lossless (quality 90+), but not good for low quality (< 50). Butteraugli has slightly evolved through the times, the latest version is available in libjxl and used by cjxl at effort 8 and 9. 3) -d 1 “an image with at most Butteraugli distance 1” ? It used to work that way, but there was some inflation of this concept, and often the maximum score is twice the asked value. Ideally the maximum should be the given value, but it is very expensive to guarantee. In guetzli we have such guarantees, but it runs 1000x slower. 4) p-norm usual metrics (SSIM, PSNR, VMAF) aggregate error over the image to a single number using the 2nd norm we use a higher norm, and actually a combination of norms. If you ask for a 3rd norm, it is an equal combination of 3rd, 6th and 12th norm. Using higher norms puts more emphasis on large errors somewhere than 2nd norm. The better the quality in image compression, best human rater correlations can be found in higher norms. 3rd norm is a good allrounder, I'd usually use 6th norm myself. A lower norm can be better for automated optimization since the cost function will have less wrinkles. 5) colorspace we don't have a good write up other than possibly the JPEG XL spec. there is a javascript implementation of xyb here: https://gist.github.com/mattdesl/c0ad8d54b3491c7fd39e222e698b2a30 https://gist.github.com/mattdesl/c0ad8d54b3491c7fd39e222e698... Please consider joining the JPEG XL discord. I'm available for guiding you to learn more if you like.
- pizza 4y ago
  Awesome, thank you!