6 ms·

Beautifully written article -- I couldn't agree more. From my personal point of view the biggest impact are: 1) e-commerce textures such as cloths are more fai

Beautifully written article -- I couldn't agree more. From my personal point of view the biggest impact are: 1) e-commerce textures such as cloths are more faitfully represented -- more trust in e-commerce, more revenue 2) more equity for selfies, people-of-color skin is better represented in JPEG XL, selfies in general look more people like with less smoothing of skin performed at compression stage -- I'm a great friend of the idea that filtering should not be built into the compression so that all kinds of images can be transmitted 3) more compression, faster and cheaper to load 4) better progression -- unlike I believe AVIF does, JPEG XL progression does not impose more decoding cost for sequential, progression does not make compression density worse 5) thermal camera support -- I believe thermal cameras have huge ecological and cost optimization opportunities and more phones should have thermal cameras in them to facilitate the shift away from fossil fuels 6) small images -- thumbnails and sprites etc. do not necessarily need spriting with JPEG XL. That is a nice increase in system level simplicity. JPEG XL overhead is about 10-20 bytes whereas AVIF overhead seems to be 600 bytes or so. 7) 'it just works, no surprises' -- the worst case image quality degradation in any quality setting are much less severe and much less common than in other formats

est 4y ago
> people-of-color skin is better represented in JPEG XL wait what? What is this woodoo and why it itsn't the case for other image formats in which encodings color in RGB?
- DaiPlusPlus 4y ago
  Preface: It's difficult to write about stuff like this and I know reading it back to myself after will leave me wanting to redraft it 5 times over but it's 1am already. But, if this post is wrong/incorrect or is missing other factors that are too important to leave out then please let me know and I'll amend or even delete it. My focus is on the technical side of things. So anyway: > why it itsn't the case for other image formats in which encodings color in RGB? The term "RGB" doesn't mean anything w.r.t. gamut, gamma, dynamic-range, nor how necessarily non-linear transformations between color-spaces are performed. But all that's irrelevant: JPEG'S colour support is actually quite fine (...if you don't mind chroma subsampling), whereas the problem here relates to how JPEG decides what part of the image are important for high-quality preservation (more bits, more quality) than other areas (less bits, less quality). Obligatory ADHD relevant-yet-irrelevant diversion: this excellent (fun and interactive!) article about how JPEG works: https://parametric.press/issue-01/unraveling-the-jpeg/ https://parametric.press/issue-01/unraveling-the-jpeg/ (Caution: incoming egregious oversimplification of JPEG): Now, just imagine a RAW/DNG photo of the sky with some well-defined cumulus clouds in, and another RAW/DNG photo of equal pixel dimensions and bit-depth (so if they were *.bmp files they'd have identical size), except the second photo shows only a grey overcast sky (i.e. the sky is just one giant miserable old grey duvet that needs washing). Now, the _information-theoretic_ content of the cumulus clouds photo is higher than the overcast clouds photo; the clearly-defined edges of the cumulus clouds are considered "high-frequency" data while the low-constrast bulbous gradients that make-up the overcast photo are considered "low-frequency" data, and less overall information is required to reconstruct or represent that overcast sky compared to the cumulus sky. This relates to JPEG because JPEG isn't simply a single "JPEG algorithm", it's actually a sequence (or pipeline?) of completely different algorithms that each operate on the previous algorithm's output. The specific step that matters here is the part of JPEG that was designed with the knowledge that human vision perception doesn't need any high-frequency data in a scene where the subject is largely low-frequency data (i.e. we don't need a JPEG to preserve fine details in overcast clouds to correctly interpret a photo of overcast clouds, whereas contrarywise we do need JPEG to preserve fine-details that make-up the high-constrast edges between cumulus clouds and the blue sky behind them, otherwise we might think it's some other cloud formation with less well-defined edges (Altrostatus? Cirrostatus? Cirrus? I'm not a meteorologist - just someone on the spectrum using abusing analogies). So my point so-far is that JPEG looks for and preserves the details in areas of images (where high-constrast data is) while disregarding high-frequency data in an overall low-frequency scene where it thinks those details don't matter to us humans with our weird-to-a-computer visual perception system. I imagine by now I've made JPEG sound like a very effective image compression algorithm (well, it is...) - so where's the problem? ...well, consider that when a person has a low-level of melanin in their skin they will feature areas of high-contrast on their faces: their skin's canvas is light - while facial-features like folds, creases, crows' legs, varicose veins, spots, etc, are like the hard-edges of those cumulous clouds from earlier: the original (pre-compression) image data will feature a relatively high image constrast where those features lie, and JPEG will try to preserve the details of that constrast by using more bits. So that's neat: JPEG tries to ensure that the parts of us that of us that are detailed remain detailed after compression. ...but what if a person's face is naturally less...contrast-y? That's the problem: when a person simply has darker skin their faces will feature less contrast between the skin-parts of their face compared to the features of their face - as caught by a camera. So if a simplistic JPEG compressor is being run against a United Colors of Benetton coffee-table book or stills taken from the end of a certain Michael Jackson music video that I'm particularly fond of[3] then we will, as you can imagine, unfortunately, see the JPEG image files of people with darker skin will feature less definition and detail of their faces compared to the photos of people with lighter-skin, simply due to how areas of higher, and lower, contrast are _mechanically_ processed by JPEG. ...which means that sloppily applying JPEG[1] to photos of everyone might work great for people who have faces-with-naturally-high-contrast/high-frequency-information on them - but not-so-great for everyone else - and I think we can agree hat we can improve on that. ----------- Confounding things, the distribution of melanin in the JPEG development working group was not representative of the general population in the US at the time[5], let alone the rest of the world, which in-practice meant that less thought (if any) was paid to JPEG's stock handling of people with lower overall skin feature contrast (i.e. black people) - though no malice or ill-will or anything of the sort is required: like all of us (I assume...) they probably thought "these photographs of me (or people who generally look like me) processed by JPEG are good enough, it's been a long week, let's ship it and go for a pint" - without realising or appreciating that it wasn't "good enough" for large chunks of the population. At least, I hope that's the explanation... (and given Hanlon's Razor too). The situation can be compared to the... uh... colourful stories about the history of Kodak's colour film, its shortcomings, and the social-consequences thereof[2]. Or for something more recent: the misbehaving face-detecting webcams of the last decade that lead HP to make a certain public statement[3]. ----------- [1] I say carelesslessly because lots of tooling for JPEG, especially at the pro-level (think: Photoshop) gives you a lot of control over how much information-loss is allowed for each macroblock (e.g. Photohop's Save-for-Web dialog (the old, good one, not the new dumbed-down one) lets you use its own paintbrush tool to mask areas that should lose fewer bits (i.e. higher-quality) even if the JPEG compressor doesn't think there's much value in that part of the image. MSPaint, on the other hand, does not. [2] https://www.nytimes.com/2019/04/25/lens/sarah-lewis-racial-bias-photography.html https://www.nytimes.com/2019/04/25/lens/sarah-lewis-racial-b... [3] https://www.reuters.com/article/urnidgns852573c40069388000257693007c9b22/hp-our-webcams-arent-racist-idUS97608933020091222 https://www.reuters.com/article/urnidgns852573c4006938800025... [4] https://www.youtube.com/watch?v=F2AitTPI5U0 https://www.youtube.com/watch?v=F2AitTPI5U0 [5] I would like to mention that there was at least something for gender representation: the co-inventor of JPEG was a woman (no, I'm not referring to Lena.jpg) - Joan L. Mitchell, who sadly passed-away relatively young a few years ago: https://en.wikipedia.org/wiki/Joan_L._Mitchell https://en.wikipedia.org/wiki/Joan_L._Mitchell
  - jl6 4y ago
    There’s a Twitter thread by a JPEG XL dev who tries to explain some of this in terms of the XYB color space giving more bits to dark tones: https://mobile.twitter.com/jonsneyers/status/1550215859305586691 https://mobile.twitter.com/jonsneyers/status/155021585930558... But I’m not clear if this was a design goal of JPEG XL, or a theoretical side-benefit, or something you would need an HDR screen to appreciate (I couldn’t tell the difference in any of the example images, but those may have been processed by Twitter).
    - Semaphor 4y ago
      Also not sure if this is a Twitter issue (wait, another reason not to use twitter? So why do people keep using it for something it’s bad at? /rant), but I got a different result. I can clearly see the differences between the encodings (I shuffled the images and sorted them by perceived quality), and I disagree that JXL looks better / is less racist. For both sides, the AVIF version looks significantly better, but especially for the black woman. The white woman already has fewer visible details that can be lost, the black woman has more, and JXL just smoothes those over, making it look like someone used the smudge tool (in general, but it’s especially bad here [0]). So going from this example, I’d say AVIF wins by quite some margin. [0]: https://i.imgur.com/2WU6iKe.png https://i.imgur.com/2WU6iKe.png
      - jonsneyers 4y ago
        It was probably a bad idea to put such examples on twitter, since twitter recompression is quite aggressive and makes a big difference. I don't have the original decoded images nearby but it is best to do these kinds of comparisons without any recompression.
        Semaphor 4y ago
        I’d happily look at them again in their actual versions
    - vanderZwan 4y ago
- wahnfrieden 4y ago
  Why do you call this “woo” pejoratively right off the bat before admitting you don’t know?
  - vanderZwan 4y ago
    I suppose using a term that colloquially means "black magic" might be a little (skin) tone deaf in this context, but I think it was intended as a positive thing. As in "what kind of magic is JPEG XL using to be better, and what is everyone else doing wrong?" I admit the wording is kind of ambiguous though
- JyrkiAlakuijala 4y ago
  Some of it relates to the modeling of spontaneous excitation bias of opsin molecule releasing their energy in the retina. These are modeled in the compression curve in JPEG XL by a bias before the compression non-linearity, whereas other codecs don't have that bias unless they use sRGB -- sRGB has modeled the bias as a short linear slope, but compresses in wrong directions (RGB) in the color space, not LMS. In JPEG XL, the compression curves happen in the direction of LMS reception directions, not in RGB, that makes reds, browns, oranges, purple colors a lot more uniform in quantization. Finally, more emphasis was spend with visual masking in JPEG XL, to avoid blurring in 'fragile' textures even if they are in the same compression block with sharp edges.
  - pizza 4y ago
    Hi Jyrki, this is all very interesting. A couple questions for you, while you’re in this thread: - where is the best documentation for the Butteraugli metric? I’ve found it quite difficult to find a high-level rationale/design notes for the metric. - I also understand that JXL doesn’t try to explicitly optimize against Butteraugli distance unless you invoke the higher effort levels, and then, it uses a slightly different function than the original Butteraugli distance - is that correct? - One q about the distance parameter -d; if I pass -d 1 -E x for some x, I’m asking for “an image with at most Butteraugli distance 1”, and if I pick another effort level higher than x, am I asking it to introduce a little bit more loss, so that the Butteraugli distance is likely to increase? Ie is the distance parameter more of an “upper bound for acceptable loss” - I’ve seen Butteraugli values quoted for different “norms/p-norms” - which one do I use? - How do I learn more about these colorspace transfer functions? Thanks again for your awesome work!
    - JyrkiAlakuijala 4y ago
      1) Butteraugli Butteraugli is largerly a 'learned' metric, just very small with its ~200 parameters. I implemented possible components and tuned them and their interactions using methods similar to machine learning. Describing it to humans should be possible, but going all the way is going to be the same as describing DNNs. SSIMULACRA 2 is taking the same approach and getting far better results than a previous effort SSIMULACRA. 2) butteraugli distance invoked in the higher effort levels JXL heuristics are optimized to give good butteraugli scores. These heuristics are mostly used at speeds 6 and 7. Speed 8 and 9 no longer trust solely on the heuristics, but they try to optimize a constant butteraugli score. It is important to note that they don't do RD-optimization, so they are quite bad at lower quality -- they will drive the quality bad no matter what. There is a hard limit for not reducing the quality too much in relation to the heuristics at speeds 6 and 7, but it is not RD optimization. Small savings can lead to large degradations. This is ok with near visually lossless (quality 90+), but not good for low quality (< 50). Butteraugli has slightly evolved through the times, the latest version is available in libjxl and used by cjxl at effort 8 and 9. 3) -d 1 “an image with at most Butteraugli distance 1” ? It used to work that way, but there was some inflation of this concept, and often the maximum score is twice the asked value. Ideally the maximum should be the given value, but it is very expensive to guarantee. In guetzli we have such guarantees, but it runs 1000x slower. 4) p-norm usual metrics (SSIM, PSNR, VMAF) aggregate error over the image to a single number using the 2nd norm we use a higher norm, and actually a combination of norms. If you ask for a 3rd norm, it is an equal combination of 3rd, 6th and 12th norm. Using higher norms puts more emphasis on large errors somewhere than 2nd norm. The better the quality in image compression, best human rater correlations can be found in higher norms. 3rd norm is a good allrounder, I'd usually use 6th norm myself. A lower norm can be better for automated optimization since the cost function will have less wrinkles. 5) colorspace we don't have a good write up other than possibly the JPEG XL spec. there is a javascript implementation of xyb here: https://gist.github.com/mattdesl/c0ad8d54b3491c7fd39e222e698b2a30 https://gist.github.com/mattdesl/c0ad8d54b3491c7fd39e222e698... Please consider joining the JPEG XL discord. I'm available for guiding you to learn more if you like.
conaclos 4y ago
Could you elaborate on 5) thermal camera support? How thermal camera could replace normal camera (this only capture infrared light?)? Why thermal camera are more ecological-friendly?
- syrgian 4y ago
  It would not replace the existing cameras. A thermal camera is a great tool for energy conservation: you can find misbehaving appliances consuming too much electricity idling, compare waste of different appliances, heat leaks at home, find hotspots in devices you maintain (say, a badly connected cable can offer extra resistance and become hot). Though I don't think thermal cameras will be user affordable for a long while.
  - edent 4y ago
    There are already cheap (ish) add-on cameras for Android and iOS. For example my review of https://shkspr.mobi/blog/2021/10/gadget-review-seek-infrared-camera/ https://shkspr.mobi/blog/2021/10/gadget-review-seek-infrared... There are a bunch of Android phones which already have this tech built in. Some are high end, but a surprising number of mid-range devices as well.
  - danielheath 4y ago
    I picked up a basic one for about $60 on aliexpress. It's good enough to find heat leaks at home, but only barely, and to get usable results I had to turn the home heating to maximum (to increase the thermal contrast). The ones that let you just walk into a house and immediately see cold areas cost more like $800.
vanderZwan 4y ago
Personally, even if Chrome goes ahead with removing support I hope JPEG XL will "win" in the long run by being an attractive "universal" format for all other use-cases for image compression. So in other words: that it finds an ecosystem elsewhere, slowly takes over outside of the browsers, and then the browsers will adopt it after all. For example, I remember reading somewhere that there are people looking at whether or not it makes sense to convert all of the existing JPEGs to JPEG XL on their servers for storage, then reconstruct the JPEG when needed. If so, then there is a niche where it can find adoption even if browsers don't support it yet. It also looks like it might find a use for medical and scientific imaging. What I personally really would like to know is if it would be possible to losslessly recompress the RAW files from my camera. It looks like it should support everything required, and the claim that it "is the first serious candidate to become a universal image format that “works across the workflow”, in the sense that it is suitable for the lifecycle of a digital image, from capture and authoring to interchange, archival, and delivery" kind of implies that it is, but I haven't seen anyone say it out loud yet. Because I must say: DNG is really slow and clunky to work with, and I bet has terrible compression on top of that.
- ksec 4y ago
  What if Google demand every Android devices to support AVIF and not JPEG XL? This isn't a fight about whether something is technically superior, it is about marketing, cult and ideology. It wont be long before major force of AOM supporters arrives with new PR points to fight back.
  - magicalist 4y ago
    > What if Google demand every Android devices to support AVIF and not JPEG XL? Has there ever been a case when a company has demanded a device not support a codec that wasn't a patent dispute? And what does it mean for a device not to support an image format? Almost all image decoding is done in software. This seems like an unfounded concern.
    - vanderZwan 4y ago

est 4y ago

> people-of-color skin is better represented in JPEG XL wait what? What is this woodoo and why it itsn't the case for other image formats in which encodings color in RGB?

DaiPlusPlus 4y ago
Preface: It's difficult to write about stuff like this and I know reading it back to myself after will leave me wanting to redraft it 5 times over but it's 1am already. But, if this post is wrong/incorrect or is missing other factors that are too important to leave out then please let me know and I'll amend or even delete it. My focus is on the technical side of things. So anyway: > why it itsn't the case for other image formats in which encodings color in RGB? The term "RGB" doesn't mean anything w.r.t. gamut, gamma, dynamic-range, nor how necessarily non-linear transformations between color-spaces are performed. But all that's irrelevant: JPEG'S colour support is actually quite fine (...if you don't mind chroma subsampling), whereas the problem here relates to how JPEG decides what part of the image are important for high-quality preservation (more bits, more quality) than other areas (less bits, less quality). Obligatory ADHD relevant-yet-irrelevant diversion: this excellent (fun and interactive!) article about how JPEG works: https://parametric.press/issue-01/unraveling-the-jpeg/ https://parametric.press/issue-01/unraveling-the-jpeg/ (Caution: incoming egregious oversimplification of JPEG): Now, just imagine a RAW/DNG photo of the sky with some well-defined cumulus clouds in, and another RAW/DNG photo of equal pixel dimensions and bit-depth (so if they were *.bmp files they'd have identical size), except the second photo shows only a grey overcast sky (i.e. the sky is just one giant miserable old grey duvet that needs washing). Now, the _information-theoretic_ content of the cumulus clouds photo is higher than the overcast clouds photo; the clearly-defined edges of the cumulus clouds are considered "high-frequency" data while the low-constrast bulbous gradients that make-up the overcast photo are considered "low-frequency" data, and less overall information is required to reconstruct or represent that overcast sky compared to the cumulus sky. This relates to JPEG because JPEG isn't simply a single "JPEG algorithm", it's actually a sequence (or pipeline?) of completely different algorithms that each operate on the previous algorithm's output. The specific step that matters here is the part of JPEG that was designed with the knowledge that human vision perception doesn't need any high-frequency data in a scene where the subject is largely low-frequency data (i.e. we don't need a JPEG to preserve fine details in overcast clouds to correctly interpret a photo of overcast clouds, whereas contrarywise we do need JPEG to preserve fine-details that make-up the high-constrast edges between cumulus clouds and the blue sky behind them, otherwise we might think it's some other cloud formation with less well-defined edges (Altrostatus? Cirrostatus? Cirrus? I'm not a meteorologist - just someone on the spectrum using abusing analogies). So my point so-far is that JPEG looks for and preserves the details in areas of images (where high-constrast data is) while disregarding high-frequency data in an overall low-frequency scene where it thinks those details don't matter to us humans with our weird-to-a-computer visual perception system. I imagine by now I've made JPEG sound like a very effective image compression algorithm (well, it is...) - so where's the problem? ...well, consider that when a person has a low-level of melanin in their skin they will feature areas of high-contrast on their faces: their skin's canvas is light - while facial-features like folds, creases, crows' legs, varicose veins, spots, etc, are like the hard-edges of those cumulous clouds from earlier: the original (pre-compression) image data will feature a relatively high image constrast where those features lie, and JPEG will try to preserve the details of that constrast by using more bits. So that's neat: JPEG tries to ensure that the parts of us that of us that are detailed remain detailed after compression. ...but what if a person's face is naturally less...contrast-y? That's the problem: when a person simply has darker skin their faces will feature less contrast between the skin-parts of their face compared to the features of their face - as caught by a camera. So if a simplistic JPEG compressor is being run against a United Colors of Benetton coffee-table book or stills taken from the end of a certain Michael Jackson music video that I'm particularly fond of[3] then we will, as you can imagine, unfortunately, see the JPEG image files of people with darker skin will feature less definition and detail of their faces compared to the photos of people with lighter-skin, simply due to how areas of higher, and lower, contrast are _mechanically_ processed by JPEG. ...which means that sloppily applying JPEG[1] to photos of everyone might work great for people who have faces-with-naturally-high-contrast/high-frequency-information on them - but not-so-great for everyone else - and I think we can agree hat we can improve on that. ----------- Confounding things, the distribution of melanin in the JPEG development working group was not representative of the general population in the US at the time[5], let alone the rest of the world, which in-practice meant that less thought (if any) was paid to JPEG's stock handling of people with lower overall skin feature contrast (i.e. black people) - though no malice or ill-will or anything of the sort is required: like all of us (I assume...) they probably thought "these photographs of me (or people who generally look like me) processed by JPEG are good enough, it's been a long week, let's ship it and go for a pint" - without realising or appreciating that it wasn't "good enough" for large chunks of the population. At least, I hope that's the explanation... (and given Hanlon's Razor too). The situation can be compared to the... uh... colourful stories about the history of Kodak's colour film, its shortcomings, and the social-consequences thereof[2]. Or for something more recent: the misbehaving face-detecting webcams of the last decade that lead HP to make a certain public statement[3]. ----------- [1] I say carelesslessly because lots of tooling for JPEG, especially at the pro-level (think: Photoshop) gives you a lot of control over how much information-loss is allowed for each macroblock (e.g. Photohop's Save-for-Web dialog (the old, good one, not the new dumbed-down one) lets you use its own paintbrush tool to mask areas that should lose fewer bits (i.e. higher-quality) even if the JPEG compressor doesn't think there's much value in that part of the image. MSPaint, on the other hand, does not. [2] https://www.nytimes.com/2019/04/25/lens/sarah-lewis-racial-bias-photography.html https://www.nytimes.com/2019/04/25/lens/sarah-lewis-racial-b... [3] https://www.reuters.com/article/urnidgns852573c40069388000257693007c9b22/hp-our-webcams-arent-racist-idUS97608933020091222 https://www.reuters.com/article/urnidgns852573c4006938800025... [4] https://www.youtube.com/watch?v=F2AitTPI5U0 https://www.youtube.com/watch?v=F2AitTPI5U0 [5] I would like to mention that there was at least something for gender representation: the co-inventor of JPEG was a woman (no, I'm not referring to Lena.jpg) - Joan L. Mitchell, who sadly passed-away relatively young a few years ago: https://en.wikipedia.org/wiki/Joan_L._Mitchell https://en.wikipedia.org/wiki/Joan_L._Mitchell
- jl6 4y ago
  There’s a Twitter thread by a JPEG XL dev who tries to explain some of this in terms of the XYB color space giving more bits to dark tones: https://mobile.twitter.com/jonsneyers/status/1550215859305586691 https://mobile.twitter.com/jonsneyers/status/155021585930558... But I’m not clear if this was a design goal of JPEG XL, or a theoretical side-benefit, or something you would need an HDR screen to appreciate (I couldn’t tell the difference in any of the example images, but those may have been processed by Twitter).
  - Semaphor 4y ago
    Also not sure if this is a Twitter issue (wait, another reason not to use twitter? So why do people keep using it for something it’s bad at? /rant), but I got a different result. I can clearly see the differences between the encodings (I shuffled the images and sorted them by perceived quality), and I disagree that JXL looks better / is less racist. For both sides, the AVIF version looks significantly better, but especially for the black woman. The white woman already has fewer visible details that can be lost, the black woman has more, and JXL just smoothes those over, making it look like someone used the smudge tool (in general, but it’s especially bad here [0]). So going from this example, I’d say AVIF wins by quite some margin. [0]: https://i.imgur.com/2WU6iKe.png https://i.imgur.com/2WU6iKe.png
    - jonsneyers 4y ago
      It was probably a bad idea to put such examples on twitter, since twitter recompression is quite aggressive and makes a big difference. I don't have the original decoded images nearby but it is best to do these kinds of comparisons without any recompression.
      - Semaphor 4y ago
        I’d happily look at them again in their actual versions
  - vanderZwan 4y ago
wahnfrieden 4y ago
Why do you call this “woo” pejoratively right off the bat before admitting you don’t know?
- vanderZwan 4y ago
  I suppose using a term that colloquially means "black magic" might be a little (skin) tone deaf in this context, but I think it was intended as a positive thing. As in "what kind of magic is JPEG XL using to be better, and what is everyone else doing wrong?" I admit the wording is kind of ambiguous though
JyrkiAlakuijala 4y ago
Some of it relates to the modeling of spontaneous excitation bias of opsin molecule releasing their energy in the retina. These are modeled in the compression curve in JPEG XL by a bias before the compression non-linearity, whereas other codecs don't have that bias unless they use sRGB -- sRGB has modeled the bias as a short linear slope, but compresses in wrong directions (RGB) in the color space, not LMS. In JPEG XL, the compression curves happen in the direction of LMS reception directions, not in RGB, that makes reds, browns, oranges, purple colors a lot more uniform in quantization. Finally, more emphasis was spend with visual masking in JPEG XL, to avoid blurring in 'fragile' textures even if they are in the same compression block with sharp edges.
- pizza 4y ago
  Hi Jyrki, this is all very interesting. A couple questions for you, while you’re in this thread: - where is the best documentation for the Butteraugli metric? I’ve found it quite difficult to find a high-level rationale/design notes for the metric. - I also understand that JXL doesn’t try to explicitly optimize against Butteraugli distance unless you invoke the higher effort levels, and then, it uses a slightly different function than the original Butteraugli distance - is that correct? - One q about the distance parameter -d; if I pass -d 1 -E x for some x, I’m asking for “an image with at most Butteraugli distance 1”, and if I pick another effort level higher than x, am I asking it to introduce a little bit more loss, so that the Butteraugli distance is likely to increase? Ie is the distance parameter more of an “upper bound for acceptable loss” - I’ve seen Butteraugli values quoted for different “norms/p-norms” - which one do I use? - How do I learn more about these colorspace transfer functions? Thanks again for your awesome work!
  - JyrkiAlakuijala 4y ago
    1) Butteraugli Butteraugli is largerly a 'learned' metric, just very small with its ~200 parameters. I implemented possible components and tuned them and their interactions using methods similar to machine learning. Describing it to humans should be possible, but going all the way is going to be the same as describing DNNs. SSIMULACRA 2 is taking the same approach and getting far better results than a previous effort SSIMULACRA. 2) butteraugli distance invoked in the higher effort levels JXL heuristics are optimized to give good butteraugli scores. These heuristics are mostly used at speeds 6 and 7. Speed 8 and 9 no longer trust solely on the heuristics, but they try to optimize a constant butteraugli score. It is important to note that they don't do RD-optimization, so they are quite bad at lower quality -- they will drive the quality bad no matter what. There is a hard limit for not reducing the quality too much in relation to the heuristics at speeds 6 and 7, but it is not RD optimization. Small savings can lead to large degradations. This is ok with near visually lossless (quality 90+), but not good for low quality (< 50). Butteraugli has slightly evolved through the times, the latest version is available in libjxl and used by cjxl at effort 8 and 9. 3) -d 1 “an image with at most Butteraugli distance 1” ? It used to work that way, but there was some inflation of this concept, and often the maximum score is twice the asked value. Ideally the maximum should be the given value, but it is very expensive to guarantee. In guetzli we have such guarantees, but it runs 1000x slower. 4) p-norm usual metrics (SSIM, PSNR, VMAF) aggregate error over the image to a single number using the 2nd norm we use a higher norm, and actually a combination of norms. If you ask for a 3rd norm, it is an equal combination of 3rd, 6th and 12th norm. Using higher norms puts more emphasis on large errors somewhere than 2nd norm. The better the quality in image compression, best human rater correlations can be found in higher norms. 3rd norm is a good allrounder, I'd usually use 6th norm myself. A lower norm can be better for automated optimization since the cost function will have less wrinkles. 5) colorspace we don't have a good write up other than possibly the JPEG XL spec. there is a javascript implementation of xyb here: https://gist.github.com/mattdesl/c0ad8d54b3491c7fd39e222e698b2a30 https://gist.github.com/mattdesl/c0ad8d54b3491c7fd39e222e698... Please consider joining the JPEG XL discord. I'm available for guiding you to learn more if you like.

conaclos 4y ago

Could you elaborate on 5) thermal camera support? How thermal camera could replace normal camera (this only capture infrared light?)? Why thermal camera are more ecological-friendly?

syrgian 4y ago
It would not replace the existing cameras. A thermal camera is a great tool for energy conservation: you can find misbehaving appliances consuming too much electricity idling, compare waste of different appliances, heat leaks at home, find hotspots in devices you maintain (say, a badly connected cable can offer extra resistance and become hot). Though I don't think thermal cameras will be user affordable for a long while.
- edent 4y ago
  There are already cheap (ish) add-on cameras for Android and iOS. For example my review of https://shkspr.mobi/blog/2021/10/gadget-review-seek-infrared-camera/ https://shkspr.mobi/blog/2021/10/gadget-review-seek-infrared... There are a bunch of Android phones which already have this tech built in. Some are high end, but a surprising number of mid-range devices as well.
- danielheath 4y ago
  I picked up a basic one for about $60 on aliexpress. It's good enough to find heat leaks at home, but only barely, and to get usable results I had to turn the home heating to maximum (to increase the thermal contrast). The ones that let you just walk into a house and immediately see cold areas cost more like $800.

vanderZwan 4y ago

Personally, even if Chrome goes ahead with removing support I hope JPEG XL will "win" in the long run by being an attractive "universal" format for all other use-cases for image compression. So in other words: that it finds an ecosystem elsewhere, slowly takes over outside of the browsers, and then the browsers will adopt it after all. For example, I remember reading somewhere that there are people looking at whether or not it makes sense to convert all of the existing JPEGs to JPEG XL on their servers for storage, then reconstruct the JPEG when needed. If so, then there is a niche where it can find adoption even if browsers don't support it yet. It also looks like it might find a use for medical and scientific imaging. What I personally really would like to know is if it would be possible to losslessly recompress the RAW files from my camera. It looks like it should support everything required, and the claim that it "is the first serious candidate to become a universal image format that “works across the workflow”, in the sense that it is suitable for the lifecycle of a digital image, from capture and authoring to interchange, archival, and delivery" kind of implies that it is, but I haven't seen anyone say it out loud yet. Because I must say: DNG is really slow and clunky to work with, and I bet has terrible compression on top of that.

ksec 4y ago
What if Google demand every Android devices to support AVIF and not JPEG XL? This isn't a fight about whether something is technically superior, it is about marketing, cult and ideology. It wont be long before major force of AOM supporters arrives with new PR points to fight back.
- magicalist 4y ago
  > What if Google demand every Android devices to support AVIF and not JPEG XL? Has there ever been a case when a company has demanded a device not support a codec that wasn't a patent dispute? And what does it mean for a device not to support an image format? Almost all image decoding is done in software. This seems like an unfounded concern.
  - vanderZwan 4y ago