Preface: It's difficult to write about stuff like this and I know reading it back to myself after will leave me wanting to redraft it 5 times over but it's 1am

Preface: It's difficult to write about stuff like this and I know reading it back to myself after will leave me wanting to redraft it 5 times over but it's 1am already. But, if this post is wrong/incorrect or is missing other factors that are too important to leave out then please let me know and I'll amend or even delete it. My focus is on the technical side of things. So anyway: > why it itsn't the case for other image formats in which encodings color in RGB? The term "RGB" doesn't mean anything w.r.t. gamut, gamma, dynamic-range, nor how necessarily non-linear transformations between color-spaces are performed. But all that's irrelevant: JPEG'S colour support is actually quite fine (...if you don't mind chroma subsampling), whereas the problem here relates to how JPEG decides what part of the image are important for high-quality preservation (more bits, more quality) than other areas (less bits, less quality). Obligatory ADHD relevant-yet-irrelevant diversion: this excellent (fun and interactive!) article about how JPEG works: https://parametric.press/issue-01/unraveling-the-jpeg/ https://parametric.press/issue-01/unraveling-the-jpeg/ (Caution: incoming egregious oversimplification of JPEG): Now, just imagine a RAW/DNG photo of the sky with some well-defined cumulus clouds in, and another RAW/DNG photo of equal pixel dimensions and bit-depth (so if they were *.bmp files they'd have identical size), except the second photo shows only a grey overcast sky (i.e. the sky is just one giant miserable old grey duvet that needs washing). Now, the _information-theoretic_ content of the cumulus clouds photo is higher than the overcast clouds photo; the clearly-defined edges of the cumulus clouds are considered "high-frequency" data while the low-constrast bulbous gradients that make-up the overcast photo are considered "low-frequency" data, and less overall information is required to reconstruct or represent that overcast sky compared to the cumulus sky. This relates to JPEG because JPEG isn't simply a single "JPEG algorithm", it's actually a sequence (or pipeline?) of completely different algorithms that each operate on the previous algorithm's output. The specific step that matters here is the part of JPEG that was designed with the knowledge that human vision perception doesn't need any high-frequency data in a scene where the subject is largely low-frequency data (i.e. we don't need a JPEG to preserve fine details in overcast clouds to correctly interpret a photo of overcast clouds, whereas contrarywise we do need JPEG to preserve fine-details that make-up the high-constrast edges between cumulus clouds and the blue sky behind them, otherwise we might think it's some other cloud formation with less well-defined edges (Altrostatus? Cirrostatus? Cirrus? I'm not a meteorologist - just someone on the spectrum using abusing analogies). So my point so-far is that JPEG looks for and preserves the details in areas of images (where high-constrast data is) while disregarding high-frequency data in an overall low-frequency scene where it thinks those details don't matter to us humans with our weird-to-a-computer visual perception system. I imagine by now I've made JPEG sound like a very effective image compression algorithm (well, it is...) - so where's the problem? ...well, consider that when a person has a low-level of melanin in their skin they will feature areas of high-contrast on their faces: their skin's canvas is light - while facial-features like folds, creases, crows' legs, varicose veins, spots, etc, are like the hard-edges of those cumulous clouds from earlier: the original (pre-compression) image data will feature a relatively high image constrast where those features lie, and JPEG will try to preserve the details of that constrast by using more bits. So that's neat: JPEG tries to ensure that the parts of us that of us that are detailed remain detailed after compression. ...but what if a person's face is naturally less...contrast-y? That's the problem: when a person simply has darker skin their faces will feature less contrast between the skin-parts of their face compared to the features of their face - as caught by a camera. So if a simplistic JPEG compressor is being run against a United Colors of Benetton coffee-table book or stills taken from the end of a certain Michael Jackson music video that I'm particularly fond of[3] then we will, as you can imagine, unfortunately, see the JPEG image files of people with darker skin will feature less definition and detail of their faces compared to the photos of people with lighter-skin, simply due to how areas of higher, and lower, contrast are _mechanically_ processed by JPEG. ...which means that sloppily applying JPEG[1] to photos of everyone might work great for people who have faces-with-naturally-high-contrast/high-frequency-information on them - but not-so-great for everyone else - and I think we can agree hat we can improve on that. ----------- Confounding things, the distribution of melanin in the JPEG development working group was not representative of the general population in the US at the time[5], let alone the rest of the world, which in-practice meant that less thought (if any) was paid to JPEG's stock handling of people with lower overall skin feature contrast (i.e. black people) - though no malice or ill-will or anything of the sort is required: like all of us (I assume...) they probably thought "these photographs of me (or people who generally look like me) processed by JPEG are good enough, it's been a long week, let's ship it and go for a pint" - without realising or appreciating that it wasn't "good enough" for large chunks of the population. At least, I hope that's the explanation... (and given Hanlon's Razor too). The situation can be compared to the... uh... colourful stories about the history of Kodak's colour film, its shortcomings, and the social-consequences thereof[2]. Or for something more recent: the misbehaving face-detecting webcams of the last decade that lead HP to make a certain public statement[3]. ----------- [1] I say carelesslessly because lots of tooling for JPEG, especially at the pro-level (think: Photoshop) gives you a lot of control over how much information-loss is allowed for each macroblock (e.g. Photohop's Save-for-Web dialog (the old, good one, not the new dumbed-down one) lets you use its own paintbrush tool to mask areas that should lose fewer bits (i.e. higher-quality) even if the JPEG compressor doesn't think there's much value in that part of the image. MSPaint, on the other hand, does not. [2] https://www.nytimes.com/2019/04/25/lens/sarah-lewis-racial-bias-photography.html https://www.nytimes.com/2019/04/25/lens/sarah-lewis-racial-b... [3] https://www.reuters.com/article/urnidgns852573c40069388000257693007c9b22/hp-our-webcams-arent-racist-idUS97608933020091222 https://www.reuters.com/article/urnidgns852573c4006938800025... [4] https://www.youtube.com/watch?v=F2AitTPI5U0 https://www.youtube.com/watch?v=F2AitTPI5U0 [5] I would like to mention that there was at least something for gender representation: the co-inventor of JPEG was a woman (no, I'm not referring to Lena.jpg) - Joan L. Mitchell, who sadly passed-away relatively young a few years ago: https://en.wikipedia.org/wiki/Joan_L._Mitchell https://en.wikipedia.org/wiki/Joan_L._Mitchell