6 ms·

The non-Riemannian nature of perceptual color space (2022)

85 points by cpach 2y ago

bbor 2y ago
The paper in PDF: https://www.pnas.org/doi/pdf/10.1073/pnas.2119753119?download=true https://www.pnas.org/doi/pdf/10.1073/pnas.2119753119?downloa... there’s a lot of math symbols but IMO the usage is pretty straightforward for anyone familiar with linear algebra and proofs. Ramsay (30) suggests that the principle of diminishing returns or its opposite may have been spuriously identified by otherresearchers (daringly including his PhD advisor, Helm) because of experimental procedures (successive intervals, paired compar-isons) ill suited to the task. Indeed, open-ended and criterion-dependent tasks give less accurate measures of similarity because of individual factors and the difficulty of the task… Instead, we use a more reliable two-alternative forced choice (2AFC) task, where the participant simply answers the following question: "Which is more different?" Specifically, we use a triad arrangement of stimuli with the reference in the middle and one test on either side. Each of 320 triads covering the neutral axis was judged by at least 250 different participants in a crowdsourced study on Amazon Mechanical Turk (MTurk) (52).* The core of it seems strong, IMO. I would have liked a bit more cognitive science / philosophy to distinguish between an ideal color space and whatever imperfect model is created by individual human’s neural software, and skipping Goethe and Schopenhauer in the intro was downright criminal, but that’s more on me than them. Color has long been a convenient “entry point” into the study of human unconscious and subconscious data transformations, most notably through Wittgenstein’s musings about red squares. I look forward to this technique being extended to other spaces, even ones as abstract as “moral space” or “valence space”!
- robwwilliams 2y ago
  Great comment. You made me laugh out loud on the other spaces we HAVE TO NAIL DOWN. You would get a kick out of Winograd and Flores 1986.
piannucci 2y ago
I love the drama of how the abstract is written, but TBH I don't think this is a surprise. I believe it's well-known among color theorists that large perceptual distances are inconsistent with sums of small differences. So maybe the most generous thing to say here is, good on them for bringing awareness of this subtlety to a broader audience.
- robwwilliams 2y ago
  Yep: color space is a pragmatic kludge not a “real thing” divorced from the human neural network that is its fundamental basis.
- mpweiher 2y ago
  Not only that, it is also well known that the smallest perceptible color difference (ΔE=1) is not actually consistent, even in the "perceptually uniform" color spaces. So ΔE is actually 1 in some parts of the space, but up to 4 in others. However, that is "good enough" for the purpose for which these color spaces were created: quality standards for color ("can I buy more of the same color and it will look the same?"). If your measured and computed ΔE is below one, the difference will not be perceptible by most humans regardless. And last I checked, new and improved "perceptually uniform" color spaces are proposed every couple of years.
Sniffnoy 2y ago
While this phrases things in terms of Riemannian geometry, it seems to me like this is really about more fundamental metric properties, but I find its terminology a bit unclear. Would it be correct to summarize this as saying is that color space is not a length space / geodesic space? (See e.g. https://en.wikipedia.org/wiki/Intrinsic_metric https://en.wikipedia.org/wiki/Intrinsic_metric ) (Yes "length space" (or "path space") and "geodesic space" are not exactly the same thing, but "length space but not geodesic space" doesn't exactly seem like a very likely possibility.)
- woopsn 2y ago
  That's an interesting related thought that is not quite the same as the paper. They claim that what we perceive as "color similarity" is not a metric at all. That would be required in order to define arclength and geodesy. I agree with others that it is not surprising and is a technicality. If you put that aside then the situation is much closer to your intuition.
  - Sniffnoy 2y ago
    Are you sure it's claiming that? Looking quickly I didn't see anything to indicate that, I didn't see anything claiming triangle inequality violations. (But this is why I wish it had been phrased explicitly in these terms, instead of talking about whether it's specifically Riemannian!)
    - mannykannot 2y ago
      At one point, the article says "importantly, [the principle of diminishing returns] holds even along geodesics, making it distinct from and stronger than the triangle inequality." Later, they say "it is not trivial to verify whether any given path through color space is a geodesic. We chose the neutral axis because it is the one path on which all available data agree that it is indeed a geodesic", and go on to argue (if I am following it correctly) that it is unlikely that their principle of diminishing returns is just an artifact from this choice. I do not know if the authors are claiming that what we perceive as color similarity is not a metric at all, but personally, I would not be surprised if it were not. See my other post for how my subjective perception of color differences seems to me.
jimsimmons 2y ago
Is it non Riemann or non Eucliden
- OvbiousError 2y ago
  Both
- robwwilliams 2y ago
  Neither! Or perhaps Darwinian. It is an idiosyncratic space all in our brains. It is our own learned synaptic weights and embeddings that allows each of us to effectively/efficiently use some of the data generated by our photoreceptors. Sure, color space can be reformulated many other ways for applications but all just operational kludges.
- yccs27 2y ago
  It's non Riemannian and therefore also non Euclidean. In Riemannian space, the distance between points is the length of the shortest path between them. Euclidean space is a subtype of Riemannian space, where angles in every triangle sum to 180°.
ucarion 2y ago
> Consequently, we need to adapt how we model color differences, as the current standard, ΔE, recognized by the International Commission for Weights and Measures, does not account for diminishing returns in color difference perception. This isn't to reduce the notability of the article, but isn't BIPM's interest in any color-perception stuff basically limited to making lightbulbs roughly equally bright? (i.e. https://en.wikipedia.org/wiki/Luminous_efficiency_function https://en.wikipedia.org/wiki/Luminous_efficiency_function) I guess I don't see how even nonlinearity in perceptual space matters for that purpose.
contravariant 2y ago
Changing the triangle inequality to a strict one and claiming that the result is non-Riemannian is a bit dubious. First of all the difference between a strict and a non-strict inequality is not that big, stating two things are never exactly equal is nigh impossible to measure, especially for something subjective. Secondly that could happen simply by reporting distances in some non-linear fashion, and who cares if the unit of distances is non-linear? If there is a metric that correctly identifies which colour is closer then we're in business, nobody ever said this had to correspond linearly to whatever measure of perceptual distance someone came up with.
- yccs27 2y ago
  Yes, it's important to note the measure of distance used. Quoting from the paper: Humans are not good at judging questions of the type “How big is the difference?” that form the basis for many large difference experiments. Instead, the authors use data from decisions of "which color step is larger". They find that as the steps get larger, you need a higher difference in color steps to reach the same certainty. This is not that surprising, since people generally get worse at noticing small differences as the absolute quantities get larger. Unfortunately, the model of perception uncertainty used in the paper (Thurstone’s theory of Gaussian perceptual process) does not take this effect into account.
  - mannykannot 2y ago
    Thanks, that clears up one thing I was puzzling about: if I were asked to give a quantitative value to how different two colors seem to be, my first problem would be to figure out what a number - any number - would mean in this context. While 'barely perceivable difference' seems like a reasonable unit of measure, I don't think I could judge multiples of that unit with any degree of accuracy or consistency, or, indeed, in any meaningful way. On thinking about this issue, it seems to me that I think of large differences categorically (red vs. green, for example) while thinking of small differences as being on a continuum, even when the colors in question lie in the borderlands of the above categories.
robwwilliams 2y ago
At some level the fundamental assumptions of a “color” space as a real representation of the human perceptual space is almost comical. The color yellow has been an issue for many decades and diverse combinations of wavelengths evoke identical conscious yellow percepts. The fundamental “not-geometry” of color space is the output of a very complex deep neural net. Thinking about color space as “real” world is almost nonsensical ;-). Trying to map “real” world into brain now has a deservedly bad wrap. Modern philosophy has largely given up on Descartes, sharp subject-object dichotomies, and the brain and mind as a warped mirror of nature. If puzzling at first then check out Humberto Maturana and F. Valera (both experts in color vision electrophysiologist and philosophers), Martin Heidegger, Richard Rorty, and Daniel Dennett have powerful arguments against representationalism of the type implicit in most neuroscience. Valera, Rorty, Dennett are all readable. Heidegger and Matura are more cryptic unless you are a philosopher. https://en.wikipedia.org/wiki/Francisco_Varela https://en.wikipedia.org/wiki/Francisco_Varela Valera’s “The Embodied Mind: Cognitive Science and Human Experience”, co-authored with Evan Thompson and Eleanor Rosch, is a good intro on this and related concepts. But for Hacker News readers by far the best intro to this topic are chapters 4 and 5 of is the CS classic by Terry Winograd and Fernando Flores that is often highlighted on HN. It is a key text in understanding both what goes on in our brains and what is and may go on in machine brains. 1986. Understanding Computers and Cognition: A New Foundation for Design (with Fernando Flores) Ablex Publ Corp
- BobbyTables2 2y ago
  I’ve always found it deeply disturbing that we ascribe special significance to red/green/blue (that’s only what the eye is best at detecting) as “primary” colors. Why do we teach this to kids? It’s ridiculous that linearly adding “red” and “blue” would make “purple” — all different wavelengths — without a nonlinear effect occurring. CA prism separates these two back out and shows us purple didn’t exist in such an experiment. But we do see “purple” in a rainbow !?!? We see beautiful pictures on LCD screens but we only sense only 3 colors from it? An alien that could see the whole spectrum would think we were completely crazy!
  - SAI_Peregrinus 2y ago
turtledragonfly 2y ago
For those interested in such topics, I'm a fan of the "Oklab"[1] color space. It's not perfect, but well documented and discussed, has freely-available implementations, efficient shader code, etc. and overall works pretty well. [1] https://bottosson.github.io/posts/oklab/ https://bottosson.github.io/posts/oklab/
woolion 2y ago
This reminds of Poincaré's remark that all geometries are wrong, they are just useful abstractions of space. The different color spaces all have some features they are good at with their own algebra. So, RGB is close to how screen works, but is very un-intuitive as its algebra is additive rather than subtractive (like the one we have with paints). Then HSV is a middle ground to be more practical for humans, as it separates the color into its 3 most significant dimensions, that are value, hue, and saturation. It does so while being a fast linear transform, but it is not a really good mapping with regards to human perception. For example a given fixed value will appear as very different when given two different hues. Still, it's a nice middle ground that made it a good choice for painting and image manipulation software. There are other spaces that are "more" perceptual, in that changing the hue of a color should not affect how its value is perceived. This can be useful in design work as a palette could be hue-shifted and always stay legible, which was not possible in the general case with HSV or RGB. Yet, these fancy color spaces are not used a lot because in most practical applications there isn't enough justifications to use them. In particular because professionals are used to workaround the space deficiencies, so there's a cost to re-learn some of the primitives. The only really practical application I have ever seen of these perceptual models is to simulate 'color blindness', which theoretically give the possibility to design charts and maps that are more accessible to people. I believe such a validation tool should be required for public communications. The article is interesting and well written, as it gives an history of how some of these abstractions were developed, and highlight some of its shortcomings.
fredgrott 2y ago
If you look at the CIE Lab def and the HCT definition that Google uses it makes the math more clear....as with HCT there is a perception adjustment mathematically calculated
deleted 2y ago
[deleted]

85 points by cpach 2y ago

bbor 2y ago

The paper in PDF: https://www.pnas.org/doi/pdf/10.1073/pnas.2119753119?download=true https://www.pnas.org/doi/pdf/10.1073/pnas.2119753119?downloa... there’s a lot of math symbols but IMO the usage is pretty straightforward for anyone familiar with linear algebra and proofs. Ramsay (30) suggests that the principle of diminishing returns or its opposite may have been spuriously identified by otherresearchers (daringly including his PhD advisor, Helm) because of experimental procedures (successive intervals, paired compar-isons) ill suited to the task. Indeed, open-ended and criterion-dependent tasks give less accurate measures of similarity because of individual factors and the difficulty of the task… Instead, we use a more reliable two-alternative forced choice (2AFC) task, where the participant simply answers the following question: "Which is more different?" Specifically, we use a triad arrangement of stimuli with the reference in the middle and one test on either side. Each of 320 triads covering the neutral axis was judged by at least 250 different participants in a crowdsourced study on Amazon Mechanical Turk (MTurk) (52).* The core of it seems strong, IMO. I would have liked a bit more cognitive science / philosophy to distinguish between an ideal color space and whatever imperfect model is created by individual human’s neural software, and skipping Goethe and Schopenhauer in the intro was downright criminal, but that’s more on me than them. Color has long been a convenient “entry point” into the study of human unconscious and subconscious data transformations, most notably through Wittgenstein’s musings about red squares. I look forward to this technique being extended to other spaces, even ones as abstract as “moral space” or “valence space”!

robwwilliams 2y ago
Great comment. You made me laugh out loud on the other spaces we HAVE TO NAIL DOWN. You would get a kick out of Winograd and Flores 1986.

piannucci 2y ago

I love the drama of how the abstract is written, but TBH I don't think this is a surprise. I believe it's well-known among color theorists that large perceptual distances are inconsistent with sums of small differences. So maybe the most generous thing to say here is, good on them for bringing awareness of this subtlety to a broader audience.

robwwilliams 2y ago
Yep: color space is a pragmatic kludge not a “real thing” divorced from the human neural network that is its fundamental basis.
mpweiher 2y ago
Not only that, it is also well known that the smallest perceptible color difference (ΔE=1) is not actually consistent, even in the "perceptually uniform" color spaces. So ΔE is actually 1 in some parts of the space, but up to 4 in others. However, that is "good enough" for the purpose for which these color spaces were created: quality standards for color ("can I buy more of the same color and it will look the same?"). If your measured and computed ΔE is below one, the difference will not be perceptible by most humans regardless. And last I checked, new and improved "perceptually uniform" color spaces are proposed every couple of years.

Sniffnoy 2y ago

While this phrases things in terms of Riemannian geometry, it seems to me like this is really about more fundamental metric properties, but I find its terminology a bit unclear. Would it be correct to summarize this as saying is that color space is not a length space / geodesic space? (See e.g. https://en.wikipedia.org/wiki/Intrinsic_metric https://en.wikipedia.org/wiki/Intrinsic_metric ) (Yes "length space" (or "path space") and "geodesic space" are not exactly the same thing, but "length space but not geodesic space" doesn't exactly seem like a very likely possibility.)

woopsn 2y ago
That's an interesting related thought that is not quite the same as the paper. They claim that what we perceive as "color similarity" is not a metric at all. That would be required in order to define arclength and geodesy. I agree with others that it is not surprising and is a technicality. If you put that aside then the situation is much closer to your intuition.
- Sniffnoy 2y ago
  Are you sure it's claiming that? Looking quickly I didn't see anything to indicate that, I didn't see anything claiming triangle inequality violations. (But this is why I wish it had been phrased explicitly in these terms, instead of talking about whether it's specifically Riemannian!)
  - mannykannot 2y ago
    At one point, the article says "importantly, [the principle of diminishing returns] holds even along geodesics, making it distinct from and stronger than the triangle inequality." Later, they say "it is not trivial to verify whether any given path through color space is a geodesic. We chose the neutral axis because it is the one path on which all available data agree that it is indeed a geodesic", and go on to argue (if I am following it correctly) that it is unlikely that their principle of diminishing returns is just an artifact from this choice. I do not know if the authors are claiming that what we perceive as color similarity is not a metric at all, but personally, I would not be surprised if it were not. See my other post for how my subjective perception of color differences seems to me.

jimsimmons 2y ago

Is it non Riemann or non Eucliden

OvbiousError 2y ago
Both
robwwilliams 2y ago
Neither! Or perhaps Darwinian. It is an idiosyncratic space all in our brains. It is our own learned synaptic weights and embeddings that allows each of us to effectively/efficiently use some of the data generated by our photoreceptors. Sure, color space can be reformulated many other ways for applications but all just operational kludges.
yccs27 2y ago
It's non Riemannian and therefore also non Euclidean. In Riemannian space, the distance between points is the length of the shortest path between them. Euclidean space is a subtype of Riemannian space, where angles in every triangle sum to 180°.

ucarion 2y ago

> Consequently, we need to adapt how we model color differences, as the current standard, ΔE, recognized by the International Commission for Weights and Measures, does not account for diminishing returns in color difference perception. This isn't to reduce the notability of the article, but isn't BIPM's interest in any color-perception stuff basically limited to making lightbulbs roughly equally bright? (i.e. https://en.wikipedia.org/wiki/Luminous_efficiency_function https://en.wikipedia.org/wiki/Luminous_efficiency_function) I guess I don't see how even nonlinearity in perceptual space matters for that purpose.

contravariant 2y ago

Changing the triangle inequality to a strict one and claiming that the result is non-Riemannian is a bit dubious. First of all the difference between a strict and a non-strict inequality is not that big, stating two things are never exactly equal is nigh impossible to measure, especially for something subjective. Secondly that could happen simply by reporting distances in some non-linear fashion, and who cares if the unit of distances is non-linear? If there is a metric that correctly identifies which colour is closer then we're in business, nobody ever said this had to correspond linearly to whatever measure of perceptual distance someone came up with.

yccs27 2y ago
Yes, it's important to note the measure of distance used. Quoting from the paper: Humans are not good at judging questions of the type “How big is the difference?” that form the basis for many large difference experiments. Instead, the authors use data from decisions of "which color step is larger". They find that as the steps get larger, you need a higher difference in color steps to reach the same certainty. This is not that surprising, since people generally get worse at noticing small differences as the absolute quantities get larger. Unfortunately, the model of perception uncertainty used in the paper (Thurstone’s theory of Gaussian perceptual process) does not take this effect into account.
- mannykannot 2y ago
  Thanks, that clears up one thing I was puzzling about: if I were asked to give a quantitative value to how different two colors seem to be, my first problem would be to figure out what a number - any number - would mean in this context. While 'barely perceivable difference' seems like a reasonable unit of measure, I don't think I could judge multiples of that unit with any degree of accuracy or consistency, or, indeed, in any meaningful way. On thinking about this issue, it seems to me that I think of large differences categorically (red vs. green, for example) while thinking of small differences as being on a continuum, even when the colors in question lie in the borderlands of the above categories.

robwwilliams 2y ago

At some level the fundamental assumptions of a “color” space as a real representation of the human perceptual space is almost comical. The color yellow has been an issue for many decades and diverse combinations of wavelengths evoke identical conscious yellow percepts. The fundamental “not-geometry” of color space is the output of a very complex deep neural net. Thinking about color space as “real” world is almost nonsensical ;-). Trying to map “real” world into brain now has a deservedly bad wrap. Modern philosophy has largely given up on Descartes, sharp subject-object dichotomies, and the brain and mind as a warped mirror of nature. If puzzling at first then check out Humberto Maturana and F. Valera (both experts in color vision electrophysiologist and philosophers), Martin Heidegger, Richard Rorty, and Daniel Dennett have powerful arguments against representationalism of the type implicit in most neuroscience. Valera, Rorty, Dennett are all readable. Heidegger and Matura are more cryptic unless you are a philosopher. https://en.wikipedia.org/wiki/Francisco_Varela https://en.wikipedia.org/wiki/Francisco_Varela Valera’s “The Embodied Mind: Cognitive Science and Human Experience”, co-authored with Evan Thompson and Eleanor Rosch, is a good intro on this and related concepts. But for Hacker News readers by far the best intro to this topic are chapters 4 and 5 of is the CS classic by Terry Winograd and Fernando Flores that is often highlighted on HN. It is a key text in understanding both what goes on in our brains and what is and may go on in machine brains. 1986. Understanding Computers and Cognition: A New Foundation for Design (with Fernando Flores) Ablex Publ Corp

BobbyTables2 2y ago
I’ve always found it deeply disturbing that we ascribe special significance to red/green/blue (that’s only what the eye is best at detecting) as “primary” colors. Why do we teach this to kids? It’s ridiculous that linearly adding “red” and “blue” would make “purple” — all different wavelengths — without a nonlinear effect occurring. CA prism separates these two back out and shows us purple didn’t exist in such an experiment. But we do see “purple” in a rainbow !?!? We see beautiful pictures on LCD screens but we only sense only 3 colors from it? An alien that could see the whole spectrum would think we were completely crazy!
- SAI_Peregrinus 2y ago

turtledragonfly 2y ago

For those interested in such topics, I'm a fan of the "Oklab"[1] color space. It's not perfect, but well documented and discussed, has freely-available implementations, efficient shader code, etc. and overall works pretty well. [1] https://bottosson.github.io/posts/oklab/ https://bottosson.github.io/posts/oklab/

woolion 2y ago

This reminds of Poincaré's remark that all geometries are wrong, they are just useful abstractions of space. The different color spaces all have some features they are good at with their own algebra. So, RGB is close to how screen works, but is very un-intuitive as its algebra is additive rather than subtractive (like the one we have with paints). Then HSV is a middle ground to be more practical for humans, as it separates the color into its 3 most significant dimensions, that are value, hue, and saturation. It does so while being a fast linear transform, but it is not a really good mapping with regards to human perception. For example a given fixed value will appear as very different when given two different hues. Still, it's a nice middle ground that made it a good choice for painting and image manipulation software. There are other spaces that are "more" perceptual, in that changing the hue of a color should not affect how its value is perceived. This can be useful in design work as a palette could be hue-shifted and always stay legible, which was not possible in the general case with HSV or RGB. Yet, these fancy color spaces are not used a lot because in most practical applications there isn't enough justifications to use them. In particular because professionals are used to workaround the space deficiencies, so there's a cost to re-learn some of the primitives. The only really practical application I have ever seen of these perceptual models is to simulate 'color blindness', which theoretically give the possibility to design charts and maps that are more accessible to people. I believe such a validation tool should be required for public communications. The article is interesting and well written, as it gives an history of how some of these abstractions were developed, and highlight some of its shortcomings.

fredgrott 2y ago

If you look at the CIE Lab def and the HCT definition that Google uses it makes the math more clear....as with HCT there is a perception adjustment mathematically calculated

deleted 2y ago

[deleted]