10 ms·
Oklab: A perceptual color space for image processing
- kelsolaar 6y agoThis is some great work. The notable point here is that the work is done by a Computer Graphics Programmer using Open Source Free software (colorio and colour-science both for Python ) and not a Researcher using Closed Source Commercial software like Matlab.
- s_gourichon 6y agoFWIW, researchers I know use C++, Python, Linux and many open-source tools.
- kelsolaar 6y agoFor colour science though?
- deleted 6y ago[deleted]
- ska 6y agoSure. I think you are seeing a path dependence thing - more so in the past, but still there are engineering departments that are very matlab oriented, and others that are not. So depending who you work with, you'll see a ton of it or little. This is a bit like the python/R dichotomy in data science.
- CyberDildonics 6y agoI'm not sure why all these are capitalized as some sort of formal titles, but matlab is very rarely used in computer graphics. Computer graphics is mostly fairly simple math that needs to scale well and not have edge cases. Some papers have had their algorithms in matlab but it is the exception in research and basically never used in any sort of production sense, because you would just have to port it to C++ to use it.
- deleted 6y ago[deleted]
- ska 6y agocomputer graphics and color science communities are fairly distinct, although obviously there is some overlap.
- jiggawatts 6y agoThis is a tour de force of colour theory, and should be mandatory reading for anyone serious about computer colour! Björn Ottosson not only did some mighty fine work, producing a simple equation that produces smooth colour gradients, but he "showed his work" too. Instead of just journal references, he littered this page with incredibly useful hyperlinks to difficult to find things such as the raw data for the Munsell colour chart and the Luo-Rigg data! This should also be mandatory reading for people not so serious about computer colour, because heads up: If you do any kind of arithmetic on RGB bytes, you've screwed up much more than you think you have! It's one of the most common examples of the Dunning–Kruger effect. Need a nice gradient of colours to represent something graphically, such as low-medium-high values? Go to this page first. Need to blur a background picture? Go to this page first. Resizing images? Read this first. Making colour schemes for a web page theme? This page first. Adding a "colour picker" control? Definitely read this first.
- bjornornorn 6y agoAuthor here. Wow, thanks a lot! If anyone has any questions feel free to post here and I can try to answer. I have another post that goes into more detail onhow software often gets color wrong: https://bottosson.github.io/posts/colorwrong/ https://bottosson.github.io/posts/colorwrong/
- olivierduval 6y agoHello Really nice work!!! How does it compare with HSLuv (https://www.hsluv.org/ https://www.hsluv.org/)? It seems that both scheme try to manage the perceptual color problem...
- bjornornorn 6y agoThat is based on CIELUV, which is one of the spaces tested against in the article. CIELUV is useful because its chromaticity space is a projective transform of XYZ. That also means it doesn't predict perception of hue as well as more complex models.
- 6y ago
- doomlaser 6y ago> Converting from linear sRGB to Oklab With public domain C++ code! This is incredibly useful to me as a Unity developer. Can almost be plugged right in to C# & Unity Colors. Could be used for interesting real-time color effects. One of my favorite simple things to do in HSV is to animate the hue. This could be used to do something similar, but across that more attractive Oklab gradient. Great post.
- reitzensteinm 6y agoIt looks really cheap, you could easily hue shift every pixel in the game with this. One trick I've used in the past is using the alpha channel to mask multiple hue shifts in a shader to give a lot of variations. Eg 0 = shift A, 0.5 = don't shift, 1 = shift B. That way you can make eg leather armor with metal attachments, and tan or fade the leather and swap metal types independently.
- jacobolus 6y ago> It looks really cheap It is pretty much the same cost as CIELAB. Just uses 2 full 3x3 matrices where CIELAB can be thought of as using matrices with a bunch of zeros in them.
- coldcode 6y agoAs an artist who uses code a lot, this is going right into my toolset. Manipulating colors is a huge pain.
- mark-r 6y agoYou can change hues in RGB without having to use HSV. See https://stackoverflow.com/a/8510751/5987 https://stackoverflow.com/a/8510751/5987
- dTal 6y agoI don't understand why they don't compare it with YCbCr, which is a very similar colorspace that is much more widely used than some of the obscure ones they compare it with - notably, in JPEG.
- jacobolus 6y agoYCrCb is roughly based on the ones mentioned (which are not at all obscure from the perspective of color science), but is supposed to be as computationally cheap as possible, and so doesn't bother doing even basic steps like undoing gamma encoding. As a result, YCrCb yields crappy results for any purpose other than image encoding. The further you get from neutral gray, the worse YCrCb gets at cleanly separating lightness from hue/chroma.
- bjornornorn 6y agoYeah, that’s pretty much why I didn’t include it. I haven’t really seen YCrCb used except anywhere for in compression, so didn’t think of including it (an neither do most papers related to perceptual color spaces).
- thewebcount 6y agoIt was used quite a bit in video editing in the early 2000s because the video frames often came from disk or a card in that format, and you could do some fast processing tricks with it. Those days have largely passed, though.
- batty_alex 6y agoI was thinking the same thing. My guess is the author’s background is in games and that’s where this research is focused. It did give an impression that, to someone who works with color spaces, the author is operating in a different domain
- HelloNurse 6y agoYCbCr was designed for the needs of analog color TV transmission and recording, not for image processing and editing.
- mncharity 6y agoApropos D65, just a reminder that 65K is perceptually blue tinted. Just as D50 is red tinted. So if users will view your rendered image surrounded by an rgb-white screen, if/when the resulting small red/yellow/green/white shift is problematic, you might consider rendering using a non-standard 58K whitepoint. Just saw someone burned by this the other day, with a "supposed to be white with a localized tint" object (the Sun), which D65 then confused.
- spider-mario 6y ago> Apropos D65, just a reminder that 65K is perceptually blue tinted. Just as D50 is red tinted. That really depends on the viewer’s adaptation state. > So if users will view your rendered image surrounded by an rgb-white screen, if/when the resulting small red/yellow/green/white shift is problemaic, you might consider rendering using a non-standard 58K whitepoint. Just saw someone burned by this the other day, with a "supposed to be white with a localized tint" object (the Sun), which D65 then confused. Isn’t that what the ICC’s relative rendering intent is supposed to take care of, if I understand what you mean?
- mncharity 6y ago> depends on the viewer’s adaptation state IIRC, both D65 and D50 are sufficiently extreme as to prevent full chromatic adaptation. Viewers will be aware a scene is lit "cold" or "warm"ly. > ICC’s relative rendering intent is supposed to take care of Yes. Browser support used to be poor, but I've not been following it. Firefox seems to still require user config?[1] A perhaps outdated WP article[2] suggests chrome support of V4 is OS dependent? But yes, someday this will just work. [1] https://cameratico.com/color-management/firefox/ https://cameratico.com/color-management/firefox/ [2] hhttps://en.wikipedia.org/wiki/Color_management#Application_level https://en.wikipedia.org/wiki/Color_management#Application_l...
- raphlinus 6y agoChromatic adaptation is definitely one of the trickier questions in color management. Here's my take on it. For the most part, when looking at a monitor, chromatic adaptation is pretty good. In other words, RGB #FFFFFF looks white, not light blue, and #808080 looks gray, not bluish-gray. It does start falling apart when you mix different lighting, for example holding a white piece of paper to the screen under normal room light. I think the main reason D65 is chosen for the sRGB white point is that it is fairly accurate, in other words it's pretty close to the actual white point of the monitor I'm looking at now (Dell P2415Q). I have yet to find a source on chromatic adaptation that I consider a really excellent tutorial. Certainly the Wikipedia page is cursory and abstract, which is a shame.
- fxtentacle 6y agoThis is an interesting one, because they optimize for an untypical goal: "If the scale/exposure of colors are changed, the perceptual coordinates should just be scaled by a factor" For most computer vision applications, that is the opposite of what you want. When you analyze a video stream, it is quite common for frames to have different brightness due to things like aliasing between the flicker of LEDs in the room and the camera shutter. That's why CV needs a color space where colors remain close to each other, no matter the frame brightness.
- mkl 6y agoI think it means the hue stays the same, since it's angle in polar coordinates, which is scale invariant. Saturation (chroma) changes though.
- CyberDildonics 6y agoBrightness is by far the biggest factor in perceptual differences and perceptual color spaces are approximations. Also LEDs don't typically flicker unless a dimmer is on the circuit and their PCB isn't made for that. You might be thinking of florescent lights that are pulsed with a ballast.
- nitrogen 6y agoQuite a large number of LED fixtures flicker at line frequency, sadly, even in "luxury" buildings.
- CyberDildonics 6y agoYou are probably seeing early LED fixtures with terrible electronics put in luxury buildings as the hip new thing years ago. All it takes is a capacitor above 170 volts after the rectifier to smooth out the AC.
- knolan 6y agoThe example[1] given at the start of the article is almost like the end of the first order of a Micheal Levi chart[2]. This can be used to describe thin film interference such as a layer of oil on water or a bubble’s colour. Certainly gives it a natural look. [1]https://bottosson.github.io/img/oklab/hue_oklab.png https://bottosson.github.io/img/oklab/hue_oklab.png [2]https://www.doitpoms.ac.uk/tlplib/liquid_crystals/images/michel-levy-chart.jpg https://www.doitpoms.ac.uk/tlplib/liquid_crystals/images/mic...
- JyrkiAlakuijala 6y agoThis looks very similar to what I came up for pik, consequently used as main default colorspace of JPEG XL, i.e., XYB. Also butteraugli's XYB has similar ideas, but is slightly more expensive to calculate due to the biased logarithm in the compression function (instead of cubic root), but possibly scales better for HDR (say above 200 nits). JPEG XL's XYB includes more red and less green in S-receptor modeling (for the blue-yellow axis). If I look at literature of LMS receptor spectra, it makes me wonder why there is so much green in Oklab. When I optimized similar for XYB, the optimization would favor adding slightly more red for the S than green. S component in JPEG XL XYB before non-linearity: 0.24 * R + 0.20 * G + 0.56 * B S component in Oklab before non-linearity: 0.05 * R + 0.26 * G + 0.63 * B Given the similarity of Oklab and XYB I suspect (but i'm not completely sure) that JPEG XL's format is powerful enough to model Oklab, too. Very very likely it can perfectly model the M1 matrix and the cubic root. I believe for M2 there may be some need for approximations. There JPEG XL can have local variations for M2 from chroma-from-luma fields, but likely luma needs to be slightly different from Oklab.
- JyrkiAlakuijala 6y agoAnother rather substantial difference is in M2 matrix of Oklab. In my experiments I don't see S-participation in colors with high spatial frequency. Because in image compression a lot of information is in high spatial frequency, one gets favorable image compression when M2 matrix is without S contribution in Luma. We use just [b b 0, a -a 0, -0.5 * c, -0.5 * c, c] in the M2 phase in JPEG XL. The two 0s there don't bring S reception into Luma and redness-greenness observations. This difference can be because Oklab is based on XYZ which is based on 2 degree color samples. XYB is based on about 0.03 degree color samples. Perception seems to be different there -- to me it looks like S is not yet integrated into Luma experience at that resolution. In butteraugli color modeling is more complex: it is divided into high spatial frequency and low spatial frequency. S is brought only to the low spatial frequency color transforms. (Frequency separation there is by Laplacian pyramid.)
- JyrkiAlakuijala 6y agoOne more interesting and substantial difference between Oklab and XYB is that XYB includes biases before the nonlinearity, i.e., one can consider the M1 matrix as a homogeneous matrix. These biases make the receptive model (more) linear close to zero, and the non-linearity ramps up when one goes further into high intensity. The idea there is to model the spontaneous opsin isomerization in the receptors. I believe sRGB approximated this by gluing a small linear ramp into the nonlinearity with if-then-logic.
- michaelrpeskin 6y agoGreat article! I wish I would have had this a couple of years ago when I was working on improving a very old lossy image compression algorithm. I don't want to into great detail here, but there's a highly compressed image format that's designed so that it's super easy for low power (think 1990's embedded processors) to read and render. Basically the decompression is a bunch of LUTs - not just palette lookups, but also pattern lookups. The hard part is creating the LUTs which can be done on fast computers. As the images we were trying to compress got more detail in them we had to redo the compression algorithm so that we could keep as much detail as possible. The key was to do the compression in a new color space and then undo the color space transform before we saved the LUTs. I ended up creating an empirical color space that was the best for each tile we were compressing (think PCA), but now that I look at this, it might even do better since the perception is so consistent.
- nitrogen 6y agoReminds me that back in high school my programming competition team won an image compression challenge by implementing RLE compression in a YUV-ish colorspace (the competition template provided colorspace transforms and we picked the one that worked best). Somehow that looked terrible but fooled the automated grading program. Another team had attempted JPEG, but they couldn't get their codec to work. They probably would have won otherwise.
- buildbot 6y agoReally cool! Do I understand the process for using this correctly? Take something in another color space, convert it to oklab, then perform whatever editing operations you want on the values, convert it back out to whatever color space for the file format you are using, and the results will be more accurate?
- bjornornorn 6y agoYes, exactly. The results will be more accurate as in closer to perception of light (for rendering for example you would want a linear color space)
- quag 6y agobjornornorn, thank you! I’m looking forward to trying it out! I’ve been using a variant of JzAzBz for the last year or so. Perceptual color spaces make image generation much easier. Can you help me understand how JzAzBz falls down?
- bjornornorn 6y agoThe largest difference is that JzAzBz has been designed to model lightness perception of very bright colors and very dark colors. Depending on your use case this could either be useful or a problem, since it means that you have to be careful with the scaling of the input data to JzAzBz. E.g, if you scale a color by some value before converting to JzAzBz, the predicted hue will be different than without scaling. Oklab will predict the same hue regardless of scale (but could also mean it does worse for extreme colors). So it is a trade off, with oklab being designed to be less complex.
- quag 6y agoThanks! That helps, and JzAzBz is complex when it comes to calculations.
- mnouquet 6y agoObligatory XKCD... https://xkcd.com/927/ https://xkcd.com/927/ [FWIW, thanks to prove the HN crowd has strictly no sense of humor what-so-ever...]
- mkl 6y agoIt's not that we have no sense of humour, it's that: a) Almost all of us have seen it before. b) A comment with just an opaque link doesn't really add anything to the conversation. c) That comic doesn't even apply in this case, as Oklab is not trying to be a universal standard covering all use cases. Its intended purposes are quite specific and different from the other colour spaces.
- refulgentis 6y agoIt's not nearly as clear as you purport, and thus, the link is funny and appropriate - there's nothing on that page to suggest its _not_ the best for all use cases - the sum total of its argument is its better in every case. However, the data isn't 100% there if you work in the spaces, i.e. I'd want to understand more how the CAM16 gradients were generated (CAM16 has two correlates for lightness and another three for 'color', saturation, colorfulness, and hue. CAM16UCS 'only' has colorfulness. How could the CAM16UCS gradient get 'saturated' more quickly?)
- mattdesl 6y agoGreat work, kudos to the author. Keen to try this in my own applications. I wonder how it fares with cubic spline gradients, I noticed some odd hues/tones depending on the colour space and Oklab might solve some of that. [1] [1] https://observablehq.com/@mattdesl/perceptually-smooth-multi-color-linear-gradients https://observablehq.com/@mattdesl/perceptually-smooth-multi...
- raphlinus 6y agoI'm very curious about this too. I'm going to seriously consider this color space, as well as cubic splines, as native gradient types in piet-gpu, because I think it's likely we can see higher quality at modest GPU shader cost.
- bjornornorn 6y agoNice! Definitely the the type of use case I had in mind.
- mattdesl 6y agoI added Oklab to the Observable notebook. With the current set of palettes, it produces results that are very similar to Lab, but I imagine with some specific color/hue transitions Oklab will outperform Lab. https://observablehq.com/@mattdesl/perceptually-smooth-multi-color-linear-gradients https://observablehq.com/@mattdesl/perceptually-smooth-multi...
- bjornornorn 6y agoYeah, should be similar a lot of the time. Deep blues are the colors that are the most different.
- mark-r 6y agoI've seen CIELab used to do color transformations, and it has a problem. This color space looks like it will suffer from the same problem. If you try to shift hues, keeping a constant luminance causes a problem. You can't transform a bright blue to a bright yellow, because the luminance of blue is considered much lower than yellow. The best you'll get is a very dark brown.
- jacobolus 6y agoWhat is the goal of your hue shift? If your goal is to otherwise preserve color relationships, then you should use a perceptually relevant color space. But maybe you have some different goal...?
- mark-r 6y agoSometimes the goal is just to change the color of an object while retaining the shadows, highlights, and textures. Suppose you had a picture of a blue car and wanted to know what it would look like in yellow.
- jacobolus 6y agoPersonally I use this tool I built as a Photoshop action over a decade ago, https://www.hcs.harvard.edu/~jrus/colortheory/jl.html https://www.hcs.harvard.edu/~jrus/colortheory/jl.html (see the animation at the bottom showing almost precisely what you are asking about) Initial discussion: https://www.ledet.com/margulis/2010HTM/ACT08-Jacobs_Ladder.htm https://www.ledet.com/margulis/2010HTM/ACT08-Jacobs_Ladder.h... An HSV hue shift (or whatever similar thing you are thinking of) yields horrible results in this use case.
- mark-r 6y agoThat example looks like it's working directly with RGB, which I expect to be correct. I've also seen it work well in HLS. But I worked on a tool where we were told to change from HLS to CIELab because it would be better, and it turned out to be much worse in practice.
- mark-r 6y agoI've noticed for a long time that in a color wheel, Cyan Magenta and Yellow are much brighter than Blue Red and Green. A couple of years ago it finally dawned on me why that was - the way our monitors are designed, Cyan Magenta and Yellow are producing twice as many photons! They each produce full intensity on two primary colors instead of just one. They don't look fully twice as bright because of the eye's non-linear response, but it's definitely a noticeable effect.