4 ms·

What does it mean to filter in gamma space vs linear space?

by hoten 4y ago

dTal 4y ago
"Filtering" is a graphics term of art that roughly means "interpolation". Imagine you want to represent a star with a single max brightness pixel. Suppose you want to smoothly pan that pixel to its neighbor. If you simply switch the pixel off and switch the neighbor on, it will look jumpy. So what you want to do is gradually dim the first pixel, while gradually brightening the second, so that the intensity smoothly moves over instead of jumping. The catch: at all times, the sum of the light intensity from both pixels must be constant. You want the star to move, not flicker. Seems easy, right? Subtract 1 from the first pixel and add 1 to the second, keeping the sum the same? Wrong! The relationship between the actual light emitted by a pixel and the number we use to represent it in software is not linear. It generally follows a power law, usually with an exponent of about 2.2, since this is a more efficient representation given that human perception also follows a power law. If 255 were actually twice as many photons as 127, you would find that they didn't actually look all that different - the range 0-255 would skew towards most values looking pretty bright. Instead the values "encode" the real brightness, and if you do math on these encoded values you are operating in "gamma space". So, in order to correctly do our 1-pixel panning, we need to "decode" our pixel value to linear brightness by taking the logarithm (base 2.2), add 1 / subtract 1 there, and then raise it back up. This is what Carmack means by "filtering in linear space". Obviously, many programmers are not aware of the requirement to do this, and just treat the pixel values as if they were real brightness values. Result: flicker.
- hoten 4y ago
  Thanks for breaking it down! It seems like this was a necessity to allow for a larger range of representable colours in a single byte. Is this system largely a carry over from more constrained hardware days, or is it still appropriate for modern hardware? Like, we could represent colors with a few more bytes and toss out the gamma encoding to get back to the same level of control. Or am I overlooking more fundamental reasons why this is a good abstraction?
  - midnightclubbed 4y ago
    Still appropriate for modern hardware. Despite hardware getting faster, display resolutions and refresh rates have also increased to eat the memory bandwidth gains. Especially so at VR resolutions. Modern rendering engines will use a mixture of 8bit, 10bit, and 16bit per color channel buffers - the final 'lit' color buffer is where you will generally write fp16 (8bytes per pixel) linear HDR color values, if you can spare the performance hit, as this is where you need the highest range.
  - dTal 4y ago
    It's a good abstraction, because most of the time you care about perceived brightness. With gamma encoding, 255 looks twice as bright as 127. You are going to need to deal with that perceptual power law at some point no matter what representation you use, so you may as well pick the efficient one. It's probably not a good tradeoff to make everyone deal with a color scale that doesn't look right, and is inefficient, just to make intensity operations linear so that unwary programmers don't trip.
DiabloD3 4y ago
Light displayed on computer monitors are not linear, instead follow a power law transformation, also known (less precisely) as a non-linear transfer function. Pixels should only be stored in gamma (non-linear) space for final display; all math on them should be performed in linear space instead. sRGB has a badly defined gamma space of an encoding of { [0-15] (linear), [16-255] (power of 2.4 over [0-239] with an offset of 16) } with a decoding of 2.2 (but explicitly taking into account "dark room" ambient lighting, not the actual display response). This is often quoted as "power of 2.2 encoding and decoding", and this has never been correct. For modern viewing conditions with a 100 nits monitor in an ambient condition of "bright room", the proper interpretation of sRGB is to read as "actual monitor response of 2.4", which leads to.... bt1886 defines this as 2.4 encoding combined with a 2.2 and 2.4 decoding where the crossover point is calculated in a way that takes into account actual black levels observed on the device to maintain perceptual quality while avoiding black and white crush (devices with actually zero black (ex: CRT, Plasma, OLED) would just be pure 2.4 decoding). This allows all commercial content (NTSC, PAL, and BT709/BT2020) to display favorably no matter how badly it was mastered originally and no matter how bad the viewing conditions actually are. BBC, taking into account "ye average display over the past 30 years", encodes as 2.35 to take into account NTSC, PAL, 2.2, 2.4, and actually-sRGB displays. This is arguably the most correct approach: bt1886 and sRGB/bt709 on displays that are too dark will be too dark but perceptually correct on displays that are too bright; "encode as 2.2" (which is still wrong) will be very dark on every display; 2.35 will look correct on "wrong, but too dark", "wrong, but too light", "actually sRGB", and "actually bt1886" monitors. There also is a third (fourth?) standard of 2.6 that is purely found in extremely low display light (40 nits or less), zero ambient completely blacked out conditions, used for movie theatres, due to perceptual changes in human eyesight; sRGB, bt1886, and 2.35 would all be too dark and/or too undersaturated here otherwise.

What does it mean to filter in gamma space vs linear space?

by hoten 4y ago

What does it mean to filter in gamma space vs linear space?

dTal 4y ago

"Filtering" is a graphics term of art that roughly means "interpolation". Imagine you want to represent a star with a single max brightness pixel. Suppose you want to smoothly pan that pixel to its neighbor. If you simply switch the pixel off and switch the neighbor on, it will look jumpy. So what you want to do is gradually dim the first pixel, while gradually brightening the second, so that the intensity smoothly moves over instead of jumping. The catch: at all times, the sum of the light intensity from both pixels must be constant. You want the star to move, not flicker. Seems easy, right? Subtract 1 from the first pixel and add 1 to the second, keeping the sum the same? Wrong! The relationship between the actual light emitted by a pixel and the number we use to represent it in software is not linear. It generally follows a power law, usually with an exponent of about 2.2, since this is a more efficient representation given that human perception also follows a power law. If 255 were actually twice as many photons as 127, you would find that they didn't actually look all that different - the range 0-255 would skew towards most values looking pretty bright. Instead the values "encode" the real brightness, and if you do math on these encoded values you are operating in "gamma space". So, in order to correctly do our 1-pixel panning, we need to "decode" our pixel value to linear brightness by taking the logarithm (base 2.2), add 1 / subtract 1 there, and then raise it back up. This is what Carmack means by "filtering in linear space". Obviously, many programmers are not aware of the requirement to do this, and just treat the pixel values as if they were real brightness values. Result: flicker.

hoten 4y ago
Thanks for breaking it down! It seems like this was a necessity to allow for a larger range of representable colours in a single byte. Is this system largely a carry over from more constrained hardware days, or is it still appropriate for modern hardware? Like, we could represent colors with a few more bytes and toss out the gamma encoding to get back to the same level of control. Or am I overlooking more fundamental reasons why this is a good abstraction?
- midnightclubbed 4y ago
  Still appropriate for modern hardware. Despite hardware getting faster, display resolutions and refresh rates have also increased to eat the memory bandwidth gains. Especially so at VR resolutions. Modern rendering engines will use a mixture of 8bit, 10bit, and 16bit per color channel buffers - the final 'lit' color buffer is where you will generally write fp16 (8bytes per pixel) linear HDR color values, if you can spare the performance hit, as this is where you need the highest range.
- dTal 4y ago
  It's a good abstraction, because most of the time you care about perceived brightness. With gamma encoding, 255 looks twice as bright as 127. You are going to need to deal with that perceptual power law at some point no matter what representation you use, so you may as well pick the efficient one. It's probably not a good tradeoff to make everyone deal with a color scale that doesn't look right, and is inefficient, just to make intensity operations linear so that unwary programmers don't trip.

DiabloD3 4y ago

Light displayed on computer monitors are not linear, instead follow a power law transformation, also known (less precisely) as a non-linear transfer function. Pixels should only be stored in gamma (non-linear) space for final display; all math on them should be performed in linear space instead. sRGB has a badly defined gamma space of an encoding of { [0-15] (linear), [16-255] (power of 2.4 over [0-239] with an offset of 16) } with a decoding of 2.2 (but explicitly taking into account "dark room" ambient lighting, not the actual display response). This is often quoted as "power of 2.2 encoding and decoding", and this has never been correct. For modern viewing conditions with a 100 nits monitor in an ambient condition of "bright room", the proper interpretation of sRGB is to read as "actual monitor response of 2.4", which leads to.... bt1886 defines this as 2.4 encoding combined with a 2.2 and 2.4 decoding where the crossover point is calculated in a way that takes into account actual black levels observed on the device to maintain perceptual quality while avoiding black and white crush (devices with actually zero black (ex: CRT, Plasma, OLED) would just be pure 2.4 decoding). This allows all commercial content (NTSC, PAL, and BT709/BT2020) to display favorably no matter how badly it was mastered originally and no matter how bad the viewing conditions actually are. BBC, taking into account "ye average display over the past 30 years", encodes as 2.35 to take into account NTSC, PAL, 2.2, 2.4, and actually-sRGB displays. This is arguably the most correct approach: bt1886 and sRGB/bt709 on displays that are too dark will be too dark but perceptually correct on displays that are too bright; "encode as 2.2" (which is still wrong) will be very dark on every display; 2.35 will look correct on "wrong, but too dark", "wrong, but too light", "actually sRGB", and "actually bt1886" monitors. There also is a third (fourth?) standard of 2.6 that is purely found in extremely low display light (40 nits or less), zero ambient completely blacked out conditions, used for movie theatres, due to perceptual changes in human eyesight; sRGB, bt1886, and 2.35 would all be too dark and/or too undersaturated here otherwise.