6 ms·
Premultiplied alpha vs. not-premultiplied alpha blending and compositing
- gus_massa 4y agoDid you also see the Captain Disillusion video today? "CD / The Horrors of the Alpha Channel" https://www.youtube.com/watch?v=XobSAXZaKJ8 https://www.youtube.com/watch?v=XobSAXZaKJ8
- stoplying1 4y agoI clicked on this hn link specifically because I had just skimmed through that video and assumed it had to be related. Obv it could be a coincidence, but... (Not the worst YT to reference anyway, in my view)
- gus_massa 4y agoJust in case, to avoid confusion, I'm not complaining about the current submission. * 50% curiosity about the coincidence * 50% readers that liked this article may also like ...
- im3w1l 4y agoBecause I like being difficult, what about gamma-correction?
- stateoff 4y agoGamma should not have any effect on the alpha channel, which is linear by definition. The alpha channel represents the average coverage of (infinite) subpixels within the pixel. The color encoding of an object should not have any impact on the coverage within that pixel. I recommend reading the technical memo "Alpha and the History of Digital Compositing" by Alvy Ray Smith [1] to get a better intuition on the matter. [1] http://alvyray.com/Memos/CG/Microsoft/7_alpha.pdf http://alvyray.com/Memos/CG/Microsoft/7_alpha.pdf
- im3w1l 4y agoGamma does however have an effect on blurring. And I suspect that because of gamma correction, applying a blur filter to a premultiplied image is (very subtly) incorrect.
- ljkbkhvj 4y agoYes, I think the possibilities, in practice, for bad interactions between gamma-related and alpha-related operations are real. "PS3 does sRGB conversion before alpha-blending, so the blending is done in gamma space, which is not quite right." https://tomforsyth1000.github.io/blog.wiki.html https://tomforsyth1000.github.io/blog.wiki.html
- Sesse__ 4y agoApplying a blur filter to a premultiplied image is (very subtly) _correct_, at least if the goal is to emulate what happens if you used a physical lens to blur the same image. Not only does postmultiplied alpha mess up the correct pixel values (as the original post shows), but even without alpha, you get a “halo” that is weird and unphysical.
- im3w1l 4y agoI think corysama had the right of it, you need to do all the operations in linear colors, i.e. original image -> convert to linear color space -> multiply alpha -> blur -> composite -> possibly convert back to sRGB if needed The wrong way I meant was original image -> multiply alpha -> gamma-correct-blur -> composite where gamma-correct-blur = convert to linear color space -> blur filter -> convert to sRGB
- corysama 4y agoYou need to go gamma -> linear before doing any linear algebra on your color values. There is some debate over whether the alpha should be stored linearly or not in low precision (8 bit) formats. I think the standard practice is linear, but there’s good arguments it should be gamma encoded. And, from there it’s important to point out that while pre-mul alpha is great for the math, having a color recorded in an 8-bit per channel image pre-multiplied doesn’t leave many possible values in the low range. So, you can get a lot of banding. Dithering becomes important. You should probably store the image non-premultiplied and do the premul as part of the math at runtime. Unfortunately, almost all image editors and encoders make it impossible to control the rgb values of low or zero alpha pixels. They all assume those colors conceptually “don’t exist” and don’t matter. But, they matter here because the filtered transition between solid and transparent pixels is the whole issue we are struggling with here.
- jb1991 4y agoDo you have any recommended resources for self-learning a ground-up education in these concepts? Book, etc? I've struggled to find any good ones.
- lukecameron 4y agoNot a book, but I had a similar question yesterday and ended up finding this article https://tomforsyth1000.github.io/blog.wiki.html#%5B%5BThe%20sRGB%20learning%20curve%5D%5D https://tomforsyth1000.github.io/blog.wiki.html#%5B%5BThe%20.... Goes into more detail than I thought I needed!
- kevingadd 4y agoIf you create a sRGB texture on a modern GPU, the R/G/B channels are stored in sRGB (functionally a lot like gamma correction in this case) while the alpha channel is always linear 0-255. So while the GPU's texture samplers will convert the R/G/B channels to linear before filtering or sending the result to a shader, nothing gets done to the alpha channel.
- turtledragonfly 4y agoFor another nice discussion of this topic, see this older publication by Blinn (yes that Blinn): https://courses.cs.washington.edu/courses/cse576/03sp/readings/blinnCGA94.pdf https://courses.cs.washington.edu/courses/cse576/03sp/readin... In this older terminology, pre-multiplied alpha is called associated alpha. He goes through the math, but a key takeaway is: "... downsampling and, in fact, all filtering operations should operate on arrays of associated pixel colors ..." I really enjoy Blinn's articles (and iq's :).
- TazeTSchnitzel 4y agoOne reason “associated alpha” is a better name is that it allows doing useful things with images where the alpha isn't correlated to the colour. In the simplest example, you have zero alpha but you have non-zero (non-black) colour. That gives you a pure additive blending effect! But you can arbitrarily choose your alpha values, so you can choose how much occlusion you want and how much added light. Classic non-premultiplied alpha has none of this power. I wish I could find the article I read that argued for this. It had a lovely demonstration of how you could use this to combine fire and smoke effects into a single texture (smoke occludes but fire effects tend not to).
- amluto 4y agoFor similar reasons, one should blend and scale in a linear color space. It doesn’t fundamentally matter which linear color space, but sRGB and most other color spaces used in image files are nonlinear, and that nonlinearity scales wrong much like the (alpha * color term) is nonlinear and scales wrong.
- Asooka 4y agoThis is wrong, sRGB is not a "nonlinear color space". In fact, the term itself makes no sense, all color spaces are defined in terms of linear intensity. What you mean is that sRGB is usually stored and transmitted with a transfer function that is nonlinear. Once you linearise, i.e. apply the reverse transfer function, you are left with linear sRGB, as opposed to some other color space like Adobe RGB, or ACEScg. But yes, you must do your blending in linear values, otherwise what you get is nonsense.
- pornel 4y agosRGB and its transfer function are inseparable, by definition. IEC 61966-2-1:1999 defines sRGB as a color space with its particular non-linear transfer function. When you linearize sRGB you create a new not-sRGB-anymore color space that happens to have a compatible white point and RGB primaries.
- esperent 4y ago> a new not-sRGB-anymore color space that happens to have a compatible white point and RGB primaries. Do you know if this new color space has an official name? I work with it basically everyday via the three.js renderer internals and mostly see it refered to as simply "linear", but if more specificity is needed usually I see "linear sRGB". I think that's common even if it's not correct. However as HDR color spaces take over from sRGB I expect it'll become more important to have a correct name for this color space. I have searched for one previously but haven't been able to come up with anything better than "linear sRGB".
- 4y ago
- jb1991 4y agoDoes anyone know of any good books that teach these concepts ground-up? I would think that this area of graphics would have dedicated books on the topic, but I have not found any. Color spaces, compositing, linear, srgb, all that good stuff.
- bsenftner 4y agoThe book you want is called within the graphics industry "Foley Van Dam", after the original authors, but the actual title is "Computer Graphics: Principles and Practice". https://www.amazon.com/Computer-Graphics-Principles-Practice-3rd-dp-0321399528/dp/0321399528/ref=dp_ob_title_bk https://www.amazon.com/Computer-Graphics-Principles-Practice... You may also be interested in "ACM: Transactions on Graphics", the Association of Computing Machinery's publication of computer graphics research papers. I suggest going to a University technical/research library, where you should be able to access the collection of issues from the 80's, where the original scan line, ray tracing, CSG, and pretty much every single advanced graphics technique (minus the deep learning) used today is documented by the original innovators. At that same University research library they might have the collected set of course text books (mimeographs and photocopies) used for the 3-day long courses taught at SIGGRAPH every year. These items are invaluable, and I reference them multiple times a year.
- jb1991 4y agoThis is incredibly useful, thank you so much!
- dahart 4y agoI only have 2nd edition of Foley & vanDam’s GCPP, which predates sRGB, I assume the 3rd edition has been modernized? I wouldn’t have thought of this book as the first place to learn color & compositing specifically, but it’s probably decent. I haven’t referred to CGPP much lately, but maybe it has something to do with the time Andy van Dam loaned me one of his swimsuits and it was slightly too small for me. :P
- dahart 4y agoCheck out Andrew Glassner’s Principles of Digital Image Synthesis, it starts with an overview of color. http://www.realtimerendering.com/Principles_of_Digital_Image_Synthesis_v1.0.1.pdf http://www.realtimerendering.com/Principles_of_Digital_Image... Also these days there may be better and more focused resources online if you know where to look. http://poynton.ca/notes/colour_and_gamma/ColorFAQ.html http://poynton.ca/notes/colour_and_gamma/ColorFAQ.html http://www.cvrl.org/ http://www.cvrl.org/ https://www.handprint.com/HP/WCL/color2.html https://www.handprint.com/HP/WCL/color2.html (https://www.handprint.com/CE/book.html https://www.handprint.com/CE/book.html) https://www.realtimerendering.com/ https://www.realtimerendering.com/ http://paulbourke.net/miscellaneous/colourspace/ http://paulbourke.net/miscellaneous/colourspace/ (Old but still useful) The original compositing paper (which didn’t address issues of perception or non-linearity): https://keithp.com/~keithp/porterduff/p253-porter.pdf https://keithp.com/~keithp/porterduff/p253-porter.pdf