9 ms·

Beware of Transparent Pixels

567 points by tsemple 9y ago

Kiro 9y ago
> pay attention to what color you put inside the transparent pixels I don't understand this. When I make transparency I don't use any color? I use the Eraser tool or Ctrl-X, not a color with 0 opacity.
- saurik 9y ago
  "And be careful to export the RGB values of transparent pixels when you save to PNG for example, many programs will by default discard transparent pixel RGB data and replace it with a solid color (white or black) during the export to help with the compression." Here is some information for Photoshop and a plugin you can use: https://graphicdesign.stackexchange.com/questions/63783/how-do-i-tell-photoshop-to-preserve-the-rgb-color-of-fully-transparent-pixels-wh https://graphicdesign.stackexchange.com/questions/63783/how-...
- XaspR8d 9y ago
  Regardless of how you perceive making it, the representation is still (nearly always) a color underneath an alpha channel. If you inspect the image in a full-featured editor like Photoshop or GIMP, you can inspect the channels individually or remove the transparency entirely to see this fact. I think with most GUI programs, the eraser and cut tool will leave color as what it was before it became transparent. EDIT: saurik has a good point that I forgot about -- many editors may actually throw the colors away when you export unless you ask them not to.
- llccbb 9y ago
  Each pixel is defined by 32 bits -- 8 R, 8 B, 8 G, 8 A. Even if alpha is 0, there has to be information stored in Red, Green, Blue. There can't be "no information", because then a pixel is not defined by 32 bits of information. There is always a "color" (RGB values) for transparent pixels. Some editors will set the RGB values to 0 or 255 when fully-transparent pixels are saved/output. Others don't. As the author mentions, certain resampling algorithms might be naive or plain ignorant about how to resample images with potentially transparent pixels. Should transparent pixels not count to the final pixel? Should all pixels be averaged? Should the output pixel be the median or mode value? If the image is being resampled to 1/3 its size the resampling can be very cheap if only the middle of the 3x3 pixel cluster is selected as the output value.
- munchbunny 9y ago
  This is actually a very common problem with 3-D stuff and transparency in textures. This isn't an issue with the colors of the pixels themselves, it's an issue with texture filtering. nVidia has a pretty good explanation as it applies to games and 3d graphics: https://developer.nvidia.com/content/alpha-blending-pre-or-not-pre https://developer.nvidia.com/content/alpha-blending-pre-or-n... Say you have two adjacent pixels using floating point RGBA values of (0,0,0,0) and (1,1,1,1), and you apply it to a 3-d shape. Because of the rasterization algorithm, you will be sampling weighted averages of the two pixels, either because you're scaling up and need to interpolate, or because you're scaling down and need to average. The average of (0,0,0,0) (fully transparent) and (1,1,1,1) (opaque white) is (0.5,0.5,0.5,0.5), a half transparent gray. But you'd intuitively expect (1,1,1,0.5), half transparent white. This is the essence of the problem. The fix is to make sure that your transparent pixel was (1,1,1,0) and not (0,0,0,0).
  - twelvechairs 9y ago
    Surely the answer if you want this is to weight the final RGB by the transparency. E.g. the final red channel would be (R1xT1 + R2xT2)/(T1+T2)
    - munchbunny 9y ago
      You're mostly right. The industry-wide accepted answer is to multiply the opacity into the colors before interpolating, so the formula would be (R1xT1 + R2xT2)/2 for the average, and then to do later transparency blending as if the opacity term was already multiplied in.
      - twelvechairs 9y ago
        Which means that your output pixel cant be both white and low transparency. I guess its a typical graphics 'close enough and better performance' outcome (where mine is marginally more difficult to calculate and needs some more logic to avoid divide by 0)
        9y ago
- shawnz 9y ago
  When you erase, it has a soft edge, right? And in the edge, you see the colour that was there before is still there, just more transparent. Well similarly, in the area that is fully erased, it is fully transparent.
- evincarofautumn 9y ago
  An eraser tool and a paintbrush tool do essentially the same thing—overwrite an area of the image with new pixel values, typically blended in some way with the old values. It’s just that the eraser is for painting in the alpha channel, while the paintbrush also affects colour channels. An alpha mask (on a layer with no alpha channel) is essentially a different way of viewing & editing the same data, and there you probably have no logical trouble with using a paintbrush tool.
ninjakeyboard 9y ago
s/sawn/swan/
tantalor 9y ago
Reminds me of "Is there a reason Hillary Clinton's logo has hidden notches?" https://graphicdesign.stackexchange.com/questions/73601/is-there-a-reason-hillary-clintons-logo-has-hidden-notches https://graphicdesign.stackexchange.com/questions/73601/is-t...
- microcolonel 9y ago
  I had to do a similar thing with the Gitlab logo[0] (and it's still messed up on their marketing sites). There are limitations to the SVG renderers in browsers (and elsewhere), it is kinda sad that we don't have AGG-quality rendering as a standard feature of browsers these days. :- ( [0]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/2741 https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/2741
- zeckalpha 9y ago
  ... which reminds me of similar notches from typography at small sizes. Interesting how physical bleeding ink and rendering issues result in similar workarounds.
  - minikites 9y ago
    For the curious: https://en.wikipedia.org/wiki/Ink_trap https://en.wikipedia.org/wiki/Ink_trap
    - zeckalpha 9y ago
      Thanks! I couldn't remember the term.
- bsimpson 9y ago
  Yakk's explanation there is great.
- jerryr 9y ago
  Yeah! I recently built a website that heavily uses Illustrator assets exported as svg. I was happy to have read that explanation before pulling my hair out over weird artifacts.
dvt 9y ago
> Even with an alpha of 0, a pixel still has some RGB color value associated with it. Wish the article was more clear as to why this happens. Let me elucidate: this happens because, per the PNG standard[0], 0-alpha pixels have their color technically undefined. This means that image editors can use these values (e.g. XX XX XX 00) for whatever -- generally some way of optimizing, or, more often than not, just garbage. There are ways to get around this by using an actual alpha channel in Photoshop[1], or by using certain flags in imagemagick[2]. [0] https://www.w3.org/TR/PNG/ https://www.w3.org/TR/PNG/ [1] https://feedback.photoshop.com/photoshop_family/topics/png-transparent-alpha-0-pixels-undefined-rgb-values https://feedback.photoshop.com/photoshop_family/topics/png-t... [2] http://www.imagemagick.org/discourse-server/viewtopic.php?t=13746 http://www.imagemagick.org/discourse-server/viewtopic.php?t=...
- munchbunny 9y ago
  While what you wrote is correct, it's not actually the problem being described in the posted link. The problem in the posted link is just as relevant if the alpha value was "01" for "you pretty much can't see it but it's meaningfully there", and has to do with image filtering artifacts as opposed to purely data representation.
  - tgb 9y ago
    Also the problem can occur with any image format; what matters is how it's stored in the texture not on disc.
    - mark-r 9y ago
      But it's specifically a problem with PNG because the spec explicitly allows encoders to substitute their own RGB values for any transparent pixel. Most other formats are expected to store the values you give them without modification.
      - tgb 9y ago
        No, the problem discussed in the article can occur with any image format (that supports alpha) and occurs later in the pipeline so it really doesn't matter how the texture was stored on disc. Or even whether it was stored - this could happen with procedurally generated textures.
jamesbowman 9y ago
Using premultiplied alpha avoids this. Jim Blinn's books from the 90s give a very thoughtful treatment of the topic.
- tastybberries 9y ago
  The article already does recommend using premultiplied alpha. Thank you for the background source though!
- AnimalMuppet 9y ago
  The Porter and Duff compositing paper is good, too. One clarification, though: With premultiplied colors, something like (1,1,1,0) is either illegal or a light source. It's not a valid normal color.
  - tgb 9y ago
    But then you can do additive blending without changing blend modes - or even some additive and some standard transparency modes in the same sprite!
  - foolrush 9y ago
    Entirely false in that it is indeed a completely valid combination. See luminescent pixels in the Porter Duff paper. It represents a pixel that has no occlusion and is emitting.
modeless 9y ago
I don't like this article because it blames the wrong people and buries the real solution, premultiplied alpha, at the bottom. Already there are many comments here that are confused because they didn't even see the premultiplied alpha part of the article. The issue with the Limbo logo was not that the source image was incorrect. The image was fine. The blending was incorrect because the PS3 XMB has a bug. Not using premultiplied alpha when you are doing texture filtering is a bug.
- imron 9y ago
  Exactly. As I was reading the article I was thinking to myself 'this guy needs to learn about pre-multiplied alpha', and then he gets to the end and says, btw use pre-multiplied alpha to avoid this problem entirely.
- kllrnohj 9y ago
  I don't think that's a fair summary of the layout of the article. Once the article reaches the "How to Prevent This Issue" section there's far more space given to using premultiplied alpha than manually bled images, and it's not blaming anyone. It just tells an artist how they could fix it and tells programmers how they could fix it. Nobody is blamed and the "correct" solution isn't buried at all.
fnayr 9y ago
This is extremely useful to take advantage of (that you can store RGB values in 0-alpha pixels). I've written some pretty simple but powerful shaders for a game I'm working on by utilizing transparent pixels' "extra storage" which allowed for either neat visuals or greatly reduced the number of images required to achieve a certain affect. For instance, I wrote a shader for a characters hair that had source images colorized in pure R, G, and B and then mapped those to a set of three colors defining a "hair color" (e.g. R=dark brown, G=light brown, B=brown). If I didn't have the transparent pixels storing rgb nonzero values, the blending between pixels within the image would jagged and the approach would have been unacceptable for production quality leading to each hair style being exported in each hair color. As a total side note I really enjoyed the markup on the website. Seeing the matrices colored to represent their component color value is really helpful for understanding. Nice job author!
- mark-r 9y ago
  Paint Shop Pro had some picture frames that were fully transparent in the middle. As an Easter egg, a couple of those frames had pictures of the staff in those transparent areas and you could use the unerase tool to see them.
  - wink 9y ago
    Aww, just checked my Paint Shop Pro 4.12 (Build Date: Dec 21 1996) and it doesn't yet have these picture frames :/
    - mark-r 9y ago
      That's because 4.12 didn't have partial transparency.
- munchbunny 9y ago
  I'm a bit confused about your explanation. Are you using the transparent pixels to store extra data, or are you using the alpha channel to store extra data? And if you're using the transparent pixels, then what data is it and how do your shaders know how to find the transparent pixels?
VikingCoder 9y ago
Premultiplied alpha results in less color depth, though. If my alpha is 10%, then my possible RGB values become 0-25. Even if I multiply by 10, I still lose the maximum possible values 251-255, and only values 0, 10, 20, 30... 250, are possible. The correct solution is to pay close attention to all of the factors... and to be ESPECIALLY aware of pixel scaling. Provide your RGBA textures at the 1:1 pixel scale they will be rendered (or higher!) if at all possible.
- deleted 9y ago
  [deleted]
- robohamburger 9y ago
  If you were just doing image compositing would this be an issue? I can easily imagine how that would adversely effect a 3d game where the images are used as texture maps and the values of the diffuse texture map may change from other lighting contributions and shaders. It seems like in that case doing some kind of HDR/higher precision texturing would be good right? Disclaimer: not a graphics programmer just a hobbyist seeking clarification. :)
  - munchbunny 9y ago
    In practice it's the other way around. Pre-multiplying alpha adversely affects image editing more than 3d rendering. For image editing, the problem is that you might send the same pixel through many different operations, and if you flatten something and further manipulate, then the rounding error will continue to accumulate. So losing granularity early in the process due to alpha can result in accumulated rounding error. In 3d graphics, the texture typically goes through a very predictable and short number of transformations, and they will rarely need to "stretch" the color range of the transparent pixel. When you alpha blend with pre-multiplied alpha, you're literally just adding the whole color value. And then you're slapping a whole lot of other colors onto it in the lighting/shadow/other passes, so the subtle nuances of that mostly transparent window get lost in the bustle.
    - robohamburger 9y ago
xchip 9y ago
DR;TL: use premultiplied alpha for transparency
Kenji 9y ago
You also have a similar problem when you render opaque, rectangular images without the clamp edge mode, and the renderer is in tiling mode, so the borders wrap around when your picture is halfway between pixels and become a mix between the top/bottom or left/right colour, corrupting the edges. Easy to fix, but annoying until you get what it is that corrupts your edges. Also: "The original color can still be retrieved easily: dividing by alpha will reverse the transformation." C'mon, you can't say that and then make an example with alpha=0. Do you want me to divide by zero? The ability to store values in completely transparent pixels is lost.
- mark-r 9y ago
  It would be more accurate to say that the original color can be approximated. The approximation quality goes down as the transparency goes up, until at zero it gets lost entirely.
  - Kenji 9y ago
    You're right. I didn't want to mention precision because it's a whole other can of worms, but one worth knowing how to deal with. I was looking at it from a mathematician's point of view, using rational numbers.
panic 9y ago
Premultiplied alpha is also more "correct" in that it separates how much each pixel covers things behind it (the alpha value) from the amount of light it is reflecting or emitting (the color values). These two values should really be interpolated separately, and that's what premultiplied alpha gives you.
dahart 9y ago
Really nice article! Succinctly demonstrates the problem with not using premultiplied alpha. > As an Artist: Make it Bleed! > If you’re in charge of producing the asset, be defensive and don’t trust the programmers or the engine down the line. If you are an artist working with programmers that can fix the engine, your absolute first choice should be to ask them to fix the blending so they convert your non-premultiplied images into premultiplied images before rendering them! Do not start bleeding your mattes manually if you have any say in the matter at all, that doesn't solve the whole problem, and it sets you up for future pain. The only right answer is for the programmers to use premultiplied images. What if someone decides to blur your bled transparent image? It will break. (And there are multiple valid reasons this might happen without your input.) Even if you have no control over the engine, file a bug report. But in that case, go ahead and bleed your transparent images manually & do whatever you have to, to get your work done. Eric Haines wrote a more technical piece on this problem that elaborates on the other issues besides halo-ing: http://www.realtimerendering.com/blog/gpus-prefer-premultiplication/ http://www.realtimerendering.com/blog/gpus-prefer-premultipl...
- mark-r 9y ago
  I'm not sure I understand your concern. If the software converts all your assets into a premultiplied form, the bleeding you applied won't hurt anything even if it doesn't help. Yes, it's extra work that shouldn't be necessary - but we often find ourselves living in an imperfect world. I completely agree that premultiplied alpha should be used everywhere. I'd even go a step farther and say that you should use high bit depth linear values too, but that's a topic for another day.
  - theoh 9y ago
    Alvy Ray Smith may have the canonical paper on this, from 1995: http://alvyray.com/Memos/CG/Microsoft/4_comp.pdf http://alvyray.com/Memos/CG/Microsoft/4_comp.pdf
  - ioquatix 9y ago
    A 16bpp linear pre-multiplied format would be awesome. sRGB is a pain to use in practice: - Slow to convert to linear colour space (requires a pow).. except - GPUs use an 8-bit lookup table to convert input sRGB to output linear value. - This doesn't work for more than 8 bits as the table gets excessively large very quickly. It's a pity PNG doesn't have flag to mark the image data as being pre-multiplied.
jayshua 9y ago
While reading this article, it struck me that the amount of "useless" data increases as the alpha value approaches 0. For example: in a pixel with rgba values of (1.0, 0.4, 0.5, 0.0), the rgb values are redundant. Is there a color format that would prevent this redundancy? Perhaps by some clever equation that incorporates the alpha values into the rgb values? I don't think Premultiplied alpha would work, because you still need to store the alpha value for compositing later...
- mark-r 9y ago
  PNG uses compression to reduce the redundancy. The average number of bits per pixel is much less than 32. A=0 is a special case, and many PNG encoders will take advantage of that by setting RGB to some constant so it compresses better.
- dahart 9y ago
  Would you count compression schemes as valid answers to your question, or are you asking about the raw data not having any redundancy? There're no channel formats or clever equations I'm aware of that avoids the redundancy part. But your question totally reminds me of Greg Ward's RGBE format, which is a high-dynamic range format stored in 8 bits per channel, with an extra 8 bit exponent channel. http://www.graphics.cornell.edu/~bjw/rgbe.html http://www.graphics.cornell.edu/~bjw/rgbe.html RGBE isn't doing exactly what you're asking about, but it's similar in a way. Instead of storing 16 bits per channel separately for each channel, what you really get instead is 16 bits (sort-of) for the brightest channel, and the other 2 channels in 8 bits each - discarding the extra bits. You can't see them because the bright channel will prevent you from seeing anything super dark in another channel, so you can discard the extra color resolution in the darker channels. If you count compression, then pre-multiplying would help the situation. Anytime the alpha value gets low or goes to 0, the color channels do too, so run length encoding or DCT or whatever else will collapse large transparent areas into all zeroes.
- panic 9y ago
  In premultiplied alpha, you can think of all the legal (non-emissive) colors as a triangle. At an alpha of 1, all colors are legal, tapering off as alpha decreases until you reach an alpha of 0, where no colors are legal. To turn the triangle into a square, you could scale it up by two, cut off the top, and rotate the top over into the empty space (like this: https://i.imgur.com/6hjNe7X.png https://i.imgur.com/6hjNe7X.png). That would give you twice the colors you had before.
CurtMonash 9y ago
I thought this would be about web or email tracking.
leni536 9y ago
Note that SVG 1.1 doesn't have an option for color interpolation to work in premultiplied/associated alpha. SVG 2 is not finalized though, I added an issue some time ago. https://github.com/w3c/svgwg/issues/303 https://github.com/w3c/svgwg/issues/303 It affects gradients, animations and imported+scaled raster images. Maybe other stuff too, I don't know.
qwerta 9y ago
There is also performance overhead.
deleted 9y ago
[deleted]
blauditore 9y ago
This is also relevant for CSS: Some browsers (I think Safari) treat "transparent" as "transparent black" for gradiets, so "linear-gradient(transparent, white)" will result in unexpected grayish parts in the gradient. As a workaround, one needs to use "linear-gradient(rgba(255, 255, 255, 0), white)" instead.
br1 9y ago
I'm surprised that John Carmack seems not to use premultiplied alpha and recommends bleeding: https://www.facebook.com/permalink.php?story_fbid=1818885715012604&id=100006735798590 https://www.facebook.com/permalink.php?story_fbid=1818885715...
eukara 9y ago
Used to experience this all the time when making maps with custom textures for older games... Lots of people sure didn't though. Especially source ports that would apply filtering to games that didn't have any in the first place and you'd see blue or purple outlines because their original formats were obviously paletted

567 points by tsemple 9y ago

Kiro 9y ago

> pay attention to what color you put inside the transparent pixels I don't understand this. When I make transparency I don't use any color? I use the Eraser tool or Ctrl-X, not a color with 0 opacity.

saurik 9y ago
"And be careful to export the RGB values of transparent pixels when you save to PNG for example, many programs will by default discard transparent pixel RGB data and replace it with a solid color (white or black) during the export to help with the compression." Here is some information for Photoshop and a plugin you can use: https://graphicdesign.stackexchange.com/questions/63783/how-do-i-tell-photoshop-to-preserve-the-rgb-color-of-fully-transparent-pixels-wh https://graphicdesign.stackexchange.com/questions/63783/how-...
XaspR8d 9y ago
Regardless of how you perceive making it, the representation is still (nearly always) a color underneath an alpha channel. If you inspect the image in a full-featured editor like Photoshop or GIMP, you can inspect the channels individually or remove the transparency entirely to see this fact. I think with most GUI programs, the eraser and cut tool will leave color as what it was before it became transparent. EDIT: saurik has a good point that I forgot about -- many editors may actually throw the colors away when you export unless you ask them not to.
llccbb 9y ago
Each pixel is defined by 32 bits -- 8 R, 8 B, 8 G, 8 A. Even if alpha is 0, there has to be information stored in Red, Green, Blue. There can't be "no information", because then a pixel is not defined by 32 bits of information. There is always a "color" (RGB values) for transparent pixels. Some editors will set the RGB values to 0 or 255 when fully-transparent pixels are saved/output. Others don't. As the author mentions, certain resampling algorithms might be naive or plain ignorant about how to resample images with potentially transparent pixels. Should transparent pixels not count to the final pixel? Should all pixels be averaged? Should the output pixel be the median or mode value? If the image is being resampled to 1/3 its size the resampling can be very cheap if only the middle of the 3x3 pixel cluster is selected as the output value.
munchbunny 9y ago
This is actually a very common problem with 3-D stuff and transparency in textures. This isn't an issue with the colors of the pixels themselves, it's an issue with texture filtering. nVidia has a pretty good explanation as it applies to games and 3d graphics: https://developer.nvidia.com/content/alpha-blending-pre-or-not-pre https://developer.nvidia.com/content/alpha-blending-pre-or-n... Say you have two adjacent pixels using floating point RGBA values of (0,0,0,0) and (1,1,1,1), and you apply it to a 3-d shape. Because of the rasterization algorithm, you will be sampling weighted averages of the two pixels, either because you're scaling up and need to interpolate, or because you're scaling down and need to average. The average of (0,0,0,0) (fully transparent) and (1,1,1,1) (opaque white) is (0.5,0.5,0.5,0.5), a half transparent gray. But you'd intuitively expect (1,1,1,0.5), half transparent white. This is the essence of the problem. The fix is to make sure that your transparent pixel was (1,1,1,0) and not (0,0,0,0).
- twelvechairs 9y ago
  Surely the answer if you want this is to weight the final RGB by the transparency. E.g. the final red channel would be (R1xT1 + R2xT2)/(T1+T2)
  - munchbunny 9y ago
    You're mostly right. The industry-wide accepted answer is to multiply the opacity into the colors before interpolating, so the formula would be (R1xT1 + R2xT2)/2 for the average, and then to do later transparency blending as if the opacity term was already multiplied in.
    - twelvechairs 9y ago
      Which means that your output pixel cant be both white and low transparency. I guess its a typical graphics 'close enough and better performance' outcome (where mine is marginally more difficult to calculate and needs some more logic to avoid divide by 0)
      - 9y ago
shawnz 9y ago
When you erase, it has a soft edge, right? And in the edge, you see the colour that was there before is still there, just more transparent. Well similarly, in the area that is fully erased, it is fully transparent.
evincarofautumn 9y ago
An eraser tool and a paintbrush tool do essentially the same thing—overwrite an area of the image with new pixel values, typically blended in some way with the old values. It’s just that the eraser is for painting in the alpha channel, while the paintbrush also affects colour channels. An alpha mask (on a layer with no alpha channel) is essentially a different way of viewing & editing the same data, and there you probably have no logical trouble with using a paintbrush tool.

ninjakeyboard 9y ago

s/sawn/swan/

tantalor 9y ago

Reminds me of "Is there a reason Hillary Clinton's logo has hidden notches?" https://graphicdesign.stackexchange.com/questions/73601/is-there-a-reason-hillary-clintons-logo-has-hidden-notches https://graphicdesign.stackexchange.com/questions/73601/is-t...

microcolonel 9y ago
I had to do a similar thing with the Gitlab logo[0] (and it's still messed up on their marketing sites). There are limitations to the SVG renderers in browsers (and elsewhere), it is kinda sad that we don't have AGG-quality rendering as a standard feature of browsers these days. :- ( [0]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/2741 https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/2741
zeckalpha 9y ago
... which reminds me of similar notches from typography at small sizes. Interesting how physical bleeding ink and rendering issues result in similar workarounds.
- minikites 9y ago
  For the curious: https://en.wikipedia.org/wiki/Ink_trap https://en.wikipedia.org/wiki/Ink_trap
  - zeckalpha 9y ago
    Thanks! I couldn't remember the term.
bsimpson 9y ago
Yakk's explanation there is great.
jerryr 9y ago
Yeah! I recently built a website that heavily uses Illustrator assets exported as svg. I was happy to have read that explanation before pulling my hair out over weird artifacts.

dvt 9y ago

> Even with an alpha of 0, a pixel still has some RGB color value associated with it. Wish the article was more clear as to why this happens. Let me elucidate: this happens because, per the PNG standard[0], 0-alpha pixels have their color technically undefined. This means that image editors can use these values (e.g. XX XX XX 00) for whatever -- generally some way of optimizing, or, more often than not, just garbage. There are ways to get around this by using an actual alpha channel in Photoshop[1], or by using certain flags in imagemagick[2]. [0] https://www.w3.org/TR/PNG/ https://www.w3.org/TR/PNG/ [1] https://feedback.photoshop.com/photoshop_family/topics/png-transparent-alpha-0-pixels-undefined-rgb-values https://feedback.photoshop.com/photoshop_family/topics/png-t... [2] http://www.imagemagick.org/discourse-server/viewtopic.php?t=13746 http://www.imagemagick.org/discourse-server/viewtopic.php?t=...

munchbunny 9y ago
While what you wrote is correct, it's not actually the problem being described in the posted link. The problem in the posted link is just as relevant if the alpha value was "01" for "you pretty much can't see it but it's meaningfully there", and has to do with image filtering artifacts as opposed to purely data representation.
- tgb 9y ago
  Also the problem can occur with any image format; what matters is how it's stored in the texture not on disc.
  - mark-r 9y ago
    But it's specifically a problem with PNG because the spec explicitly allows encoders to substitute their own RGB values for any transparent pixel. Most other formats are expected to store the values you give them without modification.
    - tgb 9y ago
      No, the problem discussed in the article can occur with any image format (that supports alpha) and occurs later in the pipeline so it really doesn't matter how the texture was stored on disc. Or even whether it was stored - this could happen with procedurally generated textures.

jamesbowman 9y ago

Using premultiplied alpha avoids this. Jim Blinn's books from the 90s give a very thoughtful treatment of the topic.

tastybberries 9y ago
The article already does recommend using premultiplied alpha. Thank you for the background source though!
AnimalMuppet 9y ago
The Porter and Duff compositing paper is good, too. One clarification, though: With premultiplied colors, something like (1,1,1,0) is either illegal or a light source. It's not a valid normal color.
- tgb 9y ago
  But then you can do additive blending without changing blend modes - or even some additive and some standard transparency modes in the same sprite!
- foolrush 9y ago
  Entirely false in that it is indeed a completely valid combination. See luminescent pixels in the Porter Duff paper. It represents a pixel that has no occlusion and is emitting.

modeless 9y ago

I don't like this article because it blames the wrong people and buries the real solution, premultiplied alpha, at the bottom. Already there are many comments here that are confused because they didn't even see the premultiplied alpha part of the article. The issue with the Limbo logo was not that the source image was incorrect. The image was fine. The blending was incorrect because the PS3 XMB has a bug. Not using premultiplied alpha when you are doing texture filtering is a bug.

imron 9y ago
Exactly. As I was reading the article I was thinking to myself 'this guy needs to learn about pre-multiplied alpha', and then he gets to the end and says, btw use pre-multiplied alpha to avoid this problem entirely.
kllrnohj 9y ago
I don't think that's a fair summary of the layout of the article. Once the article reaches the "How to Prevent This Issue" section there's far more space given to using premultiplied alpha than manually bled images, and it's not blaming anyone. It just tells an artist how they could fix it and tells programmers how they could fix it. Nobody is blamed and the "correct" solution isn't buried at all.

fnayr 9y ago

This is extremely useful to take advantage of (that you can store RGB values in 0-alpha pixels). I've written some pretty simple but powerful shaders for a game I'm working on by utilizing transparent pixels' "extra storage" which allowed for either neat visuals or greatly reduced the number of images required to achieve a certain affect. For instance, I wrote a shader for a characters hair that had source images colorized in pure R, G, and B and then mapped those to a set of three colors defining a "hair color" (e.g. R=dark brown, G=light brown, B=brown). If I didn't have the transparent pixels storing rgb nonzero values, the blending between pixels within the image would jagged and the approach would have been unacceptable for production quality leading to each hair style being exported in each hair color. As a total side note I really enjoyed the markup on the website. Seeing the matrices colored to represent their component color value is really helpful for understanding. Nice job author!

mark-r 9y ago
Paint Shop Pro had some picture frames that were fully transparent in the middle. As an Easter egg, a couple of those frames had pictures of the staff in those transparent areas and you could use the unerase tool to see them.
- wink 9y ago
  Aww, just checked my Paint Shop Pro 4.12 (Build Date: Dec 21 1996) and it doesn't yet have these picture frames :/
  - mark-r 9y ago
    That's because 4.12 didn't have partial transparency.
munchbunny 9y ago
I'm a bit confused about your explanation. Are you using the transparent pixels to store extra data, or are you using the alpha channel to store extra data? And if you're using the transparent pixels, then what data is it and how do your shaders know how to find the transparent pixels?

VikingCoder 9y ago

Premultiplied alpha results in less color depth, though. If my alpha is 10%, then my possible RGB values become 0-25. Even if I multiply by 10, I still lose the maximum possible values 251-255, and only values 0, 10, 20, 30... 250, are possible. The correct solution is to pay close attention to all of the factors... and to be ESPECIALLY aware of pixel scaling. Provide your RGBA textures at the 1:1 pixel scale they will be rendered (or higher!) if at all possible.

deleted 9y ago
[deleted]
robohamburger 9y ago
If you were just doing image compositing would this be an issue? I can easily imagine how that would adversely effect a 3d game where the images are used as texture maps and the values of the diffuse texture map may change from other lighting contributions and shaders. It seems like in that case doing some kind of HDR/higher precision texturing would be good right? Disclaimer: not a graphics programmer just a hobbyist seeking clarification. :)
- munchbunny 9y ago
  In practice it's the other way around. Pre-multiplying alpha adversely affects image editing more than 3d rendering. For image editing, the problem is that you might send the same pixel through many different operations, and if you flatten something and further manipulate, then the rounding error will continue to accumulate. So losing granularity early in the process due to alpha can result in accumulated rounding error. In 3d graphics, the texture typically goes through a very predictable and short number of transformations, and they will rarely need to "stretch" the color range of the transparent pixel. When you alpha blend with pre-multiplied alpha, you're literally just adding the whole color value. And then you're slapping a whole lot of other colors onto it in the lighting/shadow/other passes, so the subtle nuances of that mostly transparent window get lost in the bustle.
  - robohamburger 9y ago

xchip 9y ago

DR;TL: use premultiplied alpha for transparency

Kenji 9y ago

You also have a similar problem when you render opaque, rectangular images without the clamp edge mode, and the renderer is in tiling mode, so the borders wrap around when your picture is halfway between pixels and become a mix between the top/bottom or left/right colour, corrupting the edges. Easy to fix, but annoying until you get what it is that corrupts your edges. Also: "The original color can still be retrieved easily: dividing by alpha will reverse the transformation." C'mon, you can't say that and then make an example with alpha=0. Do you want me to divide by zero? The ability to store values in completely transparent pixels is lost.

mark-r 9y ago
It would be more accurate to say that the original color can be approximated. The approximation quality goes down as the transparency goes up, until at zero it gets lost entirely.
- Kenji 9y ago
  You're right. I didn't want to mention precision because it's a whole other can of worms, but one worth knowing how to deal with. I was looking at it from a mathematician's point of view, using rational numbers.

panic 9y ago

Premultiplied alpha is also more "correct" in that it separates how much each pixel covers things behind it (the alpha value) from the amount of light it is reflecting or emitting (the color values). These two values should really be interpolated separately, and that's what premultiplied alpha gives you.

dahart 9y ago

Really nice article! Succinctly demonstrates the problem with not using premultiplied alpha. > As an Artist: Make it Bleed! > If you’re in charge of producing the asset, be defensive and don’t trust the programmers or the engine down the line. If you are an artist working with programmers that can fix the engine, your absolute first choice should be to ask them to fix the blending so they convert your non-premultiplied images into premultiplied images before rendering them! Do not start bleeding your mattes manually if you have any say in the matter at all, that doesn't solve the whole problem, and it sets you up for future pain. The only right answer is for the programmers to use premultiplied images. What if someone decides to blur your bled transparent image? It will break. (And there are multiple valid reasons this might happen without your input.) Even if you have no control over the engine, file a bug report. But in that case, go ahead and bleed your transparent images manually & do whatever you have to, to get your work done. Eric Haines wrote a more technical piece on this problem that elaborates on the other issues besides halo-ing: http://www.realtimerendering.com/blog/gpus-prefer-premultiplication/ http://www.realtimerendering.com/blog/gpus-prefer-premultipl...

mark-r 9y ago
I'm not sure I understand your concern. If the software converts all your assets into a premultiplied form, the bleeding you applied won't hurt anything even if it doesn't help. Yes, it's extra work that shouldn't be necessary - but we often find ourselves living in an imperfect world. I completely agree that premultiplied alpha should be used everywhere. I'd even go a step farther and say that you should use high bit depth linear values too, but that's a topic for another day.
- theoh 9y ago
  Alvy Ray Smith may have the canonical paper on this, from 1995: http://alvyray.com/Memos/CG/Microsoft/4_comp.pdf http://alvyray.com/Memos/CG/Microsoft/4_comp.pdf
- ioquatix 9y ago
  A 16bpp linear pre-multiplied format would be awesome. sRGB is a pain to use in practice: - Slow to convert to linear colour space (requires a pow).. except - GPUs use an 8-bit lookup table to convert input sRGB to output linear value. - This doesn't work for more than 8 bits as the table gets excessively large very quickly. It's a pity PNG doesn't have flag to mark the image data as being pre-multiplied.

jayshua 9y ago

While reading this article, it struck me that the amount of "useless" data increases as the alpha value approaches 0. For example: in a pixel with rgba values of (1.0, 0.4, 0.5, 0.0), the rgb values are redundant. Is there a color format that would prevent this redundancy? Perhaps by some clever equation that incorporates the alpha values into the rgb values? I don't think Premultiplied alpha would work, because you still need to store the alpha value for compositing later...

mark-r 9y ago
PNG uses compression to reduce the redundancy. The average number of bits per pixel is much less than 32. A=0 is a special case, and many PNG encoders will take advantage of that by setting RGB to some constant so it compresses better.
dahart 9y ago
Would you count compression schemes as valid answers to your question, or are you asking about the raw data not having any redundancy? There're no channel formats or clever equations I'm aware of that avoids the redundancy part. But your question totally reminds me of Greg Ward's RGBE format, which is a high-dynamic range format stored in 8 bits per channel, with an extra 8 bit exponent channel. http://www.graphics.cornell.edu/~bjw/rgbe.html http://www.graphics.cornell.edu/~bjw/rgbe.html RGBE isn't doing exactly what you're asking about, but it's similar in a way. Instead of storing 16 bits per channel separately for each channel, what you really get instead is 16 bits (sort-of) for the brightest channel, and the other 2 channels in 8 bits each - discarding the extra bits. You can't see them because the bright channel will prevent you from seeing anything super dark in another channel, so you can discard the extra color resolution in the darker channels. If you count compression, then pre-multiplying would help the situation. Anytime the alpha value gets low or goes to 0, the color channels do too, so run length encoding or DCT or whatever else will collapse large transparent areas into all zeroes.
panic 9y ago
In premultiplied alpha, you can think of all the legal (non-emissive) colors as a triangle. At an alpha of 1, all colors are legal, tapering off as alpha decreases until you reach an alpha of 0, where no colors are legal. To turn the triangle into a square, you could scale it up by two, cut off the top, and rotate the top over into the empty space (like this: https://i.imgur.com/6hjNe7X.png https://i.imgur.com/6hjNe7X.png). That would give you twice the colors you had before.

CurtMonash 9y ago

I thought this would be about web or email tracking.

leni536 9y ago

Note that SVG 1.1 doesn't have an option for color interpolation to work in premultiplied/associated alpha. SVG 2 is not finalized though, I added an issue some time ago. https://github.com/w3c/svgwg/issues/303 https://github.com/w3c/svgwg/issues/303 It affects gradients, animations and imported+scaled raster images. Maybe other stuff too, I don't know.

qwerta 9y ago

There is also performance overhead.

deleted 9y ago

[deleted]

blauditore 9y ago

This is also relevant for CSS: Some browsers (I think Safari) treat "transparent" as "transparent black" for gradiets, so "linear-gradient(transparent, white)" will result in unexpected grayish parts in the gradient. As a workaround, one needs to use "linear-gradient(rgba(255, 255, 255, 0), white)" instead.

br1 9y ago

I'm surprised that John Carmack seems not to use premultiplied alpha and recommends bleeding: https://www.facebook.com/permalink.php?story_fbid=1818885715012604&id=100006735798590 https://www.facebook.com/permalink.php?story_fbid=1818885715...

eukara 9y ago

Used to experience this all the time when making maps with custom textures for older games... Lots of people sure didn't though. Especially source ports that would apply filtering to games that didn't have any in the first place and you'd see blue or purple outlines because their original formats were obviously paletted