4 ms·

Porter/Duff Compositing and Blend Modes (2013)

21 points by teruakohatu 3y ago

pavlov 3y ago
Porter-Duff refers to a seminal 1984 paper titled simply "Compositing Digital Images": https://graphics.pixar.com/library/Compositing/paper.pdf https://graphics.pixar.com/library/Compositing/paper.pdf This research was sponsored by Lucasfilm because at the time, alpha compositing of bitmap images wasn't really a concern for desktop computers. CPU, memory, and display hardware wasn't there to do anything useful with alpha (other than 1-bit masks used for game characters and GUI icons). OTOH Lucasfilm (and other film visual effects companies) had a long history of using mattes, the physical photography equivalents of alpha compositing. When you see a group of Stormtroopers marching inside a Death Star hangar in 1977 "Star Wars", the hangar is a matte painting. They were typically created by artists directly on a large glass that was carefully placed in front of the camera to create the illusion. Porter and Duff came up with a compositing model that's appealingly simple: it's a single formula, three variables, and you get 12 compositing operators by simply enumerating the combinations. You can still find these Porter-Duff operators pretty much anywhere that 2D graphics are processed. They're in HTML Canvas, PDF, Apple CoreGraphics... But, as neatly mathematical as these operators are, most of them aren't really practical for rendering actual scenes consisting of graphics layers. They're not terribly intuitive for designers, and they don't even attempt to simulate the effects of combining light. So when Photoshop was created, its designers needed a set of blending operations (initially for drawing tools, and soon after for combining layers, which were introduced in PS version 3.0). Exposing the Porter-Duff operators to designers in the Photoshop GUI wouldn't have been very useful. Instead they came up with a set of blending modes that were loosely derived from operations that you could do on physical photography negatives, since that was the inspiration for the application. (The PS blending modes still retain names like "dodge" and "burn" which refer to darkroom techniques that no designer aged under 50 probably remembers.) These PS blending modes are conceptually pretty much the opposite of the Porter-Duff modes: they're not mathematically rigorous, but just a collection of algorithms that could be executed in reasonable time on a 68k processor back in the early 1990s. These original limitations are visible in the design of the algorithms: for example, they don't even try to account for gamma correction in the source images, which means that the algorithms do a rather poor job of simulating the behavior of actual light. (Photoshop has later added "linear" versions of some blending modes to address this.) Still, they are the closest thing we have to a standard for blending modes. Many 2D graphics APIs that originally only supported Porter-Duff have added support for PS modes too, including the aforementioned PDF and CoreGraphics.
- fsloth 3y ago
  Thank you, I got really confused when the linked page started to discuss Photoshop blending modes and I had never considered they could be mapped directly to Porter-Duff ops. As a graphics programmer I always want to point out that it's good to remember NONE of these are scientifically foundational. These are more or less just hacks that looks nice - pure heuristics that is. They are really pragmatic heuristics that produce nice outcomes.
  - atoav 3y ago
    I mean add and multiply pretty much work as expected (and I would say except for overlay, color, saturation and lumiosity they are the only ones you will ever need).
    - Daub 3y ago
      I would add to that list color dodge. Used in the right way, it produces results that mimic natural highlights. A highlight will impact all values except black (I.e. deep shadow) which it will ignore. My teaching notes on color dodge here: https://rmit.instructure.com/courses/87565/modules/items/4213799 https://rmit.instructure.com/courses/87565/modules/items/421... On overlay (a very difficult blend mode to use) here: https://rmit.instructure.com/courses/87565/modules/items/4081525 https://rmit.instructure.com/courses/87565/modules/items/408... And on blnd modes in general here: https://rmit.instructure.com/courses/87565/modules/items/3222630 https://rmit.instructure.com/courses/87565/modules/items/322... Good to hear about porter/duff. Was news to me. The furthest back I go is 'Photoshop Channel Chops' Biedny, D., Monroy, B., & Moody, N. (1996). New Riders Publishing.
  - deleted 3y ago
    [deleted]
- creata 3y ago
  > They're not terribly intuitive for designers, and they don't even attempt to simulate the effects of combining light. Can you expand on this? I thought that to combine the effect of light from different sources, you could just add the intensities together.
mikhailfranco 3y ago
Pixel averages and Porter-Duff alpha blending will always give fuzzy results. You need sub-pixel geometric info to do a more accurate depth blend. The classic reference is the A-Buffer by Loren Carpenter (SIGGRAPH 1984). One interesting possibility is to go from a binary matte or averaged anti-aliased layer to subpixel geometry by generating a contour (e.g. Marching Squares). Then the coarse layers (e.g. green screen) can be incorporated into a high-resolution A-Buffer rendering pipeline with 3D CGI elements.

21 points by teruakohatu 3y ago

Porter-Duff refers to a seminal 1984 paper titled simply "Compositing Digital Images": https://graphics.pixar.com/library/Compositing/paper.pdf https://graphics.pixar.com/library/Compositing/paper.pdf This research was sponsored by Lucasfilm because at the time, alpha compositing of bitmap images wasn't really a concern for desktop computers. CPU, memory, and display hardware wasn't there to do anything useful with alpha (other than 1-bit masks used for game characters and GUI icons). OTOH Lucasfilm (and other film visual effects companies) had a long history of using mattes, the physical photography equivalents of alpha compositing. When you see a group of Stormtroopers marching inside a Death Star hangar in 1977 "Star Wars", the hangar is a matte painting. They were typically created by artists directly on a large glass that was carefully placed in front of the camera to create the illusion. Porter and Duff came up with a compositing model that's appealingly simple: it's a single formula, three variables, and you get 12 compositing operators by simply enumerating the combinations. You can still find these Porter-Duff operators pretty much anywhere that 2D graphics are processed. They're in HTML Canvas, PDF, Apple CoreGraphics... But, as neatly mathematical as these operators are, most of them aren't really practical for rendering actual scenes consisting of graphics layers. They're not terribly intuitive for designers, and they don't even attempt to simulate the effects of combining light. So when Photoshop was created, its designers needed a set of blending operations (initially for drawing tools, and soon after for combining layers, which were introduced in PS version 3.0). Exposing the Porter-Duff operators to designers in the Photoshop GUI wouldn't have been very useful. Instead they came up with a set of blending modes that were loosely derived from operations that you could do on physical photography negatives, since that was the inspiration for the application. (The PS blending modes still retain names like "dodge" and "burn" which refer to darkroom techniques that no designer aged under 50 probably remembers.) These PS blending modes are conceptually pretty much the opposite of the Porter-Duff modes: they're not mathematically rigorous, but just a collection of algorithms that could be executed in reasonable time on a 68k processor back in the early 1990s. These original limitations are visible in the design of the algorithms: for example, they don't even try to account for gamma correction in the source images, which means that the algorithms do a rather poor job of simulating the behavior of actual light. (Photoshop has later added "linear" versions of some blending modes to address this.) Still, they are the closest thing we have to a standard for blending modes. Many 2D graphics APIs that originally only supported Porter-Duff have added support for PS modes too, including the aforementioned PDF and CoreGraphics.

fsloth 3y ago
Thank you, I got really confused when the linked page started to discuss Photoshop blending modes and I had never considered they could be mapped directly to Porter-Duff ops. As a graphics programmer I always want to point out that it's good to remember NONE of these are scientifically foundational. These are more or less just hacks that looks nice - pure heuristics that is. They are really pragmatic heuristics that produce nice outcomes.
- atoav 3y ago
  I mean add and multiply pretty much work as expected (and I would say except for overlay, color, saturation and lumiosity they are the only ones you will ever need).
  - Daub 3y ago
    I would add to that list color dodge. Used in the right way, it produces results that mimic natural highlights. A highlight will impact all values except black (I.e. deep shadow) which it will ignore. My teaching notes on color dodge here: https://rmit.instructure.com/courses/87565/modules/items/4213799 https://rmit.instructure.com/courses/87565/modules/items/421... On overlay (a very difficult blend mode to use) here: https://rmit.instructure.com/courses/87565/modules/items/4081525 https://rmit.instructure.com/courses/87565/modules/items/408... And on blnd modes in general here: https://rmit.instructure.com/courses/87565/modules/items/3222630 https://rmit.instructure.com/courses/87565/modules/items/322... Good to hear about porter/duff. Was news to me. The furthest back I go is 'Photoshop Channel Chops' Biedny, D., Monroy, B., & Moody, N. (1996). New Riders Publishing.
- deleted 3y ago
  [deleted]
creata 3y ago
> They're not terribly intuitive for designers, and they don't even attempt to simulate the effects of combining light. Can you expand on this? I thought that to combine the effect of light from different sources, you could just add the intensities together.

mikhailfranco 3y ago

Pixel averages and Porter-Duff alpha blending will always give fuzzy results. You need sub-pixel geometric info to do a more accurate depth blend. The classic reference is the A-Buffer by Loren Carpenter (SIGGRAPH 1984). One interesting possibility is to go from a binary matte or averaged anti-aliased layer to subpixel geometry by generating a contour (e.g. Marching Squares). Then the coarse layers (e.g. green screen) can be incorporated into a high-resolution A-Buffer rendering pipeline with 3D CGI elements.