4 ms·

Thanks! I see what you're getting-at and mostly agree but I think it really would be helpful to mention a bit more about the actual pixel-production side of thi

Thanks! I see what you're getting-at and mostly agree but I think it really would be helpful to mention a bit more about the actual pixel-production side of things in a document like this, to help make it a more complete picture of the whole rendering process. For instance, just shooting N-samples-per-pixel and then averaging them, is not what most 'Production'-renderers used in VFX actually get up to. There's also the 'Pixel Filter' (Box, Gaussian, Catmull-rom, Sinc, Nearest, Farthest, Highest-coverage etc) that the samples have to go through too! And I think it's useful and interesting to cover some aspects of that, even if it's not a full-on signal-processing section.

dahart 2y ago
It is a very useful topic for a graphics degree or someone writing a production renderer, I agree with that, but it’s obvious that Morgan made a conscious decision here, and obvious that the target audience of this book is not people writing production renderers (yet). Morgan knows all about pixel filters and antialiasing techniques. He’s an insanely knowledgeable and smart person, and he gathered a ton of feedback from a lot of other insanely knowledgeable and smart people. (FWIW I also know about filtering in production, and have worked in VFX and CG films, written parts of production renderers, and I still agree with Morgan’s choice.) It’s not possible to get a complete picture of production renderers in one book, that’s a big broad topic that takes years. This is especially true for an introductory book. PBRT is the closest thing we have, and it’s not an intro book, and it still leaves out lots of things production people deal with. What you can’t see easily is what tradeoffs Morgan made in the Graphics Codex. What subjects here might have been dropped if he’d included signal processing? The truth is that everything covered in this book has an advanced side to it that wasn’t included. While filtering might be your favorite topic, someone else might say that Bidirectional & Metropolis path tracing really should have been included, someone else might insist it should cover rasterizing, or Radiosity, or CIE XYZ color response functions. There’s another thing production renderers do that isn’t covered here: color management. But, simply using sRGB gets you 90% of the way there. Box filtering via averaging N samples over a pixel gets you at least 80% or 90% of the way there, and it’s absolutely trivial to explain and implement. That other ten or twenty percent only comes up in relatively rare cases - if you don’t have any high frequency geometry or textures, Box filtering is “good enough”. Adding a Gaussian or any other pixel filter takes a lot more time to explain, requires an invasive refactor of the main render loop, still only gets you to 95% or 98%, and then leads you into a place where it’s not obvious what the best choice is or even how to find it… even production studios don’t know, and they spend time evaluating and re-evaluating pixel and texture filters. Does that make sense? I think it’s fair to ask the question and think about why pixel filters weren’t included here. I bet Morgan would happily answer the question via email. But I also think coming to a conclusion this particular book should cover signal processing is making premature assumptions. The book is meant to give a solid, practical, and easy introduction, to college students, not to ensure the reader is an expert and production ready after 1 book, right? These days it’s worth noting that production renderers are all introducing denoisers, which is eating into pixel filter territory. It’s not obvious or a given that production renderers are going to need to continue to worry about pixel filtering in the future. On the real-time side, DLSS for example, is amazing at producing nicely filtered antialiased images, both pixels and texels, without needing to implement either.
- danwills 2y ago
  Thanks heaps dahart for your extremely thoughtful reply! I totally see what you're getting at and I feel silly for having even accidentally suggested that Morgan was not across anything tbh! I still think that even an extremely cursory mention about how to make pixels like 'We will average samples from N rays within each pixel as our antialiasing strategy, this is equivalent to box-filtering (see citations [X][Y][Z])" I think something like that would be pretty helpful just to connect the dots between the amazing ideas in the book and actual-pixel-production. (Filtering/signal-processing is not totally a pet-favorite idea btw, I just think it's good to have a mention of it so that it's at least connected with the rest of a graphics-HOWTO in a non-abstract way) Very interesting point about AI-based denoisers too.. definitely changes the pixel-filtering situation in an interesting way! Although I'm super interested in it I'm sticking my fingers in my ears about that for the moment!
  - dahart 2y ago
    Fair point, you’re right it could/should mention box filters somewhere. It has the let’s average samples over a pixel part, and it has decent citations (I noticed Alvy Ray Smith’s “A Pixel is Not a Little Square”). Connecting the averaging to basic terminology is a completely reasonable thing to do. Apologies I didn’t mean to suggest filtering is your pet topic, filtering is very important (and fun & fascinating), mandatory for a graphics education, I only wanted to point out there are other equally deserving topics for an intro book/course. I was lucky enough to take graphics from Peter Shirley, and the first two topics he covered were filtering and colors, in depth. I’d never truly understood the shortcomings of box filtering until he had us rendering a 2d sinc function and try to get rid of any hint of aliasing with a million samples per pixel. That assignment made it really tangible and visceral, gave real practical intuition for filtering that stuck with me. Definitely start playing with denoisers. There’s a lot to do and it needs people who care about filtering to scrutinize it.

dahart 2y ago

It is a very useful topic for a graphics degree or someone writing a production renderer, I agree with that, but it’s obvious that Morgan made a conscious decision here, and obvious that the target audience of this book is not people writing production renderers (yet). Morgan knows all about pixel filters and antialiasing techniques. He’s an insanely knowledgeable and smart person, and he gathered a ton of feedback from a lot of other insanely knowledgeable and smart people. (FWIW I also know about filtering in production, and have worked in VFX and CG films, written parts of production renderers, and I still agree with Morgan’s choice.) It’s not possible to get a complete picture of production renderers in one book, that’s a big broad topic that takes years. This is especially true for an introductory book. PBRT is the closest thing we have, and it’s not an intro book, and it still leaves out lots of things production people deal with. What you can’t see easily is what tradeoffs Morgan made in the Graphics Codex. What subjects here might have been dropped if he’d included signal processing? The truth is that everything covered in this book has an advanced side to it that wasn’t included. While filtering might be your favorite topic, someone else might say that Bidirectional & Metropolis path tracing really should have been included, someone else might insist it should cover rasterizing, or Radiosity, or CIE XYZ color response functions. There’s another thing production renderers do that isn’t covered here: color management. But, simply using sRGB gets you 90% of the way there. Box filtering via averaging N samples over a pixel gets you at least 80% or 90% of the way there, and it’s absolutely trivial to explain and implement. That other ten or twenty percent only comes up in relatively rare cases - if you don’t have any high frequency geometry or textures, Box filtering is “good enough”. Adding a Gaussian or any other pixel filter takes a lot more time to explain, requires an invasive refactor of the main render loop, still only gets you to 95% or 98%, and then leads you into a place where it’s not obvious what the best choice is or even how to find it… even production studios don’t know, and they spend time evaluating and re-evaluating pixel and texture filters. Does that make sense? I think it’s fair to ask the question and think about why pixel filters weren’t included here. I bet Morgan would happily answer the question via email. But I also think coming to a conclusion this particular book should cover signal processing is making premature assumptions. The book is meant to give a solid, practical, and easy introduction, to college students, not to ensure the reader is an expert and production ready after 1 book, right? These days it’s worth noting that production renderers are all introducing denoisers, which is eating into pixel filter territory. It’s not obvious or a given that production renderers are going to need to continue to worry about pixel filtering in the future. On the real-time side, DLSS for example, is amazing at producing nicely filtered antialiased images, both pixels and texels, without needing to implement either.

danwills 2y ago
Thanks heaps dahart for your extremely thoughtful reply! I totally see what you're getting at and I feel silly for having even accidentally suggested that Morgan was not across anything tbh! I still think that even an extremely cursory mention about how to make pixels like 'We will average samples from N rays within each pixel as our antialiasing strategy, this is equivalent to box-filtering (see citations [X][Y][Z])" I think something like that would be pretty helpful just to connect the dots between the amazing ideas in the book and actual-pixel-production. (Filtering/signal-processing is not totally a pet-favorite idea btw, I just think it's good to have a mention of it so that it's at least connected with the rest of a graphics-HOWTO in a non-abstract way) Very interesting point about AI-based denoisers too.. definitely changes the pixel-filtering situation in an interesting way! Although I'm super interested in it I'm sticking my fingers in my ears about that for the moment!
- dahart 2y ago
  Fair point, you’re right it could/should mention box filters somewhere. It has the let’s average samples over a pixel part, and it has decent citations (I noticed Alvy Ray Smith’s “A Pixel is Not a Little Square”). Connecting the averaging to basic terminology is a completely reasonable thing to do. Apologies I didn’t mean to suggest filtering is your pet topic, filtering is very important (and fun & fascinating), mandatory for a graphics education, I only wanted to point out there are other equally deserving topics for an intro book/course. I was lucky enough to take graphics from Peter Shirley, and the first two topics he covered were filtering and colors, in depth. I’d never truly understood the shortcomings of box filtering until he had us rendering a 2d sinc function and try to get rid of any hint of aliasing with a million samples per pixel. That assignment made it really tangible and visceral, gave real practical intuition for filtering that stuck with me. Definitely start playing with denoisers. There’s a lot to do and it needs people who care about filtering to scrutinize it.