4 ms·
Typical rasterized pipelines with deferred rendering: 1. For each light, setup a virtual camera at the light and 1a. For each mesh, rasterize depth to a text
by jms55 2y ago
Typical rasterized pipelines with deferred rendering:
1. For each light, setup a virtual camera at the light and
1a. For each mesh, rasterize depth to a texture for that light (the shadow map)
2. For each mesh, rasterize material properties and depth to a texture for the main camera
3. For each pixel on screen
3a. For each light in the pixel's bin: sample the shadow map several times to determine if the pixel is in the light's shadow, and if not, apply the lighting for that light
By contrast, megalights:
1. For each mesh, rasterize material properties and depth to a texture for the main camera
2. For each pixel, pick a random light influenced by how close it is, how bright it is, etc
3. For each pixel, raytrace to the light. If you hit another object before the light, then you're in shadow. If not, the light is visible and affecting the point, so write the light to the direct light texture.
4. For each pixel, denoise the direct light texture by essentially blending spatially (with nearby pixels) and temporally (with pixels from the previous frame's direct light texture).
5. For each pixel, sampling the material properties and the denoised direct lighting texture, apply the lighting to the pixel and write out the final result.
The rasterized method is:
* Expensive - having to culling + rasterize every mesh per light means you can only have a few shadow casting lights
* Expensive a second time - Each pixel needs to loop over all the lights in its bin
* Expensive yet again - It's a lot of memory usage to store all those shadow map textures
* Expensive a fourth time - To get good results, you need multiple samples of the shadow map per pixel, which is a lot of texture reads.
* Poor quality - Shadow maps are a discrete approximation of shadows. Even with multiple samples, you'll often have poor quality and even artifacts unless you have either artists or a very complicated system to tweak shadow biases and cascades.
Megalights (stochastic light sampling), by contrast:
* Have a higher base overhead - ray tracing is expensive!
* Scale much better - With a fixed number of rays per pixel (usually 0.5, 1, or 2), the time and memory cost at a given resolution depends mostly on the BVH complexity used for accelerating raytraces (if the random sampling is done correctly, Nvidia research shows that this can be surprisingly slow if you're not careful). Forget having only a couple shadow casting lights, now you can have thousands!
* Looks much better, with much less work - No more adjusting biases or cascades or shadow sampling patterns. There's no need since raytracing is a fully continuous representation of visibiliyy between two points, unlike shadow maps. You can also integrate proper transparencies, refractions, reflections, volumetrics, emissive meshes or textures, IES light profiles, etc with raytracing, unlike rasterization.
* Has different failure modes - With rasterization, too many lights or too complicated lighting will kill performance, but not quality. With megalights, performance will be fine, but you'll end up with too much noise for the denoiser to handle, which will look bad.
- zigzag312 2y agoThank you for this wonderful comparison! With megalights, could you for each pixel, pick multiple random lights to increase accuracy?
- jms55 2y agoJust realized my comment's formatting got completely messed up. I can't seem to fix it, but hope it's readable enough. > With megalights, could you for each pixel, pick multiple random lights to increase accuracy? You could, and megalights probably does. That's what I meant by 0.5-2 rays. Each ray you pick a different light source. The problem is that you usually needs thousands of rays for good results in a single frame, and even a single ray more is very expensive. Hence the reason people go as low as 0.5 per pixel (i.e. rendering at half resolution and then upscaling), and rely so much on a spatiotemporal denoiser, among other techniques.