4 ms·

Fundamentals of Radiance Cascades

117 points by ibobev 2y ago

justin66 2y ago
They're waiting for you, Gordon. In the test chamber.
- DaiPlusPlus 2y ago
  > In the test chamber. Flashbacks to sitting the GRE
- rustcleaner 2y ago
  >it's saturday morning 1998 and you are in your coziest sleepwear, on the computer in the living room. I WANT TO GO BAAACK!
- davesque 2y ago
  Exactly what I thought of :).
gclawes 2y ago
Resonance cascade?
- speed_spread 2y ago
  A gamble, but we needed the extra resolution.
wowxserr 2y ago
So on my webcam there is a cover but it doesn't fully cover the webcam. So this technology would be able to infer something from the radiance of the light seeping in around the edges?
- pornel 2y ago
  This is a rendering technique designed for real-time graphics, and it's not applicable to that kind of image analysis. It does what has already been possible with ray tracing, but using an approximation that makes it suitable for real-time graphics. However, the technique has been used to speed up astrophysics calculations: https://arxiv.org/pdf/2408.14425 https://arxiv.org/pdf/2408.14425
Negitivefrags 2y ago
This approach was originally developed by Alex Sanikov for Path of Exile 2. Of course in PoE2 it's used in full 3d. The benefit of this approach is that you get global illumination with a constant cost for the entire scene and because it doesn't use any temporal acculation, it has zero latency as well. This means you can rely on it as the lighting for fast effects. For example: https://www.youtube.com/watch?v=p1SvodIDz6E https://www.youtube.com/watch?v=p1SvodIDz6E There is no traditional "lighting" added to these two effects. The light on the nearby surfaces is indirect light from the GI solution that you get for free by just spawning the particles. That means all effects just naturally emit light with no extra work from artists. On my GPU (which is admittedly a 4090) the GI solution runs in 0.8ms for a 4k scene in "High" quality. This is exactly what it will always cost, no matter what scene you choose.
- stonethrowaway 2y ago
  In the age of PoE is Diablo even relevant anymore?
- isaacimagine 2y ago
  From what I understand, PoE2 has a fixed-perspective camera, so radiance is calculated in screenspace (2D) as an optimization (and not using a 3D texture). It would be interesting to see how the performance of this technique scales to full 3D, as that would be significantly more expensive / require a lower-resolution voxelization of the scene.
  - corysama 2y ago
    From offhand comments I've read, you are right. It's not practical for 3D.
  - Negitivefrags 2y ago
    The source data is indeed from screen space, but this generates a full 3d data structure by ray marching the depth buffer. Its not using the 2D algorithm. In PoE2s case, it doesn’t need to go further since light sources are not usually going to be behind the camera. You can gather data in other ways.
cubefox 2y ago
This article unfortunately presupposes understanding of (ir)radiance probes, a topic on which there isn't even a Wikipedia article...
- stonethrowaway 2y ago
  This is true for quite a few graphic algorithms today. Certain terms in the equations used are pre-computed, and you can generalize all of them in a sense to be “probes”, “queries”, “function inversions”, “look up tables” or whatever. The only real way to know what can be computed on the fly vs what can be stored is to go ahead, try to implement it, think really fucking hard, and then see what you can do in real time. That’s basically how the field has advanced the formulas used into something that can run in real time, or can run in screen space complexity rather than scene complexity, and so on. An equation given to you will be one of 2 forms usually; the raw equation that may come from optics, in which case expect an ass load of calculus and diff equations with arcane terms not seen outside of Maxwell’s equations. Or, in the likely case, the presenter is a PhD ubermensch nerd who has already pulled the terms out, rewritten them, is presenting them, and you need to be really paying close attention to every single word during their talk to figure it out. It’s at your discretion to determine which of the two forms is easier for you to deal with.
arijo 2y ago
I'm trying to understand the basic idea of Radiance Cascades (I don't know much about game development and ray tracing). Is the key idea the fact that light intensity and shadowing require more resolution near the light source and lower resolution far from it? So you have higher probe density nearby the light source and then relax it as distance increases minimising the number of radiance collection points? Also using interpolation eliminates a lot of the calculations. Does this make any sense? I'm sure there's a lot more detail, but I was looking for a bird's eye understanding that I can keep in the back of my mind.
- pornel 2y ago
  Essentially yes. There's ambient occlusion that computes light intensity with high spatial resolution, but completely handwaves the direction the light is coming from. OTOH there are environment maps that are rendered from a single location, so they have no spatial resolution, but have precise light intensity for every angle. Cascade Radiance observes that these two techniques are two extremes of spatial vs angular resolution trade-off, and it's possible to render any spatial vs angular trade-off in between. Getting information about light from all angles at all points would cost (all sample points × all angles), but Radiance Cascades computes and combines (very few sample points × all angles) + (some sample points × some angles) + (all sample points × very few angles), which works out to be much cheaper, and is still sufficient to render shadows accurately if the light sources are not too small.
  - arijo 2y ago
    So I've been reading https://graphicscodex.com/app/app.html https://graphicscodex.com/app/app.html and https://mini.gmshaders.com/p/radiance-cascades https://mini.gmshaders.com/p/radiance-cascades so I could have a basic grasp of classical rendering theory. I made some assumptions: 1. There's an isometric top-down virtual camera just above the player 2. The Radiance Cascades stack on top of each other, incresing probe density as they get closer to the objects and players I suspect part of the increased algorithm efficiency results from: 1. The downsampling of radiance measuring at some of the levels 2. At higher probe density levels, ray tracing to collect radiance measurements involves less computation than classic long path ray tracing But I'm still confused about what exactly in the "virtual 3D world" is being downsampled and what the penumbra theory has to do with all thus. I've gained a huge respect for game developers though - this is not eady stuff to grasp.
ozarker 2y ago
The guy that developed the technique works on Path of Exile. They’re using it for POE2, he gave an awesome talk about it here https://youtu.be/TrHHTQqmAaM?si=xrW0XT2lsGHqUYY_ https://youtu.be/TrHHTQqmAaM?si=xrW0XT2lsGHqUYY_
abetusk 2y ago
Am I correct in thinking that this approach effectively makes real time GI a solved problem? Can someone who's more knowledgeable than myself offer any explanation on why it took so long to discover a method like this? I remember hearing about rt-GI from over 25 years ago.
- pixelpoet 2y ago
  It does not make realtime GI a solved problem.
  - abetusk 2y ago
    Could you go into more detail?
ezcrypt 2y ago
I also recommend the video by SimonDev: Exploring a New Approach to Realistic Lighting: Radiance Cascades <https://www.youtube.com/watch?v=3so7xdZHKxw https://www.youtube.com/watch?v=3so7xdZHKxw> (I recently discovered his channel, and I like his calm down-to-earth explanations of game dev techniques).

117 points by ibobev 2y ago

justin66 2y ago

They're waiting for you, Gordon. In the test chamber.

DaiPlusPlus 2y ago
> In the test chamber. Flashbacks to sitting the GRE
rustcleaner 2y ago
>it's saturday morning 1998 and you are in your coziest sleepwear, on the computer in the living room. I WANT TO GO BAAACK!
davesque 2y ago
Exactly what I thought of :).

gclawes 2y ago

Resonance cascade?

speed_spread 2y ago
A gamble, but we needed the extra resolution.

wowxserr 2y ago

So on my webcam there is a cover but it doesn't fully cover the webcam. So this technology would be able to infer something from the radiance of the light seeping in around the edges?

pornel 2y ago
This is a rendering technique designed for real-time graphics, and it's not applicable to that kind of image analysis. It does what has already been possible with ray tracing, but using an approximation that makes it suitable for real-time graphics. However, the technique has been used to speed up astrophysics calculations: https://arxiv.org/pdf/2408.14425 https://arxiv.org/pdf/2408.14425

Negitivefrags 2y ago

This approach was originally developed by Alex Sanikov for Path of Exile 2. Of course in PoE2 it's used in full 3d. The benefit of this approach is that you get global illumination with a constant cost for the entire scene and because it doesn't use any temporal acculation, it has zero latency as well. This means you can rely on it as the lighting for fast effects. For example: https://www.youtube.com/watch?v=p1SvodIDz6E https://www.youtube.com/watch?v=p1SvodIDz6E There is no traditional "lighting" added to these two effects. The light on the nearby surfaces is indirect light from the GI solution that you get for free by just spawning the particles. That means all effects just naturally emit light with no extra work from artists. On my GPU (which is admittedly a 4090) the GI solution runs in 0.8ms for a 4k scene in "High" quality. This is exactly what it will always cost, no matter what scene you choose.

stonethrowaway 2y ago
In the age of PoE is Diablo even relevant anymore?
isaacimagine 2y ago
From what I understand, PoE2 has a fixed-perspective camera, so radiance is calculated in screenspace (2D) as an optimization (and not using a 3D texture). It would be interesting to see how the performance of this technique scales to full 3D, as that would be significantly more expensive / require a lower-resolution voxelization of the scene.
- corysama 2y ago
  From offhand comments I've read, you are right. It's not practical for 3D.
- Negitivefrags 2y ago
  The source data is indeed from screen space, but this generates a full 3d data structure by ray marching the depth buffer. Its not using the 2D algorithm. In PoE2s case, it doesn’t need to go further since light sources are not usually going to be behind the camera. You can gather data in other ways.

cubefox 2y ago

This article unfortunately presupposes understanding of (ir)radiance probes, a topic on which there isn't even a Wikipedia article...

stonethrowaway 2y ago
This is true for quite a few graphic algorithms today. Certain terms in the equations used are pre-computed, and you can generalize all of them in a sense to be “probes”, “queries”, “function inversions”, “look up tables” or whatever. The only real way to know what can be computed on the fly vs what can be stored is to go ahead, try to implement it, think really fucking hard, and then see what you can do in real time. That’s basically how the field has advanced the formulas used into something that can run in real time, or can run in screen space complexity rather than scene complexity, and so on. An equation given to you will be one of 2 forms usually; the raw equation that may come from optics, in which case expect an ass load of calculus and diff equations with arcane terms not seen outside of Maxwell’s equations. Or, in the likely case, the presenter is a PhD ubermensch nerd who has already pulled the terms out, rewritten them, is presenting them, and you need to be really paying close attention to every single word during their talk to figure it out. It’s at your discretion to determine which of the two forms is easier for you to deal with.

arijo 2y ago

I'm trying to understand the basic idea of Radiance Cascades (I don't know much about game development and ray tracing). Is the key idea the fact that light intensity and shadowing require more resolution near the light source and lower resolution far from it? So you have higher probe density nearby the light source and then relax it as distance increases minimising the number of radiance collection points? Also using interpolation eliminates a lot of the calculations. Does this make any sense? I'm sure there's a lot more detail, but I was looking for a bird's eye understanding that I can keep in the back of my mind.

pornel 2y ago
Essentially yes. There's ambient occlusion that computes light intensity with high spatial resolution, but completely handwaves the direction the light is coming from. OTOH there are environment maps that are rendered from a single location, so they have no spatial resolution, but have precise light intensity for every angle. Cascade Radiance observes that these two techniques are two extremes of spatial vs angular resolution trade-off, and it's possible to render any spatial vs angular trade-off in between. Getting information about light from all angles at all points would cost (all sample points × all angles), but Radiance Cascades computes and combines (very few sample points × all angles) + (some sample points × some angles) + (all sample points × very few angles), which works out to be much cheaper, and is still sufficient to render shadows accurately if the light sources are not too small.
- arijo 2y ago
  So I've been reading https://graphicscodex.com/app/app.html https://graphicscodex.com/app/app.html and https://mini.gmshaders.com/p/radiance-cascades https://mini.gmshaders.com/p/radiance-cascades so I could have a basic grasp of classical rendering theory. I made some assumptions: 1. There's an isometric top-down virtual camera just above the player 2. The Radiance Cascades stack on top of each other, incresing probe density as they get closer to the objects and players I suspect part of the increased algorithm efficiency results from: 1. The downsampling of radiance measuring at some of the levels 2. At higher probe density levels, ray tracing to collect radiance measurements involves less computation than classic long path ray tracing But I'm still confused about what exactly in the "virtual 3D world" is being downsampled and what the penumbra theory has to do with all thus. I've gained a huge respect for game developers though - this is not eady stuff to grasp.

ozarker 2y ago

The guy that developed the technique works on Path of Exile. They’re using it for POE2, he gave an awesome talk about it here https://youtu.be/TrHHTQqmAaM?si=xrW0XT2lsGHqUYY_ https://youtu.be/TrHHTQqmAaM?si=xrW0XT2lsGHqUYY_

abetusk 2y ago

Am I correct in thinking that this approach effectively makes real time GI a solved problem? Can someone who's more knowledgeable than myself offer any explanation on why it took so long to discover a method like this? I remember hearing about rt-GI from over 25 years ago.

pixelpoet 2y ago
It does not make realtime GI a solved problem.
- abetusk 2y ago
  Could you go into more detail?

ezcrypt 2y ago

I also recommend the video by SimonDev: Exploring a New Approach to Realistic Lighting: Radiance Cascades <https://www.youtube.com/watch?v=3so7xdZHKxw https://www.youtube.com/watch?v=3so7xdZHKxw> (I recently discovered his channel, and I like his calm down-to-earth explanations of game dev techniques).