5 ms·

Revisiting the postcard pathtracer with CUDA and Optix

103 points by Luc 6y ago

K0nserv 6y ago
Denoising is one of those machine learning applications where it just seems to work too well. I was thinking about the idea of using machine learning for denoising a few weeks ago and then I stumbled on Nvidia’s work in the area and was totally blown away. The strategy Nvidia are adopting for their GPU where they utilise both ray tracing and deep learning cores for real-time applications is excellent.
- ladberg 6y ago
  I could be wrong, but I feel like deep learning denoising is mostly just blurring in the right places. Deep learning is good at segmenting each part of the image that's supposed to be a different color, and then denoising can be done by blurring within the segmented regions. I'm very curious how a DNN denoiser compares to DNN segmenter + gaussian blur, or even a classical segmenter + blur.
  - llukas 6y ago
    See comparisons of DLSS 1.0 vs DLSS 2.0. 1.0 seemed to behave like you describe but 2.0 no longer does... https://www.nvidia.com/en-us/geforce/news/nvidia-dlss-2-0-a-big-leap-in-ai-rendering/ https://www.nvidia.com/en-us/geforce/news/nvidia-dlss-2-0-a-...
    - TomVDB 6y ago
      DLSS is for anti-aliasing and sharpening, not for denoising. And the technique is different than fixed frame ray tracing denoising because DLSS makes use of multiple frames and motion vectors.
      - llukas 6y ago
        If a neural network can build an idea of what it looks at over a few frames (or one frame ie. YOLO) then use it to improve quality of an image is it not denoising? Hallucinating extra pixels definitely. Denoising, maybe, I'd say it fits the bill but I'm not a domain expert.
        TomVDB 6y ago
        With a static undersampled ray tracing image, the only thing you're doing is hallucinating extra pixels out of nothing but the surrounding pixels that you already have of the same frame. With DLSS, it's more a matter of creatively combining pixels that actually exist. When you're upscaling two 1080p consecutive images + motion vectors to one 1440p image, the amount of source pixels is higher than the final number of pixels. That said, with some config file hacking, people have been able to DLSS upscale from 540p to 1080p (with still surprisingly good results). In that case, the amount of source pixels is still lower than the number of destination pixels. So my point is only semi-valid...
  - rustybolt 6y ago
    Inigo Quilez has written a tiny bit on something he calls 'variance-based blur' which seems to do something similar. See the last section of https://www.iquilezles.org/www/articles/wavelet/wavelet.htm https://www.iquilezles.org/www/articles/wavelet/wavelet.htm
  - deleted 6y ago
    [deleted]
- Fronzie 6y ago
  It is my understanding that for ray-tracing de-noising, the de-noiser also has access to the noise-free textures (in addition to the noisy raytraced frame) This gives it far more information than typical screenshots hint at.
- thdrdt 6y ago
  I believe Blender also uses this denoiser. It is great! Especially for preview renders. The results are comparible to renders which take around 3 times longer.
xmmrm 6y ago
Surprisingly [sincosf fusion] only happened with -Ofast and not with -O3. As noted, -Ofast turns on -ffast-math which turns on -funsafe-math-optimizations which "enables optimizations that allow arbitrary reassociations and transformations with no accuracy guarantees."[0] In this case, sincosf by itself is probably more accurate. [0] https://gcc.gnu.org/wiki/FloatingPointMath https://gcc.gnu.org/wiki/FloatingPointMath
pjmlp 6y ago
Yep, Optix is so good that OctaneRender has dropped their Vulkan backend and are now using Optix 7 instead. There is a talk from GTC 2020 how they went about it.
spekcular 6y ago
Is Optix significantly better than the traditional (presumably faster) approaches used by photo editing software like Lightroom and Rawtherapee? Denoising is of course an extremely well-studied image processing problem with a highly developed state-of-the-art. I haven't looked at comparison recently, but my recollection is that the answer was "no" as of about a year ago.
- Robadob 6y ago
  The video here talks about the difference between the two approaches. There's a comparison at 2:50. "I think the computers are winning, because in the upper right you can kind of see that back wall, there's a little strip of green. It's kind of blurred out a little bit too much with the human version, but the neural network has picked up on that and kept the vertical stripes there." https://news.developer.nvidia.com/ray-tracing-essentials-part-7-denoising-for-ray-tracing/ https://news.developer.nvidia.com/ray-tracing-essentials-par...
- bayindirh 6y ago
  I didn't try AI denoisers but, Darktable's profiled denoiser works pretty well. The profiles is per camera and per iso, so the filter knows what to look for. The results are much better than advanced but unaware denoisers.

103 points by Luc 6y ago

Denoising is one of those machine learning applications where it just seems to work too well. I was thinking about the idea of using machine learning for denoising a few weeks ago and then I stumbled on Nvidia’s work in the area and was totally blown away. The strategy Nvidia are adopting for their GPU where they utilise both ray tracing and deep learning cores for real-time applications is excellent.

ladberg 6y ago
I could be wrong, but I feel like deep learning denoising is mostly just blurring in the right places. Deep learning is good at segmenting each part of the image that's supposed to be a different color, and then denoising can be done by blurring within the segmented regions. I'm very curious how a DNN denoiser compares to DNN segmenter + gaussian blur, or even a classical segmenter + blur.
- llukas 6y ago
  See comparisons of DLSS 1.0 vs DLSS 2.0. 1.0 seemed to behave like you describe but 2.0 no longer does... https://www.nvidia.com/en-us/geforce/news/nvidia-dlss-2-0-a-big-leap-in-ai-rendering/ https://www.nvidia.com/en-us/geforce/news/nvidia-dlss-2-0-a-...
  - TomVDB 6y ago
    DLSS is for anti-aliasing and sharpening, not for denoising. And the technique is different than fixed frame ray tracing denoising because DLSS makes use of multiple frames and motion vectors.
    - llukas 6y ago
      If a neural network can build an idea of what it looks at over a few frames (or one frame ie. YOLO) then use it to improve quality of an image is it not denoising? Hallucinating extra pixels definitely. Denoising, maybe, I'd say it fits the bill but I'm not a domain expert.
      - TomVDB 6y ago
        With a static undersampled ray tracing image, the only thing you're doing is hallucinating extra pixels out of nothing but the surrounding pixels that you already have of the same frame. With DLSS, it's more a matter of creatively combining pixels that actually exist. When you're upscaling two 1080p consecutive images + motion vectors to one 1440p image, the amount of source pixels is higher than the final number of pixels. That said, with some config file hacking, people have been able to DLSS upscale from 540p to 1080p (with still surprisingly good results). In that case, the amount of source pixels is still lower than the number of destination pixels. So my point is only semi-valid...
- rustybolt 6y ago
  Inigo Quilez has written a tiny bit on something he calls 'variance-based blur' which seems to do something similar. See the last section of https://www.iquilezles.org/www/articles/wavelet/wavelet.htm https://www.iquilezles.org/www/articles/wavelet/wavelet.htm
- deleted 6y ago
  [deleted]
Fronzie 6y ago
It is my understanding that for ray-tracing de-noising, the de-noiser also has access to the noise-free textures (in addition to the noisy raytraced frame) This gives it far more information than typical screenshots hint at.
thdrdt 6y ago
I believe Blender also uses this denoiser. It is great! Especially for preview renders. The results are comparible to renders which take around 3 times longer.

xmmrm 6y ago

Surprisingly [sincosf fusion] only happened with -Ofast and not with -O3. As noted, -Ofast turns on -ffast-math which turns on -funsafe-math-optimizations which "enables optimizations that allow arbitrary reassociations and transformations with no accuracy guarantees."[0] In this case, sincosf by itself is probably more accurate. [0] https://gcc.gnu.org/wiki/FloatingPointMath https://gcc.gnu.org/wiki/FloatingPointMath

pjmlp 6y ago

Yep, Optix is so good that OctaneRender has dropped their Vulkan backend and are now using Optix 7 instead. There is a talk from GTC 2020 how they went about it.

spekcular 6y ago

Is Optix significantly better than the traditional (presumably faster) approaches used by photo editing software like Lightroom and Rawtherapee? Denoising is of course an extremely well-studied image processing problem with a highly developed state-of-the-art. I haven't looked at comparison recently, but my recollection is that the answer was "no" as of about a year ago.

Robadob 6y ago
The video here talks about the difference between the two approaches. There's a comparison at 2:50. "I think the computers are winning, because in the upper right you can kind of see that back wall, there's a little strip of green. It's kind of blurred out a little bit too much with the human version, but the neural network has picked up on that and kept the vertical stripes there." https://news.developer.nvidia.com/ray-tracing-essentials-part-7-denoising-for-ray-tracing/ https://news.developer.nvidia.com/ray-tracing-essentials-par...
bayindirh 6y ago
I didn't try AI denoisers but, Darktable's profiled denoiser works pretty well. The profiles is per camera and per iso, so the filter knows what to look for. The results are much better than advanced but unaware denoisers.