3 ms·

Are these algorithms applicable to low light photography?

by nitrogen 8y ago

AstralStorm 8y ago
Approach yes, exact algorithms no.
- alkonaut 8y ago
  What is the difference that makes the algorithm not be applicable? Is it that path tracing noise is "pure monte carlo noise" (I'm sure that has a better term) so has nicer properties from photographic sensor noise (thermal noise with various patterns etc)?
  - BubRoss 8y ago
    There are two huge factors, the first even more potent than the second. First, there is much more information per sample and per pixel. Normals, depth, separate diffuse color, separate diffuse texture, first bounce normal, first bounce position, etc. There is a technique that just uses color, but uses histograms of all the samples within a pixel. Second, for a video game you can accumulate samples temporally. Many of these demos will use one sample per pixel, but filtering information can be kept on every frame and reused to some degree.
    - alkonaut 8y ago
      Oh I thought this operated on a 2D hdr image, without further info (I admit I didn’t read the details). If it can’t be used on a still I rendered 10 years ago with code that isn’t around anymore then it’s clear it can be used on a photo either.
dharma1 8y ago
This is trained on monte carlo based rendering noise, and optionally uses additional available rendering data like albedo and normal passes. So while the idea of using deep learning for noise reduction in general works, this codebase in particular isn't suitable for that task. One issue is also building large enough training data set for clean vs noisy images. Approaches like https://arxiv.org/abs/1803.04189 https://arxiv.org/abs/1803.04189 try to work around that Other approaches (like Google's pixel cameras and astrophotography) additionally take high speed burst photos and average pixels to reduce noise, this can be used together with deep learning based noise reduction (or other noise reduction techniques) for better results

Are these algorithms applicable to low light photography?

AstralStorm 8y ago

Approach yes, exact algorithms no.

alkonaut 8y ago
What is the difference that makes the algorithm not be applicable? Is it that path tracing noise is "pure monte carlo noise" (I'm sure that has a better term) so has nicer properties from photographic sensor noise (thermal noise with various patterns etc)?
- BubRoss 8y ago
  There are two huge factors, the first even more potent than the second. First, there is much more information per sample and per pixel. Normals, depth, separate diffuse color, separate diffuse texture, first bounce normal, first bounce position, etc. There is a technique that just uses color, but uses histograms of all the samples within a pixel. Second, for a video game you can accumulate samples temporally. Many of these demos will use one sample per pixel, but filtering information can be kept on every frame and reused to some degree.
  - alkonaut 8y ago
    Oh I thought this operated on a 2D hdr image, without further info (I admit I didn’t read the details). If it can’t be used on a still I rendered 10 years ago with code that isn’t around anymore then it’s clear it can be used on a photo either.

dharma1 8y ago

This is trained on monte carlo based rendering noise, and optionally uses additional available rendering data like albedo and normal passes. So while the idea of using deep learning for noise reduction in general works, this codebase in particular isn't suitable for that task. One issue is also building large enough training data set for clean vs noisy images. Approaches like https://arxiv.org/abs/1803.04189 https://arxiv.org/abs/1803.04189 try to work around that Other approaches (like Google's pixel cameras and astrophotography) additionally take high speed burst photos and average pixels to reduce noise, this can be used together with deep learning based noise reduction (or other noise reduction techniques) for better results