4 ms·

Can someone explain why they can't apply the same reconstruction technique to the data that would be captured without a diffuser; i.e. why the diffuser is requi

by mrow84 9y ago

Can someone explain why they can't apply the same reconstruction technique to the data that would be captured without a diffuser; i.e. why the diffuser is required?

icegreentea2 9y ago
I think the addition of the diffuser adds the 'depth response' to the camera. Imagine that you had two point light sources located some distance away from the sensor such that one is directly in front of the other. They're magical point light sources so the light from the rear one just passes through the front. If you didn't have anything in front of the sensor, there would be no way to distinguish that there are two point sources, as opposed to a single point source with a non uniform output.
radarsat1 9y ago
I think the diffuser ensures that there is a random distribution of incidence angle of photons. Then, if I understand, an optimisation is basically used to figure out what pixels correspond to what angles. I think the actual angle calculation is skipped and the reconstruction is done directly, but the "angle" concept is implicitly taken into account by the fact that it is needed for the 3D reconstruction to take place.
- hammock 9y ago
  In essence, the diffuser is the lens (arranging the light in a usable way onto a sensor), just not a lens as we know it.
frumiousirc 9y ago
It's a good question and I think strikes at the heart of the idea. A lens transforms a family of rays to a pixel location. Given knowledge of that pixel's intensity there is a degenerate solution for the original ray (in terms of it's location and direction at some plane). This degeneracy is one thing that leads to blurry photos. The micro lens camera referenced in the article spreads this family over more pixels in a known, analytic way to make the solution more unique. In principle it suffers from the same degeneracy but each micro lens limits the possible location of rays so if any of the pixels under it are hit then the general location is set and the exact pixel in the group determines the direction. The diffuser works similarly but spreads direction and position location over many pixels and in a random way (seeded by the material and it's precise placement). While this spread can not be calculated it can be discovered through calibration with known point sources. In both these latter cases one inverts this analytic or calibrated ray->pixel matrix and applies that to the measured pixels to reconstruct the rays that have likely caused the measurement. In the case of the diffuse "lens", the required matrix inversion can be computationally expensive at best and impossible at worse. However, the methods of compressed sensing (in particular L1 regularization) allow an approximate inversion to be done in a relatively fast manner.
- mrow84 9y ago
  Ok, thank you. As I understand what you have written, the diffuser does as its name suggests, and spreads light rays over more pixels than they would otherwise have struck, making it easier to construct the pixel -> ray function. As a matter of interest, do you think it would still be possible to apply the technique without the diffuser, presumably obtaining a lower-fidelity reconstruction, by leaning more heavily on the regularisation?
  - chowells 9y ago
    Without the diffuser, the only information you have is roughly "a light ray hit the sensor at location (x, y)". You can't derive from that information what direction the photon hit the sensor from. This technique gives you "a light ray hit the sensor at locations (x1, y1) through (xn, yn)". You can deconvolve that list to get an approximate vector the ray hit the diffuser at. Obviously there's a lot of calculation involved to apply this deconvolution over the entire image at once, but it's the same thing light field cameras have been doing for a while. The innovative bit here is working with a random diffuser, rather than a very precise lens configuration.
    - mrow84 9y ago
      Ah yes, of course, I see what you mean. Presumably there is some minimal number of pixels required to stand any chance of resolving the orientation of a particular bundle of light rays (I would imagine 3)? Also, would it be true to say that the more pixels you manage to spread a given ray bundle over, the better the reconstruction, and that the main trade-off is between the accuracy and the density of the reconstructed ray bundles, for a fixed number of pixels?
      - chowells 9y ago
        I have to admit you're moving past my level of knowledge on the topic. Both of your suppositions seem likely correct, but my understanding of the calculation technique involved is superficial at best.
lawall04 9y ago
you could try but the amount of angle information in the data will be so small that noise will destroy the results. Said another way, the diffuser makes the forward model more invertible (better condition number).

by mrow84 9y ago

Can someone explain why they can't apply the same reconstruction technique to the data that would be captured without a diffuser; i.e. why the diffuser is required?

icegreentea2 9y ago

I think the addition of the diffuser adds the 'depth response' to the camera. Imagine that you had two point light sources located some distance away from the sensor such that one is directly in front of the other. They're magical point light sources so the light from the rear one just passes through the front. If you didn't have anything in front of the sensor, there would be no way to distinguish that there are two point sources, as opposed to a single point source with a non uniform output.

radarsat1 9y ago

I think the diffuser ensures that there is a random distribution of incidence angle of photons. Then, if I understand, an optimisation is basically used to figure out what pixels correspond to what angles. I think the actual angle calculation is skipped and the reconstruction is done directly, but the "angle" concept is implicitly taken into account by the fact that it is needed for the 3D reconstruction to take place.

hammock 9y ago
In essence, the diffuser is the lens (arranging the light in a usable way onto a sensor), just not a lens as we know it.

frumiousirc 9y ago

It's a good question and I think strikes at the heart of the idea. A lens transforms a family of rays to a pixel location. Given knowledge of that pixel's intensity there is a degenerate solution for the original ray (in terms of it's location and direction at some plane). This degeneracy is one thing that leads to blurry photos. The micro lens camera referenced in the article spreads this family over more pixels in a known, analytic way to make the solution more unique. In principle it suffers from the same degeneracy but each micro lens limits the possible location of rays so if any of the pixels under it are hit then the general location is set and the exact pixel in the group determines the direction. The diffuser works similarly but spreads direction and position location over many pixels and in a random way (seeded by the material and it's precise placement). While this spread can not be calculated it can be discovered through calibration with known point sources. In both these latter cases one inverts this analytic or calibrated ray->pixel matrix and applies that to the measured pixels to reconstruct the rays that have likely caused the measurement. In the case of the diffuse "lens", the required matrix inversion can be computationally expensive at best and impossible at worse. However, the methods of compressed sensing (in particular L1 regularization) allow an approximate inversion to be done in a relatively fast manner.

mrow84 9y ago
Ok, thank you. As I understand what you have written, the diffuser does as its name suggests, and spreads light rays over more pixels than they would otherwise have struck, making it easier to construct the pixel -> ray function. As a matter of interest, do you think it would still be possible to apply the technique without the diffuser, presumably obtaining a lower-fidelity reconstruction, by leaning more heavily on the regularisation?
- chowells 9y ago
  Without the diffuser, the only information you have is roughly "a light ray hit the sensor at location (x, y)". You can't derive from that information what direction the photon hit the sensor from. This technique gives you "a light ray hit the sensor at locations (x1, y1) through (xn, yn)". You can deconvolve that list to get an approximate vector the ray hit the diffuser at. Obviously there's a lot of calculation involved to apply this deconvolution over the entire image at once, but it's the same thing light field cameras have been doing for a while. The innovative bit here is working with a random diffuser, rather than a very precise lens configuration.
  - mrow84 9y ago
    Ah yes, of course, I see what you mean. Presumably there is some minimal number of pixels required to stand any chance of resolving the orientation of a particular bundle of light rays (I would imagine 3)? Also, would it be true to say that the more pixels you manage to spread a given ray bundle over, the better the reconstruction, and that the main trade-off is between the accuracy and the density of the reconstructed ray bundles, for a fixed number of pixels?
    - chowells 9y ago
      I have to admit you're moving past my level of knowledge on the topic. Both of your suppositions seem likely correct, but my understanding of the calculation technique involved is superficial at best.

lawall04 9y ago

you could try but the amount of angle information in the data will be so small that noise will destroy the results. Said another way, the diffuser makes the forward model more invertible (better condition number).