5 ms·

It's a good question and I think strikes at the heart of the idea. A lens transforms a family of rays to a pixel location. Given knowledge of that pixel's int

It's a good question and I think strikes at the heart of the idea. A lens transforms a family of rays to a pixel location. Given knowledge of that pixel's intensity there is a degenerate solution for the original ray (in terms of it's location and direction at some plane). This degeneracy is one thing that leads to blurry photos. The micro lens camera referenced in the article spreads this family over more pixels in a known, analytic way to make the solution more unique. In principle it suffers from the same degeneracy but each micro lens limits the possible location of rays so if any of the pixels under it are hit then the general location is set and the exact pixel in the group determines the direction. The diffuser works similarly but spreads direction and position location over many pixels and in a random way (seeded by the material and it's precise placement). While this spread can not be calculated it can be discovered through calibration with known point sources. In both these latter cases one inverts this analytic or calibrated ray->pixel matrix and applies that to the measured pixels to reconstruct the rays that have likely caused the measurement. In the case of the diffuse "lens", the required matrix inversion can be computationally expensive at best and impossible at worse. However, the methods of compressed sensing (in particular L1 regularization) allow an approximate inversion to be done in a relatively fast manner.

mrow84 9y ago
Ok, thank you. As I understand what you have written, the diffuser does as its name suggests, and spreads light rays over more pixels than they would otherwise have struck, making it easier to construct the pixel -> ray function. As a matter of interest, do you think it would still be possible to apply the technique without the diffuser, presumably obtaining a lower-fidelity reconstruction, by leaning more heavily on the regularisation?
- chowells 9y ago
  Without the diffuser, the only information you have is roughly "a light ray hit the sensor at location (x, y)". You can't derive from that information what direction the photon hit the sensor from. This technique gives you "a light ray hit the sensor at locations (x1, y1) through (xn, yn)". You can deconvolve that list to get an approximate vector the ray hit the diffuser at. Obviously there's a lot of calculation involved to apply this deconvolution over the entire image at once, but it's the same thing light field cameras have been doing for a while. The innovative bit here is working with a random diffuser, rather than a very precise lens configuration.
  - mrow84 9y ago
    Ah yes, of course, I see what you mean. Presumably there is some minimal number of pixels required to stand any chance of resolving the orientation of a particular bundle of light rays (I would imagine 3)? Also, would it be true to say that the more pixels you manage to spread a given ray bundle over, the better the reconstruction, and that the main trade-off is between the accuracy and the density of the reconstructed ray bundles, for a fixed number of pixels?
    - chowells 9y ago
      I have to admit you're moving past my level of knowledge on the topic. Both of your suppositions seem likely correct, but my understanding of the calculation technique involved is superficial at best.
      - mrow84 9y ago
        Ok no worries - thanks for helping me to understand what's going on.
frozenport 9y ago
Invertibility is not trivial, and because full-field imaging is typically shift-variant this is a hard sell compared to Hartmann mask ( or lenselets) approaches which have none of the calibration steps. I work in this field. I think they are motivated by novelty.
- lawall04 9y ago
  Actually, lenslets/Hartmann mask are not shift-invariant, and getting depth info (3D) requires loss of shift invariance (see http://web.mit.edu/2.717/www/stein.pdf http://web.mit.edu/2.717/www/stein.pdf). Lenslet-based light field cameras also have very extensive calibration routines to align the lenslets on a per-pixel basis.
  - frozenport 9y ago
    You can either build a system where the field lands on the detector in a known way, or build one where it lands in an unknown way and try to figure out what the heck happened. In light of many functional designs (for example, Phasics, Zygo) I don't see why your group introduces the unknown factor. Could this have been done by a camera from Phasics?

mrow84 9y ago

Ok, thank you. As I understand what you have written, the diffuser does as its name suggests, and spreads light rays over more pixels than they would otherwise have struck, making it easier to construct the pixel -> ray function. As a matter of interest, do you think it would still be possible to apply the technique without the diffuser, presumably obtaining a lower-fidelity reconstruction, by leaning more heavily on the regularisation?

chowells 9y ago
Without the diffuser, the only information you have is roughly "a light ray hit the sensor at location (x, y)". You can't derive from that information what direction the photon hit the sensor from. This technique gives you "a light ray hit the sensor at locations (x1, y1) through (xn, yn)". You can deconvolve that list to get an approximate vector the ray hit the diffuser at. Obviously there's a lot of calculation involved to apply this deconvolution over the entire image at once, but it's the same thing light field cameras have been doing for a while. The innovative bit here is working with a random diffuser, rather than a very precise lens configuration.
- mrow84 9y ago
  Ah yes, of course, I see what you mean. Presumably there is some minimal number of pixels required to stand any chance of resolving the orientation of a particular bundle of light rays (I would imagine 3)? Also, would it be true to say that the more pixels you manage to spread a given ray bundle over, the better the reconstruction, and that the main trade-off is between the accuracy and the density of the reconstructed ray bundles, for a fixed number of pixels?
  - chowells 9y ago
    I have to admit you're moving past my level of knowledge on the topic. Both of your suppositions seem likely correct, but my understanding of the calculation technique involved is superficial at best.
    - mrow84 9y ago
      Ok no worries - thanks for helping me to understand what's going on.

frozenport 9y ago

Invertibility is not trivial, and because full-field imaging is typically shift-variant this is a hard sell compared to Hartmann mask ( or lenselets) approaches which have none of the calibration steps. I work in this field. I think they are motivated by novelty.

lawall04 9y ago
Actually, lenslets/Hartmann mask are not shift-invariant, and getting depth info (3D) requires loss of shift invariance (see http://web.mit.edu/2.717/www/stein.pdf http://web.mit.edu/2.717/www/stein.pdf). Lenslet-based light field cameras also have very extensive calibration routines to align the lenslets on a per-pixel basis.
- frozenport 9y ago
  You can either build a system where the field lands on the detector in a known way, or build one where it lands in an unknown way and try to figure out what the heck happened. In light of many functional designs (for example, Phasics, Zygo) I don't see why your group introduces the unknown factor. Could this have been done by a camera from Phasics?