3 ms·

Even after reading Wikipedia, I'm not sure I understand what a light field is. But this looks pretty interesting.

lazide 5y ago
You can think of a normal lense like an extremely specific filter. It ensures light from a specific direction, and that direction only, arrives at the sensor. Some also ensure only light oscillating in a specific way goes through (polarization), or of a specific frequency (with color filters). This filtering is necessary for us to produce useful data from the total chaos that is the actual light field - aka all the light bouncing and going in all directions at once - where ‘us’ are our normal human brains. Other ways of handling it though if you can use different types of information processing.
ReactiveJelly 5y ago
An ideal camera captures the light rays passing through a pinhole. It generates a 2D image where each pixel corresponds to a ray, and the X and Y coords determine the direction of the ray. Because every ray passes through the same pinhole point, you can't do things like change the depth-of-field or move the camera once the image is created. (An ideal pinhole camera has everything in focus) A light-field camera captures the light rays passing through an entire area. This generates (in theory) a 4D image. Still 2 degrees of freedom for ray direction, but an extra 2 degrees for ray position. Because the rays pass through different points, you can re-project a light field image to create 2D images with different depth of field, or different virtual camera positions.
- jiggawatts 5y ago
  A surprising aspect is that the lightfield merely appears to be 4D. If you stop and think about it, all of the information can be captured in the 2D plane of the aperture. So effectively, it is just a 2D image! This is akin to a hologram, which looks truly 3D, with depth of field, refocusing, and the works, but is physically a 2D photographic plate. Literally just a sheet. The hologram captures the 2D slice through the EM field the same way that a lightfield captures a 2D slice through the aperture. The key thing is that the phase information must be captured also, which in the case of holograms requires a resolution finer than the wavelength of light. Lightfield imaging does it differently, but the same concepts apply.
  - tragictrash 5y ago
    When you say hologram, are you referring to something like a holographic sticker, or a star trek hologram?
    - mlyle 5y ago
      He's referring to not the star trek hologram, and not really the holographic sticker. This is one holographic plate illuminated by a laser, viewed from different viewpoints: https://en.wikipedia.org/wiki/Holography#/media/File:Holomouse2.jpg https://en.wikipedia.org/wiki/Holography#/media/File:Holomou... And it's not like those things with the bumpy lenses, that show you an image from a few chosen viewpoints, either. You can look at that mouse from a wide range of angles and distances. Handwavy, short explanation: All the light arriving back from the mouse-- in different directions-- is encoded into the speckles on the plate. When a coherent source of light is used to illuminate the plate, again, the light exits with the same characteristics it had entering the plate.
signaru 5y ago
The idea of having a "field" is that a 2d slice of it allows calculating light elsewhere from the field using optical propagation calculations. But this requires phase measurements in addition to intensity which is only what conventional cameras capture. It could be easier to understand by looking the parts of a light field camera. Compared to a conventional camera, one type of light field camera has a micro-lens array just before the sensor. So this is like placing a Shack-Hartmann wavefront sensor over the image (apologies for another technical term, but that roughly explains the concept and is easily looked up, implementation details might be different though). An ordinary photograph only stores intensities, so changing the focus given the recorded image is not possible using physical optics calculations (ML/AI algorithms might do the trick, but that's something else). Based on the image formed from each lens in the micro-lens array (i.e. the displacement of the focus from the sensor), it is possible to retrieve the wavefront or lightfield which allows subsequent optics calculations, like post-focusing. The tradeoff is that each micro-lens in the array needs a decent number of pixels for analysis, so the effective 2d resolution is decreased as compared to using all the pixels.

You can think of a normal lense like an extremely specific filter. It ensures light from a specific direction, and that direction only, arrives at the sensor. Some also ensure only light oscillating in a specific way goes through (polarization), or of a specific frequency (with color filters). This filtering is necessary for us to produce useful data from the total chaos that is the actual light field - aka all the light bouncing and going in all directions at once - where ‘us’ are our normal human brains. Other ways of handling it though if you can use different types of information processing.

ReactiveJelly 5y ago

An ideal camera captures the light rays passing through a pinhole. It generates a 2D image where each pixel corresponds to a ray, and the X and Y coords determine the direction of the ray. Because every ray passes through the same pinhole point, you can't do things like change the depth-of-field or move the camera once the image is created. (An ideal pinhole camera has everything in focus) A light-field camera captures the light rays passing through an entire area. This generates (in theory) a 4D image. Still 2 degrees of freedom for ray direction, but an extra 2 degrees for ray position. Because the rays pass through different points, you can re-project a light field image to create 2D images with different depth of field, or different virtual camera positions.

jiggawatts 5y ago
A surprising aspect is that the lightfield merely appears to be 4D. If you stop and think about it, all of the information can be captured in the 2D plane of the aperture. So effectively, it is just a 2D image! This is akin to a hologram, which looks truly 3D, with depth of field, refocusing, and the works, but is physically a 2D photographic plate. Literally just a sheet. The hologram captures the 2D slice through the EM field the same way that a lightfield captures a 2D slice through the aperture. The key thing is that the phase information must be captured also, which in the case of holograms requires a resolution finer than the wavelength of light. Lightfield imaging does it differently, but the same concepts apply.
- tragictrash 5y ago
  When you say hologram, are you referring to something like a holographic sticker, or a star trek hologram?
  - mlyle 5y ago
    He's referring to not the star trek hologram, and not really the holographic sticker. This is one holographic plate illuminated by a laser, viewed from different viewpoints: https://en.wikipedia.org/wiki/Holography#/media/File:Holomouse2.jpg https://en.wikipedia.org/wiki/Holography#/media/File:Holomou... And it's not like those things with the bumpy lenses, that show you an image from a few chosen viewpoints, either. You can look at that mouse from a wide range of angles and distances. Handwavy, short explanation: All the light arriving back from the mouse-- in different directions-- is encoded into the speckles on the plate. When a coherent source of light is used to illuminate the plate, again, the light exits with the same characteristics it had entering the plate.

signaru 5y ago

The idea of having a "field" is that a 2d slice of it allows calculating light elsewhere from the field using optical propagation calculations. But this requires phase measurements in addition to intensity which is only what conventional cameras capture. It could be easier to understand by looking the parts of a light field camera. Compared to a conventional camera, one type of light field camera has a micro-lens array just before the sensor. So this is like placing a Shack-Hartmann wavefront sensor over the image (apologies for another technical term, but that roughly explains the concept and is easily looked up, implementation details might be different though). An ordinary photograph only stores intensities, so changing the focus given the recorded image is not possible using physical optics calculations (ML/AI algorithms might do the trick, but that's something else). Based on the image formed from each lens in the micro-lens array (i.e. the displacement of the focus from the sensor), it is possible to retrieve the wavefront or lightfield which allows subsequent optics calculations, like post-focusing. The tradeoff is that each micro-lens in the array needs a decent number of pixels for analysis, so the effective 2d resolution is decreased as compared to using all the pixels.