11 ms·

Light Field Photography

58 points by cnahr 12y ago

acous 12y ago
> "Holographic Recording ... remains irrelevant to consumers until we have convenient holographic displays to match." I'm not sure why Lytro hasn't touted this side of the technology... the first consumer holographic display (oculus rift) is due relatively soon.
- phreeza 12y ago
  The rift is not really holographic, it's 'just' stereoscopic. Nvidia is working on light field glasses though, and I believe this might be the future, because it allows for much sleeker eyepieces, at a higher computational cost. https://research.nvidia.com/publication/near-eye-light-field-displays https://research.nvidia.com/publication/near-eye-light-field...
  - acous 12y ago
    I believe[1] that the head tracking qualifies it as holographic. That light field stuff looks super interesting... 1. http://doc-ok.org/?p=337 http://doc-ok.org/?p=337
    - sp332 12y ago
      It's not holographic because you can't focus your eyes at different depths in the image.
      - dynode 12y ago
        It's not holographic because it doesn't use diffractive / interferometric effects. [1] https://en.wikipedia.org/wiki/Holography https://en.wikipedia.org/wiki/Holography
- bane 12y ago
  Holography is a specific thing, let's not confuse things by assigning magical powers to the Rift. Holographic displays already exist, usually they're transparent, they work by letting you focus on things through the display and the display stays in focus, making things like AR and HUDs work much better than other types of displays.
  - acous 12y ago
    My mistake, but I'm not assigning magical powers. I'm aware of what a holographic display is, and the rift fulfils 3 of the 4 criteria. The lack of accommodation in my experience is not critical and the applicability to lytro is still valid.
    - blincoln 12y ago
      I could be wrong, but IMO you and bane are both confusing "holography" with "3D display", and/or "display that involves depth". The author of the article made a similar mistake, and unfortunately a lot of commercial products are mislabeled as "holographic", so this is unsurprising. As bane said, holography is a very specific thing. Most of the products I'm aware of today that claim to be "holographic" are things like head-up displays and red-dot gun sights. They use optics to make it appear as though a 2D image is appearing some distance behind the display surface. It's a clever, useful effect, but it is not holographic. I think the author of the article is also leaving out one of the main interesting potential uses of depth-sensing cameras, which is to feed the output into a 3D printer. You don't need a holographic display (or even a faux-holographic depth display) for that.
      - cnahr 12y ago
        My use of the term "holographic" was taken directly from Ng's dissertation, near the end of section 3.7: "...Camahart showed that holographic stereograms can be synthesized from light fields." Whether he's strictly correct with his use of the term I don't know. Ng also mentions working with Zebra Imaging on printing such "holograms" [sic], so that's certainly a possible application. I'm not sure a lot of consumers would be interested in that, though, compared to simply showing pictures.
win_ini 12y ago
I agree with most of the statements - but one aspect I think a lytro camera is better at: street photography. Quick, shots of street moments. Shots where you have no time to focus properly, but want to focus the field of view on a subject...a lightfield camera is great for these situations and I think isn't played up enough.
NIL8 12y ago
The Lytro gallery: https://pictures.lytro.com/lytro/albums/149429/embed?token=6cb04136-c43a-11e3-9416-22000a8b14ce https://pictures.lytro.com/lytro/albums/149429/embed?token=6... I want this.
- hnriot 12y ago
  a photo where nothing is in focus. great. lytro needs a different name than photography because it's something else.
  - bobbles 12y ago
    Are you serious? Click where ever you want to focus the shot
    - dpark 12y ago
      The low sensor resolution means that nothing looks in focus. There's insufficient resolution (and maybe processing artifacts add to the issue?) for anything to look crisply focused.
    - hnriot 12y ago
      did you actually try it? click on the windshield, is the driver in focus? no.
bridger 12y ago
I took a class that explained light field photography, but I thought understanding the concepts through static diagrams was difficult. I made a little Mac app that simulates a 2D scene of light sources, lenses, and light-field sensors. The sensors shows their captured output in a graph. The source is at https://github.com/bridger/optic-workbench https://github.com/bridger/optic-workbench A bit of explanation (and a link to the pre-built binary) is at http://www.bridgermaxwell.com/index.php/blog/optic-workbench-light-field-capture/ http://www.bridgermaxwell.com/index.php/blog/optic-workbench.... (Sorry it is only for Macs. I would love to redo it in Javascript someday, once I know Javascript.)
spankalee 12y ago
I'm not sure that consumers don't care about bokeh, they just don't know how to achieve it. I've been using the Google camera app that does lens blur and everyone I show it to loves it. It's a little cumbersome to use though and require a still subject. Maybe light field could be one of the features that keep compact cameras around in the face of competition from phones. I also wonder if there are computer vision applications. I don't know how accurate depth fields from stereoscopic images are, and maybe two cameras are not always practical...
- dpark 12y ago
  Bokeh isn't just reduced depth of field. Bokeh is the quality of the out-of-focus areas. Shallow depth of field makes the bokeh more apparent, but the two are somewhat orthogonal. Looking at Lytro's examples, e.g. [1], I would say that the Illum demonstrates poor quality bokeh as a result of the sensor limitations. The bokeh is quite noisy and pixelated (which is a strange thing to say about bokeh, but it's the result of the microlens sampling the Lytro sensor does). The technology is very interesting, but looking at Lytro's pictures has convinced me that their camera will not suit my needs any time soon. [1] https://pictures.lytro.com/lytro/albums/149429/embed?token=6cb04136-c43a-11e3-9416-22000a8b14ce https://pictures.lytro.com/lytro/albums/149429/embed?token=6...
erikpukinskis 12y ago
The thing that excites me most about Light Field Photography is that LF cameras are the only thing that could take a photo that would actually feel at home in an Oculus Rift. Binocular cameras fake the 3D experience by capturing two 2d images and fixing them to each eye. But in the Oculus you can move your head around so two 2d images aren't enough. You need an actual depth map in order to generate images in response to head movements. LF cameras can provide one. http://www.cs.berkeley.edu/~ravir/lightfield_ICCV.pdf http://www.cs.berkeley.edu/~ravir/lightfield_ICCV.pdf
- nakedrobot2 12y ago
  Not really. Without the correct IOD (interocular distance) [1] you won't get "correct" stereoscopy, and the effect you get will not feel right at best, and make you sick in the worst case. A LF camera only gives a stereoscopic effect proportional to the size of the sensor. So unless you have a sensor even bigger than a medium format camera, you are not going to achieve the effect you are describing. [1] https://en.wikipedia.org/wiki/Stereoscopy https://en.wikipedia.org/wiki/Stereoscopy
  - mapt 12y ago
    The premise here would be two LF cameras, mounted rigidly in an IOD-like frame. Or ideally, more than two, for flexibility of the IOD and a better stereo-vision fix, potentially used in 2D-3D compositing.
fuzzythinker 12y ago
I don't see the $1.5k Lytro is going to gain enough market share to be profitable too. The only option I see for Lytro is to miniaturize their technology so that it is significantly better than Google's software offering and to partner with a major phone manufacturer.
porker 12y ago
Could LFP be used to 3D-map a scene - assuming you can extract the data from their proprietary format? It would be great to be able to turn a collection of photos into a virtual world.
mapt 12y ago
The problem with lightfield photography, other than the inherent image quality calculus of splitting a sensor into a bunch of lower-resolution, lower-illumination, differently-focused subfields, is that it's so easily simulated if anyone actually attempts to do scene capture properly - through adaptations of stereo-vision algorithms for higher-N arrays of semi-calibrated sensors, or near-infrared structured light (like the Kinect) for a 4D scene, or structure from motion for a static 3D scene. The new HTC One M8 uses crap smartphone sensors, only two of them, without any structured light, with a first-generation stereo-blurring algorithm dumb enough to run quickly on a smartphone, and it produces a rough approximation of the same product as Lytro.
- jychang 12y ago
  As a programmer who also happens to be a photographer, that's flat out wrong. The M8 produces a very crappy imitation of out of focus blur of what the bokeh of an camera with a thin depth of field looks like. [1] The linked article is actually being "neutral" and fairly generous; the pictures, frankly, look like shit. It is difficult to properly do DoF, even if you know the Z-distance of every object in the scene. It's not just a simple blur of things that are deemed out of focus; things like spherical differences in the lens to produce hard/soft bokeh matter. [2] "So easily simulated" couldn't be further from the truth. A bunch of academic papers have been written about this, [3] which include ways to reproduce effects that are similar to the "thin DoF" effect in large sensor cameras, but few try to tackle _all_ of the issues and nuances present. [1] http://www.trustedreviews.com/opinions/htc-one-m8-camera-vs-a-proper-camera http://www.trustedreviews.com/opinions/htc-one-m8-camera-vs-... [2] http://jtra.cz/stuff/essays/bokeh/ http://jtra.cz/stuff/essays/bokeh/ [3] http://dl.acm.org/citation.cfm?id=2389316.2389318 http://dl.acm.org/citation.cfm?id=2389316.2389318
  - dpark 12y ago
    To be fair, Bokeh on the Illum is pretty poor, too. The M8 bokeh looks fake. The Illum Bokeh looks noisy and pixelated (and the effective resolution looks far lower than 5MP to me). I wouldn't bother with either.
  - mapt 12y ago

58 points by cnahr 12y ago

acous 12y ago

> "Holographic Recording ... remains irrelevant to consumers until we have convenient holographic displays to match." I'm not sure why Lytro hasn't touted this side of the technology... the first consumer holographic display (oculus rift) is due relatively soon.

phreeza 12y ago
The rift is not really holographic, it's 'just' stereoscopic. Nvidia is working on light field glasses though, and I believe this might be the future, because it allows for much sleeker eyepieces, at a higher computational cost. https://research.nvidia.com/publication/near-eye-light-field-displays https://research.nvidia.com/publication/near-eye-light-field...
- acous 12y ago
  I believe[1] that the head tracking qualifies it as holographic. That light field stuff looks super interesting... 1. http://doc-ok.org/?p=337 http://doc-ok.org/?p=337
  - sp332 12y ago
    It's not holographic because you can't focus your eyes at different depths in the image.
    - dynode 12y ago
      It's not holographic because it doesn't use diffractive / interferometric effects. [1] https://en.wikipedia.org/wiki/Holography https://en.wikipedia.org/wiki/Holography
bane 12y ago
Holography is a specific thing, let's not confuse things by assigning magical powers to the Rift. Holographic displays already exist, usually they're transparent, they work by letting you focus on things through the display and the display stays in focus, making things like AR and HUDs work much better than other types of displays.
- acous 12y ago
  My mistake, but I'm not assigning magical powers. I'm aware of what a holographic display is, and the rift fulfils 3 of the 4 criteria. The lack of accommodation in my experience is not critical and the applicability to lytro is still valid.
  - blincoln 12y ago
    I could be wrong, but IMO you and bane are both confusing "holography" with "3D display", and/or "display that involves depth". The author of the article made a similar mistake, and unfortunately a lot of commercial products are mislabeled as "holographic", so this is unsurprising. As bane said, holography is a very specific thing. Most of the products I'm aware of today that claim to be "holographic" are things like head-up displays and red-dot gun sights. They use optics to make it appear as though a 2D image is appearing some distance behind the display surface. It's a clever, useful effect, but it is not holographic. I think the author of the article is also leaving out one of the main interesting potential uses of depth-sensing cameras, which is to feed the output into a 3D printer. You don't need a holographic display (or even a faux-holographic depth display) for that.
    - cnahr 12y ago
      My use of the term "holographic" was taken directly from Ng's dissertation, near the end of section 3.7: "...Camahart showed that holographic stereograms can be synthesized from light fields." Whether he's strictly correct with his use of the term I don't know. Ng also mentions working with Zebra Imaging on printing such "holograms" [sic], so that's certainly a possible application. I'm not sure a lot of consumers would be interested in that, though, compared to simply showing pictures.

win_ini 12y ago

I agree with most of the statements - but one aspect I think a lytro camera is better at: street photography. Quick, shots of street moments. Shots where you have no time to focus properly, but want to focus the field of view on a subject...a lightfield camera is great for these situations and I think isn't played up enough.

NIL8 12y ago

The Lytro gallery: https://pictures.lytro.com/lytro/albums/149429/embed?token=6cb04136-c43a-11e3-9416-22000a8b14ce https://pictures.lytro.com/lytro/albums/149429/embed?token=6... I want this.

hnriot 12y ago
a photo where nothing is in focus. great. lytro needs a different name than photography because it's something else.
- bobbles 12y ago
  Are you serious? Click where ever you want to focus the shot
  - dpark 12y ago
    The low sensor resolution means that nothing looks in focus. There's insufficient resolution (and maybe processing artifacts add to the issue?) for anything to look crisply focused.
  - hnriot 12y ago
    did you actually try it? click on the windshield, is the driver in focus? no.

bridger 12y ago

I took a class that explained light field photography, but I thought understanding the concepts through static diagrams was difficult. I made a little Mac app that simulates a 2D scene of light sources, lenses, and light-field sensors. The sensors shows their captured output in a graph. The source is at https://github.com/bridger/optic-workbench https://github.com/bridger/optic-workbench A bit of explanation (and a link to the pre-built binary) is at http://www.bridgermaxwell.com/index.php/blog/optic-workbench-light-field-capture/ http://www.bridgermaxwell.com/index.php/blog/optic-workbench.... (Sorry it is only for Macs. I would love to redo it in Javascript someday, once I know Javascript.)

spankalee 12y ago

I'm not sure that consumers don't care about bokeh, they just don't know how to achieve it. I've been using the Google camera app that does lens blur and everyone I show it to loves it. It's a little cumbersome to use though and require a still subject. Maybe light field could be one of the features that keep compact cameras around in the face of competition from phones. I also wonder if there are computer vision applications. I don't know how accurate depth fields from stereoscopic images are, and maybe two cameras are not always practical...

dpark 12y ago
Bokeh isn't just reduced depth of field. Bokeh is the quality of the out-of-focus areas. Shallow depth of field makes the bokeh more apparent, but the two are somewhat orthogonal. Looking at Lytro's examples, e.g. [1], I would say that the Illum demonstrates poor quality bokeh as a result of the sensor limitations. The bokeh is quite noisy and pixelated (which is a strange thing to say about bokeh, but it's the result of the microlens sampling the Lytro sensor does). The technology is very interesting, but looking at Lytro's pictures has convinced me that their camera will not suit my needs any time soon. [1] https://pictures.lytro.com/lytro/albums/149429/embed?token=6cb04136-c43a-11e3-9416-22000a8b14ce https://pictures.lytro.com/lytro/albums/149429/embed?token=6...

erikpukinskis 12y ago

The thing that excites me most about Light Field Photography is that LF cameras are the only thing that could take a photo that would actually feel at home in an Oculus Rift. Binocular cameras fake the 3D experience by capturing two 2d images and fixing them to each eye. But in the Oculus you can move your head around so two 2d images aren't enough. You need an actual depth map in order to generate images in response to head movements. LF cameras can provide one. http://www.cs.berkeley.edu/~ravir/lightfield_ICCV.pdf http://www.cs.berkeley.edu/~ravir/lightfield_ICCV.pdf

nakedrobot2 12y ago
Not really. Without the correct IOD (interocular distance) [1] you won't get "correct" stereoscopy, and the effect you get will not feel right at best, and make you sick in the worst case. A LF camera only gives a stereoscopic effect proportional to the size of the sensor. So unless you have a sensor even bigger than a medium format camera, you are not going to achieve the effect you are describing. [1] https://en.wikipedia.org/wiki/Stereoscopy https://en.wikipedia.org/wiki/Stereoscopy
- mapt 12y ago
  The premise here would be two LF cameras, mounted rigidly in an IOD-like frame. Or ideally, more than two, for flexibility of the IOD and a better stereo-vision fix, potentially used in 2D-3D compositing.

fuzzythinker 12y ago

I don't see the $1.5k Lytro is going to gain enough market share to be profitable too. The only option I see for Lytro is to miniaturize their technology so that it is significantly better than Google's software offering and to partner with a major phone manufacturer.

porker 12y ago

Could LFP be used to 3D-map a scene - assuming you can extract the data from their proprietary format? It would be great to be able to turn a collection of photos into a virtual world.

mapt 12y ago

The problem with lightfield photography, other than the inherent image quality calculus of splitting a sensor into a bunch of lower-resolution, lower-illumination, differently-focused subfields, is that it's so easily simulated if anyone actually attempts to do scene capture properly - through adaptations of stereo-vision algorithms for higher-N arrays of semi-calibrated sensors, or near-infrared structured light (like the Kinect) for a 4D scene, or structure from motion for a static 3D scene. The new HTC One M8 uses crap smartphone sensors, only two of them, without any structured light, with a first-generation stereo-blurring algorithm dumb enough to run quickly on a smartphone, and it produces a rough approximation of the same product as Lytro.

jychang 12y ago
As a programmer who also happens to be a photographer, that's flat out wrong. The M8 produces a very crappy imitation of out of focus blur of what the bokeh of an camera with a thin depth of field looks like. [1] The linked article is actually being "neutral" and fairly generous; the pictures, frankly, look like shit. It is difficult to properly do DoF, even if you know the Z-distance of every object in the scene. It's not just a simple blur of things that are deemed out of focus; things like spherical differences in the lens to produce hard/soft bokeh matter. [2] "So easily simulated" couldn't be further from the truth. A bunch of academic papers have been written about this, [3] which include ways to reproduce effects that are similar to the "thin DoF" effect in large sensor cameras, but few try to tackle _all_ of the issues and nuances present. [1] http://www.trustedreviews.com/opinions/htc-one-m8-camera-vs-a-proper-camera http://www.trustedreviews.com/opinions/htc-one-m8-camera-vs-... [2] http://jtra.cz/stuff/essays/bokeh/ http://jtra.cz/stuff/essays/bokeh/ [3] http://dl.acm.org/citation.cfm?id=2389316.2389318 http://dl.acm.org/citation.cfm?id=2389316.2389318
- dpark 12y ago
  To be fair, Bokeh on the Illum is pretty poor, too. The M8 bokeh looks fake. The Illum Bokeh looks noisy and pixelated (and the effective resolution looks far lower than 5MP to me). I wouldn't bother with either.
- mapt 12y ago