4 ms·

Isn't this the lytro camera?

by krackers 10mo ago

analog31 10mo ago
The article mentions a spatial light modulator, which I believe the Lytro camera did not have.
- Forgeties79 10mo ago
  The image(s) were also trash unfortunately and a PITA to process. Barely usable even in ideal circumstances.
  - NooneAtAll3 10mo ago
    eh?? Processing was as simple as "click on the thing you want in focus". and 4MP was just fine for casual use it was targetting
    - Forgeties79 10mo ago
      You couldn’t offload and use the refocus feature without their software and 4MP was half what smartphones were doing in 2012 (rapidly increasing above 8MP after that). Fixed lens so you couldn’t improve the image quality with glass - that’s totally understandable given the product but it is still a limitation to the visual quality. That’s a bad recipe for casual and professional users alike. Can’t ingest into your workflow quickly, images are low res, can’t improve the image, and your smartphone was better just missing one, admittedly neat, feature. If that existed in phones people would use it like crazy I imagine. Too narrow of a use case IMO, too many compromises for one feature, hence why it failed.
      - analog31 10mo ago
        I looked into the details at the time, and as I recall the camera had lower resolution than the 4 MP of the sensor because of the microlens array. A lot was written about this at the time. I remember a friend, who was a photography buff, was quite excited about the camera. But he didn't actually buy one.
        NooneAtAll3 10mo ago
        sensor was 40MP, image was 4MP
      - cycomanic 10mo ago
        I doubt resolution is a limiting factor LCoS (transmissive ones are lower resolution typically though, but you could build reflective) phase modulators are available in 4k resolution (and maybe even higher). And I don't know if you need that much resolution because the regions you're trying to focus are quite broad, in fact I suspect the resolution of your phase modulator would not limit resolution but the max distance between focused regions, because it would set the max phase ramp you can achieve. Loss, i.e. equivalent aperture is a different matter and I think this would imply quite a light loss.
        Forgeties79 10mo ago
        > I doubt resolution is a limiting factor LCoS (transmissive ones are lower resolution typically though, but you could build reflective) phase modulators are available in 4k resolution (and maybe even higher) We’re talking about a specific camera, the lytros, which had a 4MP resolution. I’m not saying there was a limitation in the technology broadly speaking. Just that this camera was not worth it for the time. It’s sacrificed too much for one feature and at $400 it just didn’t sell
stevenjgarner 10mo ago
I believe the lytro camera was a plenoptic, or light field, camera. Light field cameras capture information about the intensity together with the direction of light emanating from a scene. Conventional cameras record only light intensity at various wavelengths. While conventional cameras capture a single high-resolution focal plane and light field cameras sacrifice resolution to "re-focus" via software after the fact, the CMU Split-Lohmann camera provides a middle ground, using an adaptive computational lens to physically focus every part of the image independently. This allows it to capture a "deep-focus" image where objects at multiple distances are sharp simultaneously, maintaining the high resolution of a conventional camera while achieving the depth flexibility of a light field camera without the blur or data loss. Something I find interesting is that while holograms and the CMU camera both manipulate the "phase" of light, they do so for opposite reasons: a hologram records phase to recreate a 3D volume, whereas the CMU camera modulates phase to fix a 2D image.
- mastazi 10mo ago
  Interesting. So if I understand correctly, it's like a nonlinear version of a "tilt lens"? https://en.wikipedia.org/wiki/Tilt%E2%80%93shift_photography#Tilt https://en.wikipedia.org/wiki/Tilt%E2%80%93shift_photography...
  - stevenjgarner 10mo ago
    Yes, I love the analogy. You can think of the CMU Split-Lohmann system as a "per-pixel tilt-shift lens" or a "freeform focal surface" camera.
fainpul 10mo ago
Light field cameras are mentioned under "related work": https://imaging.cs.cmu.edu/svaf/static/pdfs/Spatially_Varying_Autofocus.pdf#page=3 https://imaging.cs.cmu.edu/svaf/static/pdfs/Spatially_Varyin...
hbarka 10mo ago
I remember Lytro. There was a lot of fanfare behind that company and then they fizzled. They had a lauded CEO/founder and their website demonstrated clearly how the post-focus worked. It felt like they were going to be the next camera revolution. Their rise and demise story would make a good Isaacson-style documentary.
- chychiu 10mo ago
  I think the product was just too early for its time, and there is not much demand for it. For what it's worth, the founder (Ren Ng) went back to academia, and was highly instrumental in computer vision research, e.g. being the PI on the paper for NeRF: (https://dl.acm.org/doi/abs/10.1145/3503250 https://dl.acm.org/doi/abs/10.1145/3503250)
  - dale_glass 10mo ago
    I don't think it was quite too early, it just makes tradeoffs that are undesirable. Lytro as I understand it, trades a huge amount of resolution for the focusing capability. Some ridiculous amount, like the user gets to see just 1/8th of the pixels on the sensor. In a way, I'd say rather than too early it was too late. Because autofocus was already quite good and getting better. You don't need to sacrifice all that resolution when you can just have good AF to start with. Refocusing in post is a very rare need if you got the focus right initially. And time has only made that even worse. Modern autofocus is darn near magic, and people love their high resolution photos.
    - amelius 10mo ago
      There is a limit to the resolution needed by consumers, so in that sense maybe they were too early.
      - dale_glass 10mo ago
        I'd argue the opposite, consumers need more resolution than pros. A pro will show up with a 300mm f/2.8, a tripod, a camera with good AF and high ISO, and the skills, plan and patience to catch birds in flight. But all that stuff is expensive. The consumer way to approximate the lack of a good lens is a small, high res sensor. That only works in bright light, but you can get good results with affordable equipment in the right conditions. Greatly reducing the resolution is far from optimal when you can't have a big fancy lens to compensate. And where is focus the hardest? Mostly where you want to have high detail. Wildlife, macro, sports.

Isn't this the lytro camera?

analog31 10mo ago

The article mentions a spatial light modulator, which I believe the Lytro camera did not have.

Forgeties79 10mo ago
The image(s) were also trash unfortunately and a PITA to process. Barely usable even in ideal circumstances.
- NooneAtAll3 10mo ago
  eh?? Processing was as simple as "click on the thing you want in focus". and 4MP was just fine for casual use it was targetting
  - Forgeties79 10mo ago
    You couldn’t offload and use the refocus feature without their software and 4MP was half what smartphones were doing in 2012 (rapidly increasing above 8MP after that). Fixed lens so you couldn’t improve the image quality with glass - that’s totally understandable given the product but it is still a limitation to the visual quality. That’s a bad recipe for casual and professional users alike. Can’t ingest into your workflow quickly, images are low res, can’t improve the image, and your smartphone was better just missing one, admittedly neat, feature. If that existed in phones people would use it like crazy I imagine. Too narrow of a use case IMO, too many compromises for one feature, hence why it failed.
    - analog31 10mo ago
      I looked into the details at the time, and as I recall the camera had lower resolution than the 4 MP of the sensor because of the microlens array. A lot was written about this at the time. I remember a friend, who was a photography buff, was quite excited about the camera. But he didn't actually buy one.
      - NooneAtAll3 10mo ago
        sensor was 40MP, image was 4MP
    - cycomanic 10mo ago
      I doubt resolution is a limiting factor LCoS (transmissive ones are lower resolution typically though, but you could build reflective) phase modulators are available in 4k resolution (and maybe even higher). And I don't know if you need that much resolution because the regions you're trying to focus are quite broad, in fact I suspect the resolution of your phase modulator would not limit resolution but the max distance between focused regions, because it would set the max phase ramp you can achieve. Loss, i.e. equivalent aperture is a different matter and I think this would imply quite a light loss.
      - Forgeties79 10mo ago
        > I doubt resolution is a limiting factor LCoS (transmissive ones are lower resolution typically though, but you could build reflective) phase modulators are available in 4k resolution (and maybe even higher) We’re talking about a specific camera, the lytros, which had a 4MP resolution. I’m not saying there was a limitation in the technology broadly speaking. Just that this camera was not worth it for the time. It’s sacrificed too much for one feature and at $400 it just didn’t sell

stevenjgarner 10mo ago

I believe the lytro camera was a plenoptic, or light field, camera. Light field cameras capture information about the intensity together with the direction of light emanating from a scene. Conventional cameras record only light intensity at various wavelengths. While conventional cameras capture a single high-resolution focal plane and light field cameras sacrifice resolution to "re-focus" via software after the fact, the CMU Split-Lohmann camera provides a middle ground, using an adaptive computational lens to physically focus every part of the image independently. This allows it to capture a "deep-focus" image where objects at multiple distances are sharp simultaneously, maintaining the high resolution of a conventional camera while achieving the depth flexibility of a light field camera without the blur or data loss. Something I find interesting is that while holograms and the CMU camera both manipulate the "phase" of light, they do so for opposite reasons: a hologram records phase to recreate a 3D volume, whereas the CMU camera modulates phase to fix a 2D image.

mastazi 10mo ago
Interesting. So if I understand correctly, it's like a nonlinear version of a "tilt lens"? https://en.wikipedia.org/wiki/Tilt%E2%80%93shift_photography#Tilt https://en.wikipedia.org/wiki/Tilt%E2%80%93shift_photography...
- stevenjgarner 10mo ago
  Yes, I love the analogy. You can think of the CMU Split-Lohmann system as a "per-pixel tilt-shift lens" or a "freeform focal surface" camera.

fainpul 10mo ago

Light field cameras are mentioned under "related work": https://imaging.cs.cmu.edu/svaf/static/pdfs/Spatially_Varying_Autofocus.pdf#page=3 https://imaging.cs.cmu.edu/svaf/static/pdfs/Spatially_Varyin...

hbarka 10mo ago

I remember Lytro. There was a lot of fanfare behind that company and then they fizzled. They had a lauded CEO/founder and their website demonstrated clearly how the post-focus worked. It felt like they were going to be the next camera revolution. Their rise and demise story would make a good Isaacson-style documentary.

chychiu 10mo ago
I think the product was just too early for its time, and there is not much demand for it. For what it's worth, the founder (Ren Ng) went back to academia, and was highly instrumental in computer vision research, e.g. being the PI on the paper for NeRF: (https://dl.acm.org/doi/abs/10.1145/3503250 https://dl.acm.org/doi/abs/10.1145/3503250)
- dale_glass 10mo ago
  I don't think it was quite too early, it just makes tradeoffs that are undesirable. Lytro as I understand it, trades a huge amount of resolution for the focusing capability. Some ridiculous amount, like the user gets to see just 1/8th of the pixels on the sensor. In a way, I'd say rather than too early it was too late. Because autofocus was already quite good and getting better. You don't need to sacrifice all that resolution when you can just have good AF to start with. Refocusing in post is a very rare need if you got the focus right initially. And time has only made that even worse. Modern autofocus is darn near magic, and people love their high resolution photos.
  - amelius 10mo ago
    There is a limit to the resolution needed by consumers, so in that sense maybe they were too early.
    - dale_glass 10mo ago
      I'd argue the opposite, consumers need more resolution than pros. A pro will show up with a 300mm f/2.8, a tripod, a camera with good AF and high ISO, and the skills, plan and patience to catch birds in flight. But all that stuff is expensive. The consumer way to approximate the lack of a good lens is a small, high res sensor. That only works in bright light, but you can get good results with affordable equipment in the right conditions. Greatly reducing the resolution is far from optimal when you can't have a big fancy lens to compensate. And where is focus the hardest? Mostly where you want to have high detail. Wildlife, macro, sports.