7 ms·

Lensless camera creates 3D images from single exposure

72 points by sizzle 4y ago

sp332 4y ago
Why does "lensless" mean "has a bunch of lenses"?
- jaclaz 4y ago
  >“We consider our camera lensless because it replaces the bulk lenses used in conventional cameras with a thin, lightweight microlens array made of flexible polymer,” said research team leader Weijian Yang from the University of California, Davis. Opinions are opinions, peronally I would consider it multi-lens or poly-lens.
  - hackernewds 4y ago
    Multi-lens would imply more lenses. Which wouldn't get the same amount of attention or marketability
- pavlov 4y ago
  It’s the same logic as “serverless”?
- jfengel 4y ago
  Because it's a very different concept from a conventional camera lens. It does not produce a focused image. Each lens corresponds to a single sensor, which is manipulated to receive light from multiple angles. A regular lens manipulates the light so that each sensor corresponds to one pixel, mimicking the way a piece of film would produce a color at each spot. With a light field camera the actual image has to be constructed computationally afterwards. In effect, the software computes the effect a lens would have had on the light -- meaning you can set the focus later. There is a lens-like apparatus built into each sensor to allow it to gather the types of data it needs but it's not at all like a conventional camera lens. You couldn't put on a conventional camera lens, and none of the things that a photographer uses a "lens" for are necesary. "Lensless" seems as good a word as any for that: it accurately conveys a camera that you do not and cannot focus, and has no object corresponding to the lens.
  - nomel 4y ago
    > It does not produce a focused image I don't think this is technically correct. There are many focused images being produced: https://www.youtube.com/watch?v=rEMP3XEgnws https://www.youtube.com/watch?v=rEMP3XEgnws > but it's not at all like a conventional camera lens The physics is exactly the same as a conventional lens, because each micro lens is conventional lenses. It's all conventional, up to the point of having to separate all of the real, focused, images to make one clear image. For proof, if you block all the micro lenses, except one, you would see a boring, focused, real image projected onto the sensor. In fact, the original light field cameras, that you could buy off the shelf, had a single, separate, image per microlens. Looking at the image in this press release, it looks the same.
    - eloeffler 4y ago
      So it's also not a single exposure because each microlense-sensor team has its own exposure. At this point we could connect each sensor to its own circuitry and storage and then say it's a cameraless photo altogether :)
- deleted 4y ago
  [deleted]
- anigbrowl 4y ago
  'compound lens camera' would be better. Much of this work has roots in the study of insect vision.
- zakki 4y ago
  i think the naming got infected from a computer virus named serverless.
gurumeditations 4y ago
Lytro?
- birdman3131 4y ago
  My thought exactly.
birdman3131 4y ago
We have had this for years. The term is Light Field Camera. There was an attempt several years back to do this with the Lytro camera. It worked but had a few major issues. For one the resolution loss was fairly substantial. There was also a fair few issues around the fact that the only way to share the photos was a cumbersome vendor lock in site iirc. With more modern sensors you might be able to fix the resolution issue as I recall it being around 2010 or so.
- GloriousKoji 4y ago
  Resolution will always be an issue since it's a trade off of dedicating a fixed area to single 2D plane or a fraction of it to multiple planes in 3D.
  - jbay808 4y ago
    That's true, but it might be mainly a cost issue. You can make up for the resolution loss by using a much larger sensor; since the microlenses are very thin, it won't increase the weight or bulk of the device very much. Just the cost.
    - aaroninsf 4y ago
      This. The overview of the 48 megapixel camera on the iPhone 14 talked chipperly about the fact that the user-land resolution is already decimated (well, quadrimated?) 4 to 1 to 12mp images, with the subtext "but those are highly determined optimized pixels!"
      - wyager 4y ago
        The 48mp sensor on the iPhone seems like a gimmick - even with purely diffraction-limited optics, I doubt you could seriously increase optical bandwidth at that sensor size beyond, say, 10-20Mp, to say nothing of aberration limiting and shot noise constraints. Just now we're starting to see 40Mp APS-C sensors, and people are debating if it's worth it to do that on a sensor that's like 10x larger than the biggest iphone sensor.
MaxikCZ 4y ago
Tittle: "Lensless camera" First sentence: "Researchers have developed a camera that uses a thin microlens array..."
- hinkley 4y ago
  Did we say "no lenses"? We meant to say, "all lenses".
postalrat 4y ago
Hard to beat the computational power of a lens.
- blevin 4y ago
  Is there a concise term for the computation it is doing, besides something like "convolution with a gradually varying kernel"? It's weird to think the real computation is physics, and machines or optical systems are nested abstractions hosted on that. I guess one person's low-level / "metal" / "wire" is always someone else's API.
  - dekhn 4y ago
    https://en.wikipedia.org/wiki/Fourier_optics https://en.wikipedia.org/wiki/Fourier_optics physicists were doing fourier transforms long before they had computers. But it's not an abstraction- it's just a straightforward application of physics (although, I think I understand what you mean).
4ndrewl 4y ago
"Lensless" but actually has multiple lenses?
drewbeck 4y ago
From the paper: “ To the best of our knowledge, we are the first to demonstrate deep learning data-driven 3D photorealistic reconstruction without system calibration and initializations, and we are the first to demonstrate imaging objects behind obstacles using lensless imagers.” That’s their actual claim, worth examining. Also AFAICT their camera has no large traditional lens at all, whereas previous work — ie lytro — put a traditional lens in front of their micro lens array.
frozenport 4y ago
Highlights that academia is not the place for cutting edge work. What appears to be a change in an algorithm is sold new lensless camera.
whycombinetor 4y ago
This technique wouldn't be novel even if it _was_ lensless. Google search "lensfree holography" for plenty of existing literature.
- nomel 4y ago
  Or you can just go buy one: https://raytrix.de/ https://raytrix.de/ The Lytro camera was on the market in 2014.
fortran77 4y ago
Except it’s not lensless. It has an array of micro lenses.
olliej 4y ago
[Oh no, I found a review I wrote of the Lytro camera I bought at the time. The last modified in 2014 so I'm really not sure about the "quality" of the writing, it may well be "kwality", but if anyone wants to read a review of the Lytro written immediately around the time I got/used it: https://nerget.com/Lytro.html https://nerget.com/Lytro.html] I bought a Lytro camera that was doing this possibly more than a decade ago. It was a “consumer” device, but it was pretty useless (though kind of pretty to look at?). Essentially you lose huge amounts of 2d resolution on the sensor so the resultant picture were both very low res and also huge. Coupled with no standard format to be available, I vaguely recall that the only way to “share” the images was via a Facebook applet thing. I assume a decade of work means they can process the data faster and get better resolution, but the tech itself is far from new. Also you can title an article “this camera has no lens” and then start with “it has a microarray of thousands of lenses” :D

72 points by sizzle 4y ago

sp332 4y ago

Why does "lensless" mean "has a bunch of lenses"?

jaclaz 4y ago
>“We consider our camera lensless because it replaces the bulk lenses used in conventional cameras with a thin, lightweight microlens array made of flexible polymer,” said research team leader Weijian Yang from the University of California, Davis. Opinions are opinions, peronally I would consider it multi-lens or poly-lens.
- hackernewds 4y ago
  Multi-lens would imply more lenses. Which wouldn't get the same amount of attention or marketability
pavlov 4y ago
It’s the same logic as “serverless”?
jfengel 4y ago
Because it's a very different concept from a conventional camera lens. It does not produce a focused image. Each lens corresponds to a single sensor, which is manipulated to receive light from multiple angles. A regular lens manipulates the light so that each sensor corresponds to one pixel, mimicking the way a piece of film would produce a color at each spot. With a light field camera the actual image has to be constructed computationally afterwards. In effect, the software computes the effect a lens would have had on the light -- meaning you can set the focus later. There is a lens-like apparatus built into each sensor to allow it to gather the types of data it needs but it's not at all like a conventional camera lens. You couldn't put on a conventional camera lens, and none of the things that a photographer uses a "lens" for are necesary. "Lensless" seems as good a word as any for that: it accurately conveys a camera that you do not and cannot focus, and has no object corresponding to the lens.
- nomel 4y ago
  > It does not produce a focused image I don't think this is technically correct. There are many focused images being produced: https://www.youtube.com/watch?v=rEMP3XEgnws https://www.youtube.com/watch?v=rEMP3XEgnws > but it's not at all like a conventional camera lens The physics is exactly the same as a conventional lens, because each micro lens is conventional lenses. It's all conventional, up to the point of having to separate all of the real, focused, images to make one clear image. For proof, if you block all the micro lenses, except one, you would see a boring, focused, real image projected onto the sensor. In fact, the original light field cameras, that you could buy off the shelf, had a single, separate, image per microlens. Looking at the image in this press release, it looks the same.
  - eloeffler 4y ago
    So it's also not a single exposure because each microlense-sensor team has its own exposure. At this point we could connect each sensor to its own circuitry and storage and then say it's a cameraless photo altogether :)
deleted 4y ago
[deleted]
anigbrowl 4y ago
'compound lens camera' would be better. Much of this work has roots in the study of insect vision.
zakki 4y ago
i think the naming got infected from a computer virus named serverless.

gurumeditations 4y ago

Lytro?

birdman3131 4y ago
My thought exactly.

birdman3131 4y ago

We have had this for years. The term is Light Field Camera. There was an attempt several years back to do this with the Lytro camera. It worked but had a few major issues. For one the resolution loss was fairly substantial. There was also a fair few issues around the fact that the only way to share the photos was a cumbersome vendor lock in site iirc. With more modern sensors you might be able to fix the resolution issue as I recall it being around 2010 or so.

GloriousKoji 4y ago
Resolution will always be an issue since it's a trade off of dedicating a fixed area to single 2D plane or a fraction of it to multiple planes in 3D.
- jbay808 4y ago
  That's true, but it might be mainly a cost issue. You can make up for the resolution loss by using a much larger sensor; since the microlenses are very thin, it won't increase the weight or bulk of the device very much. Just the cost.
  - aaroninsf 4y ago
    This. The overview of the 48 megapixel camera on the iPhone 14 talked chipperly about the fact that the user-land resolution is already decimated (well, quadrimated?) 4 to 1 to 12mp images, with the subtext "but those are highly determined optimized pixels!"
    - wyager 4y ago
      The 48mp sensor on the iPhone seems like a gimmick - even with purely diffraction-limited optics, I doubt you could seriously increase optical bandwidth at that sensor size beyond, say, 10-20Mp, to say nothing of aberration limiting and shot noise constraints. Just now we're starting to see 40Mp APS-C sensors, and people are debating if it's worth it to do that on a sensor that's like 10x larger than the biggest iphone sensor.

MaxikCZ 4y ago

Tittle: "Lensless camera" First sentence: "Researchers have developed a camera that uses a thin microlens array..."

hinkley 4y ago
Did we say "no lenses"? We meant to say, "all lenses".

postalrat 4y ago

Hard to beat the computational power of a lens.

blevin 4y ago
Is there a concise term for the computation it is doing, besides something like "convolution with a gradually varying kernel"? It's weird to think the real computation is physics, and machines or optical systems are nested abstractions hosted on that. I guess one person's low-level / "metal" / "wire" is always someone else's API.
- dekhn 4y ago
  https://en.wikipedia.org/wiki/Fourier_optics https://en.wikipedia.org/wiki/Fourier_optics physicists were doing fourier transforms long before they had computers. But it's not an abstraction- it's just a straightforward application of physics (although, I think I understand what you mean).

4ndrewl 4y ago

"Lensless" but actually has multiple lenses?

drewbeck 4y ago

From the paper: “ To the best of our knowledge, we are the first to demonstrate deep learning data-driven 3D photorealistic reconstruction without system calibration and initializations, and we are the first to demonstrate imaging objects behind obstacles using lensless imagers.” That’s their actual claim, worth examining. Also AFAICT their camera has no large traditional lens at all, whereas previous work — ie lytro — put a traditional lens in front of their micro lens array.

frozenport 4y ago

Highlights that academia is not the place for cutting edge work. What appears to be a change in an algorithm is sold new lensless camera.

whycombinetor 4y ago

This technique wouldn't be novel even if it _was_ lensless. Google search "lensfree holography" for plenty of existing literature.

nomel 4y ago
Or you can just go buy one: https://raytrix.de/ https://raytrix.de/ The Lytro camera was on the market in 2014.

fortran77 4y ago

Except it’s not lensless. It has an array of micro lenses.

olliej 4y ago

[Oh no, I found a review I wrote of the Lytro camera I bought at the time. The last modified in 2014 so I'm really not sure about the "quality" of the writing, it may well be "kwality", but if anyone wants to read a review of the Lytro written immediately around the time I got/used it: https://nerget.com/Lytro.html https://nerget.com/Lytro.html] I bought a Lytro camera that was doing this possibly more than a decade ago. It was a “consumer” device, but it was pretty useless (though kind of pretty to look at?). Essentially you lose huge amounts of 2d resolution on the sensor so the resultant picture were both very low res and also huge. Coupled with no standard format to be available, I vaguely recall that the only way to “share” the images was via a Facebook applet thing. I assume a decade of work means they can process the data faster and get better resolution, but the tech itself is far from new. Also you can title an article “this camera has no lens” and then start with “it has a microarray of thousands of lenses” :D