7 ms·

A Lens-Less Camera Built Specially for AI and Computer Vision Programs

119 points by Osiris30 8y ago

davissorenson 8y ago
Link to paper in question (with the images you wished were in the article): https://www.osapublishing.org/oe/abstract.cfm?uri=oe-26-18-22826 https://www.osapublishing.org/oe/abstract.cfm?uri=oe-26-18-2... Edit: Fixed link
- sinstein 8y ago
  Getting an empty response for this link. Do you have an alternate?
  - daliusd 8y ago
    This seems to work https://www.osapublishing.org/oe/abstract.cfm?uri=oe-26-18-22826 https://www.osapublishing.org/oe/abstract.cfm?uri=oe-26-18-2...
tlarkworthy 8y ago
Looks really cool but might be sensitive to tiny perturbations because the inverse function is over fitted to the optics which are basically random noise. I am thinking temperature changes, scratches and knocks.
- alanbernstein 8y ago
  Maybe less of an issue for glass than acrylic?
- vincnetas 8y ago
  Also glass vibrations from random ambient noise.
ariehkovler 8y ago
Another trend towards lower-power and lower-fidelity in AI-driven computer recognition systems. Last week, IBM announced it was developing better 8-bit AI hardware as an alternative to 16- and 32-bit systems, speeding computations and reducing hardware demands. https://venturebeat.com/2018/12/02/ibms-8-bit-ai-training-technique-is-two-to-four-times-faster-while-retaining-accuracy/ https://venturebeat.com/2018/12/02/ibms-8-bit-ai-training-te... I guess once the human aspect is removed, the sensor and processing systems can be tailored for the computer aspect of recognition tasks, which isn't necessarily as powerful or high-definition as we expect.
- around_here 8y ago
  That's the scary part though. A lot of the examples out there are meant to see when something is out of the ordinary and then be interpreted by people.
wnkrshm 8y ago
Extended light sources like LEDs close to the planar sheet of course couple to some TIR guided ray paths in the glass. But seeing far away objects like with a camera ... probably only those that are visible at larger angles to the plane normal. The camera is probably blind to the angular space close to the plane normal.
occamschainsaw 8y ago
This reminds me of The Expanse (Nemesis Games). In the future (~2300s), they use an "expert system" to recreate images of stars obscured by dust clouds. It creates a "computational lens that couldn't exist in the physical world". In one instance, the main characters use this technology to recreate images from a hand-terminal (smartphone) whose screen has been shattered. Relevant excerpt: https://books.google.com/books?id=A_-zBAAAQBAJ&lpg=PT239&ots=ZJ-J11PEam&dq=Nemesis%20Games%20expert%20system&pg=PT181#v=onepage&q&f=false https://books.google.com/books?id=A_-zBAAAQBAJ&lpg=PT239&ots... Read the paragraph after the lightning bolt. Genuinely excited about the applications of this technology to astronomy.
ctdonath 8y ago
I've been expecting some means of a display screen being the camera (pretty theory: phased array optics, a sensor at each display pixel). Interesting development.
cs702 8y ago
In the extreme, the logical extension of this kind of application is that over time every surface could be turned into a lens-less camera, by applying a thin photon detector film to it and using AI computation to transform low-quality photon sensor data into photos for human or machine consumption. It could very well be that in the future, any wall, glass pane, or other surface could be cheaply configured to watch/record its surroundings.
1024core 8y ago
Direct PDF: https://www.osapublishing.org/DirectPDFAccess/F9AF468F-C6F8-681A-BDD261F25D185990_396362/oe-26-18-22826.pdf?da=1&id=396362&seq=0&mobile=no https://www.osapublishing.org/DirectPDFAccess/F9AF468F-C6F8-...
nathan_long 8y ago
> So, Menon asks, “If machines are going to be seeing these images and video more than humans, then why don’t we think about redesigning the cameras purely for machines? Take the human out of the loop entirely, and think of cameras purely from a non-human perspective.” From a technical perspective, this is neat. From a societal perspective, I want the images an AI sees and the AI's decision-making process to be auditable by humans.
- dwighttk 8y ago
  yes. "If you were a computer you could see that this is a boot stamping on a human face forever..."
PaulHoule 8y ago
This reminds me of the "metric cameras" that are used in photogrammetry. Circa 1940 that field was mostly about measuring things on the ground but then it became 95% about aerial and space photography. Cameras intended for normal photography are optimized for sharpness, metric cameras are optimized for making correct measurement. That means the lens design is a bit different. I've been curious about the side-view camera in my Honda in which I'm not sure that the "focal plane" is really a plane. I guess some day I'll find a wrecked Honda at a junkyard and get a mirror assembly to play with.
- aidenn0 8y ago
  Most of those cameras have a fisheye like lens and then use a lot of post-processing to make a rectilinear image.
gleglegle 8y ago
Makes me think of Dawkins' demonstration of eye evolution, where he shows the usefulness of even simple photo-receptor designs: https://www.youtube.com/watch?v=Nwew5gHoh3E https://www.youtube.com/watch?v=Nwew5gHoh3E
carapace 8y ago
I don't have the link handy, but this reminds me of the 3D reconstruction from images taken through a pane of glass with water droplets on it. Each droplet acts like a little lens with a slightly different perspective and you can use those differences to reconstruct the 3D structure of the scene.
sebleon 8y ago
This is a really interesting concept. While this initial prototype is rudimentary, I agree with them in this important shift to building cameras "good enough" for machine-usage.

119 points by Osiris30 8y ago

Link to paper in question (with the images you wished were in the article): https://www.osapublishing.org/oe/abstract.cfm?uri=oe-26-18-22826 https://www.osapublishing.org/oe/abstract.cfm?uri=oe-26-18-2... Edit: Fixed link

sinstein 8y ago
Getting an empty response for this link. Do you have an alternate?
- daliusd 8y ago
  This seems to work https://www.osapublishing.org/oe/abstract.cfm?uri=oe-26-18-22826 https://www.osapublishing.org/oe/abstract.cfm?uri=oe-26-18-2...

tlarkworthy 8y ago

Looks really cool but might be sensitive to tiny perturbations because the inverse function is over fitted to the optics which are basically random noise. I am thinking temperature changes, scratches and knocks.

alanbernstein 8y ago
Maybe less of an issue for glass than acrylic?
vincnetas 8y ago
Also glass vibrations from random ambient noise.

ariehkovler 8y ago

Another trend towards lower-power and lower-fidelity in AI-driven computer recognition systems. Last week, IBM announced it was developing better 8-bit AI hardware as an alternative to 16- and 32-bit systems, speeding computations and reducing hardware demands. https://venturebeat.com/2018/12/02/ibms-8-bit-ai-training-technique-is-two-to-four-times-faster-while-retaining-accuracy/ https://venturebeat.com/2018/12/02/ibms-8-bit-ai-training-te... I guess once the human aspect is removed, the sensor and processing systems can be tailored for the computer aspect of recognition tasks, which isn't necessarily as powerful or high-definition as we expect.

around_here 8y ago
That's the scary part though. A lot of the examples out there are meant to see when something is out of the ordinary and then be interpreted by people.

wnkrshm 8y ago

Extended light sources like LEDs close to the planar sheet of course couple to some TIR guided ray paths in the glass. But seeing far away objects like with a camera ... probably only those that are visible at larger angles to the plane normal. The camera is probably blind to the angular space close to the plane normal.

occamschainsaw 8y ago

This reminds me of The Expanse (Nemesis Games). In the future (~2300s), they use an "expert system" to recreate images of stars obscured by dust clouds. It creates a "computational lens that couldn't exist in the physical world". In one instance, the main characters use this technology to recreate images from a hand-terminal (smartphone) whose screen has been shattered. Relevant excerpt: https://books.google.com/books?id=A_-zBAAAQBAJ&lpg=PT239&ots=ZJ-J11PEam&dq=Nemesis%20Games%20expert%20system&pg=PT181#v=onepage&q&f=false https://books.google.com/books?id=A_-zBAAAQBAJ&lpg=PT239&ots... Read the paragraph after the lightning bolt. Genuinely excited about the applications of this technology to astronomy.

ctdonath 8y ago

I've been expecting some means of a display screen being the camera (pretty theory: phased array optics, a sensor at each display pixel). Interesting development.

cs702 8y ago

In the extreme, the logical extension of this kind of application is that over time every surface could be turned into a lens-less camera, by applying a thin photon detector film to it and using AI computation to transform low-quality photon sensor data into photos for human or machine consumption. It could very well be that in the future, any wall, glass pane, or other surface could be cheaply configured to watch/record its surroundings.

1024core 8y ago

Direct PDF: https://www.osapublishing.org/DirectPDFAccess/F9AF468F-C6F8-681A-BDD261F25D185990_396362/oe-26-18-22826.pdf?da=1&id=396362&seq=0&mobile=no https://www.osapublishing.org/DirectPDFAccess/F9AF468F-C6F8-...

nathan_long 8y ago

> So, Menon asks, “If machines are going to be seeing these images and video more than humans, then why don’t we think about redesigning the cameras purely for machines? Take the human out of the loop entirely, and think of cameras purely from a non-human perspective.” From a technical perspective, this is neat. From a societal perspective, I want the images an AI sees and the AI's decision-making process to be auditable by humans.

dwighttk 8y ago
yes. "If you were a computer you could see that this is a boot stamping on a human face forever..."

PaulHoule 8y ago

This reminds me of the "metric cameras" that are used in photogrammetry. Circa 1940 that field was mostly about measuring things on the ground but then it became 95% about aerial and space photography. Cameras intended for normal photography are optimized for sharpness, metric cameras are optimized for making correct measurement. That means the lens design is a bit different. I've been curious about the side-view camera in my Honda in which I'm not sure that the "focal plane" is really a plane. I guess some day I'll find a wrecked Honda at a junkyard and get a mirror assembly to play with.

aidenn0 8y ago
Most of those cameras have a fisheye like lens and then use a lot of post-processing to make a rectilinear image.

gleglegle 8y ago

Makes me think of Dawkins' demonstration of eye evolution, where he shows the usefulness of even simple photo-receptor designs: https://www.youtube.com/watch?v=Nwew5gHoh3E https://www.youtube.com/watch?v=Nwew5gHoh3E

carapace 8y ago

I don't have the link handy, but this reminds me of the 3D reconstruction from images taken through a pane of glass with water droplets on it. Each droplet acts like a little lens with a slightly different perspective and you can use those differences to reconstruct the 3D structure of the scene.

sebleon 8y ago

This is a really interesting concept. While this initial prototype is rudimentary, I agree with them in this important shift to building cameras "good enough" for machine-usage.