4 ms·

This is absolutely awesome to see, having done (fairly modest - masters level) research in this field in the past. For the uninitiated, I'll try and summarise

This is absolutely awesome to see, having done (fairly modest - masters level) research in this field in the past. For the uninitiated, I'll try and summarise my understanding of the general concept of computational photography: A 'traditional' camera uses a physical system of lenses to focus light to a single point (area really), where the light is captured and recorded, using either film or a digital sensor. The captured light information is then processed in whatever necessary way depending on the capture mechanism to ultimately produce a photograph. But all that's produced is one, singular photograph, that's an exact mapping of the physical properties of the lens system. Computational photography tries to virtualise aspects of this process by modelling the system's optics, so that extra information can be inferred from the captured light using these models. This extra information can reveal things about the scene being captured (more the computer vision side) or be used to post-process different effects into the image after the fact in an intelligent way that uses real information about the scene (more the photography, art, side) such that the captured information represents a space of possible images of the scene, rather than just one singular point in that space (one photograph). To increase the quality and quantity of this information, extra sensors, better sensors, different sensors (light field capture etc) are used, and modifications to the system can be made such as coded apertures, adjusting parameters like focal length and exposure time by known amounts, and collecting samples over a range of these modifications. Looks like a coded aperture is used in this technique for example but this article doesn't go into a lot of detail on that. When you start thinking about the possibilities of working with image 'spaces' instead of photographs, both for practical computer vision applications and just plain cool art applications, it quickly becomes obvious what a fascinating and exciting (and important) field this is, and will be in the future. It's really, truly, awesome to see an application of this relatively new field of research which could hit the mainstream in a big way - namely thinner, lighter, solid state cameras in phones, which it looks like this research could ultimately lead to. Bravo to everyone involved!

Retric 11y ago
I would suggest 'traditional' cameras and telescopes are all about being able to trace back the origin of light. So, if light hit part if your sensor X,Y it must have come from location A,B. Pin hole cameras work just fine for this, all the lenses are just trying to collect more light without creating excessive much distortion. Computational photography instead focuses on modeling things so if you observed A, B, and C, then Y must be true. Which enables things like CT scans to work where traditional X ray force you to look at shadows.
fallingfrog 11y ago
So - score my understanding of this. A lens performs a fourier transform, right? So the idea here is that instead of doing the transform with a lens, we grab the raw information and do an fft in software?
- davnicwil 11y ago
  I'm not sure on the implementation details of this particular method, and not really qualified to score anyone's understanding of anything in this field :-) but the shape of what you say is correct - you're modelling the physics of what happens in a lens system X (X can be a range of systems of course, not necessarily one particular system, but you get the idea), and in an alternate system Y (in this case Y is no lens, but a coded aperture used to add extra contextual information to the captured light) so you can use software to do a virtualised mapping of what your light captured from the scene via system Y would have looked like if had instead passed through system X - in other words, estimating virtually what a photo of the scene would have looked like if you'd used a lens, even though you've actually got no lens. Very clever application indeed!
- Ono-Sendai 11y ago
  A lens doesn't compute the fourier transform of the image, it inverts it (geometrically speaking). On a wave level (far-field diffraction) it does something like the fourier transform.
  - tortle 11y ago
    A lens is a convolution. Imagine if a photograph was a 1D signal, then a lens would be a convolution of a signal with a square wave (Lens sliced along one plane). Now, typically when a square wave is convolved with a signal, if you look at the magnitude of the fourier transform, you'll notice that you are multiplying the frequencies of the signal with a sinc function. Sinc functions hit zero amplitudes at various points, thereby destroying information when multiplied. Therefore, if you could create a lens (known as a mask), which didn't destroy signal, but instead was more like a band-pass filter, then you could effectively recover the signal and adjust for amplitude changes digitally. Now take this model to the 3D and you've just created a lightfield.

Retric 11y ago

I would suggest 'traditional' cameras and telescopes are all about being able to trace back the origin of light. So, if light hit part if your sensor X,Y it must have come from location A,B. Pin hole cameras work just fine for this, all the lenses are just trying to collect more light without creating excessive much distortion. Computational photography instead focuses on modeling things so if you observed A, B, and C, then Y must be true. Which enables things like CT scans to work where traditional X ray force you to look at shadows.

fallingfrog 11y ago

So - score my understanding of this. A lens performs a fourier transform, right? So the idea here is that instead of doing the transform with a lens, we grab the raw information and do an fft in software?

davnicwil 11y ago
I'm not sure on the implementation details of this particular method, and not really qualified to score anyone's understanding of anything in this field :-) but the shape of what you say is correct - you're modelling the physics of what happens in a lens system X (X can be a range of systems of course, not necessarily one particular system, but you get the idea), and in an alternate system Y (in this case Y is no lens, but a coded aperture used to add extra contextual information to the captured light) so you can use software to do a virtualised mapping of what your light captured from the scene via system Y would have looked like if had instead passed through system X - in other words, estimating virtually what a photo of the scene would have looked like if you'd used a lens, even though you've actually got no lens. Very clever application indeed!
Ono-Sendai 11y ago
A lens doesn't compute the fourier transform of the image, it inverts it (geometrically speaking). On a wave level (far-field diffraction) it does something like the fourier transform.
- tortle 11y ago
  A lens is a convolution. Imagine if a photograph was a 1D signal, then a lens would be a convolution of a signal with a square wave (Lens sliced along one plane). Now, typically when a square wave is convolved with a signal, if you look at the magnitude of the fourier transform, you'll notice that you are multiplying the frequencies of the signal with a sinc function. Sinc functions hit zero amplitudes at various points, thereby destroying information when multiplied. Therefore, if you could create a lens (known as a mask), which didn't destroy signal, but instead was more like a band-pass filter, then you could effectively recover the signal and adjust for amplitude changes digitally. Now take this model to the 3D and you've just created a lightfield.