5 ms·

I don't know for sure but the code may be derived from or related to Marc Levoy's SynthCam: https://sites.google.com/site/marclevoy/ https://sites.google.com/si

by theoh 13y ago

I don't know for sure but the code may be derived from or related to Marc Levoy's SynthCam: https://sites.google.com/site/marclevoy/ https://sites.google.com/site/marclevoy/ It does similar things, and in fact I could believe it if the Google app was just a dumbed down version of the functionality intended to be usable by a wider audience.

aray 13y ago
There have been multiple implementations of this class of algorithm (collectively "synthetic aperture") using different techniques. Edit: below mentions the HTC "double camera" phones, and in fact it's also possible to create synthetic aperture photography with multiple cameras instead of moving a single camera to multiple positions. Then you have the added advantage of not needing to assume a static scene! But a single-camera algorithm is great for rolling out to the common devices most of us have in our pockets. Marc Levoy's SynthCam used a circular wiping motion, and attempted to "paint" the 3d space occupied by your theoretical processed aperture. This Google Camera only requires a single linear move, and processes the rest!
- darkmighty 13y ago
  It looks like this isn't creating a synthetic aperture, it's just creating a depth map and applying a depth-dependent blur to a single frame.
  - mark-r 13y ago
    According to the text the blur itself is simulating an actual lens. Without seeing the exact algorithm it's hard to know how accurate that claim is. There are many ways to do a blur and I know what I'd use if I were trying to generate good bokeh.
- theoh 13y ago
  Point taken. Arguably it's a case of David Pye's notion of the workmanship of certainty (in this case heavy-lifting computer vision) replacing the workmanship of risk (making adept movements of camera) But if it works better under all conditions, fine. http://en.m.wikipedia.org/wiki/David_Pye_(furniture) http://en.m.wikipedia.org/wiki/David_Pye_(furniture)
kbrower 13y ago
They did hire Marc Levoy. I have been hoping for this for awhile. Can't wait for the next iteration. There are certain things you can't replicate with just a single tiny camera, but they are doing a great job! https://news.ycombinator.com/item?id=6483182 https://news.ycombinator.com/item?id=6483182
- jychang 13y ago
  > There are certain things you can't replicate with just a single tiny camera MOST DoF and bokeh effects in photography can't be replicated with a single small camera. As a photographer, bokeh is surprisingly difficult to fake, and looks glaringly bad when you notice. The blur effect is due to focal distance ratio differences, and it's very difficult to determine the distance in software. Hell, it's hard to determine it with 2 lenses, as the HTC implementation does. If you want to compare what "fake bokeh" looks like compared to real stuff, you can look at http://www.trustedreviews.com/opinions/htc-one-m8-camera-vs-a-proper-camera http://www.trustedreviews.com/opinions/htc-one-m8-camera-vs-... for a review of the HTC One M8, which has a 2-lens setup. Look at this picture, for example: http://static.trustedreviews.com/94/00002b836/8517/blue-htc-rear.jpg http://static.trustedreviews.com/94/00002b836/8517/blue-htc-... For most software, it's extremely difficult, even with distance data, to separate the bush in the back from the blue toy. As a result... messy looking blur.
  - kbrower 13y ago
    I look forward to a review of this software vs HTC vs dSLR. My first instinct is this is much less artificial.
  - joshuak 13y ago
    Very true. We do this kind of work in VFX all the time to reduce CG rendering costs for blurs that are expensive in 3d but cheap in 2d. Even with access to sub pixel (4x, 8x etc.) depth maps there are always lots of issues to deal with around edges. But given that jpeg is good enough for most people I'm sure these types of tools are too.
  - thezilch 13y ago

by theoh 13y ago

aray 13y ago

There have been multiple implementations of this class of algorithm (collectively "synthetic aperture") using different techniques. Edit: below mentions the HTC "double camera" phones, and in fact it's also possible to create synthetic aperture photography with multiple cameras instead of moving a single camera to multiple positions. Then you have the added advantage of not needing to assume a static scene! But a single-camera algorithm is great for rolling out to the common devices most of us have in our pockets. Marc Levoy's SynthCam used a circular wiping motion, and attempted to "paint" the 3d space occupied by your theoretical processed aperture. This Google Camera only requires a single linear move, and processes the rest!

darkmighty 13y ago
It looks like this isn't creating a synthetic aperture, it's just creating a depth map and applying a depth-dependent blur to a single frame.
- mark-r 13y ago
  According to the text the blur itself is simulating an actual lens. Without seeing the exact algorithm it's hard to know how accurate that claim is. There are many ways to do a blur and I know what I'd use if I were trying to generate good bokeh.
theoh 13y ago
Point taken. Arguably it's a case of David Pye's notion of the workmanship of certainty (in this case heavy-lifting computer vision) replacing the workmanship of risk (making adept movements of camera) But if it works better under all conditions, fine. http://en.m.wikipedia.org/wiki/David_Pye_(furniture) http://en.m.wikipedia.org/wiki/David_Pye_(furniture)

kbrower 13y ago

They did hire Marc Levoy. I have been hoping for this for awhile. Can't wait for the next iteration. There are certain things you can't replicate with just a single tiny camera, but they are doing a great job! https://news.ycombinator.com/item?id=6483182 https://news.ycombinator.com/item?id=6483182

jychang 13y ago
> There are certain things you can't replicate with just a single tiny camera MOST DoF and bokeh effects in photography can't be replicated with a single small camera. As a photographer, bokeh is surprisingly difficult to fake, and looks glaringly bad when you notice. The blur effect is due to focal distance ratio differences, and it's very difficult to determine the distance in software. Hell, it's hard to determine it with 2 lenses, as the HTC implementation does. If you want to compare what "fake bokeh" looks like compared to real stuff, you can look at http://www.trustedreviews.com/opinions/htc-one-m8-camera-vs-a-proper-camera http://www.trustedreviews.com/opinions/htc-one-m8-camera-vs-... for a review of the HTC One M8, which has a 2-lens setup. Look at this picture, for example: http://static.trustedreviews.com/94/00002b836/8517/blue-htc-rear.jpg http://static.trustedreviews.com/94/00002b836/8517/blue-htc-... For most software, it's extremely difficult, even with distance data, to separate the bush in the back from the blue toy. As a result... messy looking blur.
- kbrower 13y ago
  I look forward to a review of this software vs HTC vs dSLR. My first instinct is this is much less artificial.
- joshuak 13y ago
  Very true. We do this kind of work in VFX all the time to reduce CG rendering costs for blurs that are expensive in 3d but cheap in 2d. Even with access to sub pixel (4x, 8x etc.) depth maps there are always lots of issues to deal with around edges. But given that jpeg is good enough for most people I'm sure these types of tools are too.
- thezilch 13y ago