6 ms·

I didn't realize that the word "bokeh" is fairly new to English-language discussion of photography -- only since about 1998 ( https://en.wikipedia.org/wiki/Boke

I didn't realize that the word "bokeh" is fairly new to English-language discussion of photography -- only since about 1998 ( https://en.wikipedia.org/wiki/Bokeh https://en.wikipedia.org/wiki/Bokeh ). I had this series of thoughts about smartphone camera "portrait mode" which were like this: (1) Ha, pretty funny that we're spending all this human effort and R&D time to try to realistically simulate what is essentially a defect in camera lenses that makes them unable to have a wide enough focal plane to accurately represent what is in front of them. (2) But wait, isn't that snobbish and a little shortsighted? (ha ha). What is a camera for anyway, is it making art or faithfully what it sees or something else? There's whole academic disciplines about this hmm okay. (3) Well, no, I mean, a photo which has been selectively blurred like this has _less information_ than we started with, so it's objectively telling us less, and that's why it's funny, that we are spending so much time removing information. (4) But maybe that's just the same as editing. Surely a good editor of texts takes a lot of information and removes information to make a better story; in the same way surely a "portrait mode" is really about figuring out what the most important parts are of a picture and selectively obscuring the rest, so that the 'good stuff' can shine through. That's important and hard; with an 85mm lens at f1.8 you'd do it by controlling focus carefully, but doing it after the fact is definitely interesting. So okay, the reason this feature is interesting (and hard) is that software is trying to guess what parts of a picture are noise that should be gently deemphasized. Now, it seems like the best way that software companies can think of to deemphasize unimportant parts of a picture is to mimic what a physical camera does with a portrait lens -- choose parts not on the same focal plane, pretend those parts were out of focus, then soften the image in the areas that "should have been" out of focus and blur them. Now that they have shipped products that do that I'd be interested to see what the folks at Apple & Android do next. Do their teams still exist? Have they thought about how they would emphasize the important parts of a portrait if they weren't required to simulate the mechanical effect of a camera lens? What else could they do, maybe a little more creatively, to make portraits shine? (And have they reinvented anything that Photoshop plugin makers have been doing for a decade?)

shaklee3 8y ago
I think if this feature was perfected it's far better than what an SLR does. I would much rather take the picture with as much information as possible, and blur to make the subject stand out later. It's kind of the same reason people shoot in raw.
- ArtWomb 8y ago
  It's kind of the same reason people shoot in raw Am certainly in this camp. State-of-the-art UltraHD 60FPS HDR looks terrific https://www.youtube.com/watch?v=qO6-1u0wfPk https://www.youtube.com/watch?v=qO6-1u0wfPk Wish there was a simple way to just turn off everything in mobile cameras and instantly enter raw mode at highest resolution.
devadvance 8y ago
It's important to keep in mind that while it seems to be approached as "simulating" the effects traditionally achieved with mechanical effects of a camera, it has a basis in fundamental optics. Creating a sense of focus in a portrait with a camera lens is analogous to how the optics in our eyes work. Thus, this particular effect is more natural than some of the more Photoshop-esque techniques. A good example of "what else" might be the color pop feature in the Google Photos app. It uses the depth information to selectively decolor the "out of focus" portions of the photo: https://www.androidpolice.com/2018/05/11/google-photos-color-pop-images-rolling/ https://www.androidpolice.com/2018/05/11/google-photos-color...
- mherdeg 8y ago
  Yeah that's a good point. Until a month ago when I got an eye exam and glasses, I didn't really understand that the way a telephoto lens worked at wide apertures was very similar to the way the eye was supposed to work at _most_ distances. (I was familiar with focusing on one thing and other things being out of focus, but with lens corrections this now works much better at 10+ meter distances, which is something I really didn't appreciate until I tried it.)
teawrecks 8y ago
All good points. One point you touched on that I'd like to highlight and tie back to your earlier point about removing information is that traditional cameras are bending light rays along paths that smaller phone lenses simply can't, and then they trivially record the information about how light behaved in that moment. To simulate this given only a 2D projection of some light at a certain point means trying to reconstruct the scene and apply (probably flawed, and certainly imprecise) physical models of how we understand light to behave, and then try to guess how different the image would have turned out in the presence of a theoretical lens affecting the incoming rays. In other words, there IS missing information, a LOT of it. And approximating that information to any degree of accuracy after the fact is intractable, if not impossible. So the fact that we can train a machine to generate a result this believable is just...unfathomably amazing. A human brain, with it's inherent understanding of how the physical world behaves, really is the only other thing that could feasibly accomplish a remotely similar task.
jhayward 8y ago
> Well, no, I mean, a photo which has been selectively blurred like this has _less information_ than we started with, so it's objectively telling us less, and that's why it's funny, that we are spending so much time removing information.* You are confusing entropy with information. You can increase the information in an image by selectively removing everything that isn't relevant to some aspect of its viewing. By focusing on a subject our visual and cognitive systems don't have to do that work.
nl 8y ago
Now that they have shipped products that do that I'd be interested to see what the folks at Apple & Android do next. Have you seen Night Sight on Pixel phones? It’s pretty amazing.
visarga 8y ago
> What else could they do, maybe a little more creatively, to make portraits shine? Neural nets can also colorise or even transform photos into the style of paintings. They can detect the better image of a series of shots. They can remove noise, watermarks and even reconstitute arbitrary missing pieces of an image. They can remove whole sections of an image bringing closer far away objects or shortening empty spaces between objects, in a seamless fashion. They can imagine things that are not in the training set (GANs). They can take your face and transfer it (animate it) to another person in a video. They can make summer into winter and vice-versa, in a realistic way. Really it's an explosion of things we can do now that before were impossible.

shaklee3 8y ago

I think if this feature was perfected it's far better than what an SLR does. I would much rather take the picture with as much information as possible, and blur to make the subject stand out later. It's kind of the same reason people shoot in raw.

ArtWomb 8y ago
It's kind of the same reason people shoot in raw Am certainly in this camp. State-of-the-art UltraHD 60FPS HDR looks terrific https://www.youtube.com/watch?v=qO6-1u0wfPk https://www.youtube.com/watch?v=qO6-1u0wfPk Wish there was a simple way to just turn off everything in mobile cameras and instantly enter raw mode at highest resolution.

devadvance 8y ago

It's important to keep in mind that while it seems to be approached as "simulating" the effects traditionally achieved with mechanical effects of a camera, it has a basis in fundamental optics. Creating a sense of focus in a portrait with a camera lens is analogous to how the optics in our eyes work. Thus, this particular effect is more natural than some of the more Photoshop-esque techniques. A good example of "what else" might be the color pop feature in the Google Photos app. It uses the depth information to selectively decolor the "out of focus" portions of the photo: https://www.androidpolice.com/2018/05/11/google-photos-color-pop-images-rolling/ https://www.androidpolice.com/2018/05/11/google-photos-color...

mherdeg 8y ago
Yeah that's a good point. Until a month ago when I got an eye exam and glasses, I didn't really understand that the way a telephoto lens worked at wide apertures was very similar to the way the eye was supposed to work at _most_ distances. (I was familiar with focusing on one thing and other things being out of focus, but with lens corrections this now works much better at 10+ meter distances, which is something I really didn't appreciate until I tried it.)

teawrecks 8y ago

All good points. One point you touched on that I'd like to highlight and tie back to your earlier point about removing information is that traditional cameras are bending light rays along paths that smaller phone lenses simply can't, and then they trivially record the information about how light behaved in that moment. To simulate this given only a 2D projection of some light at a certain point means trying to reconstruct the scene and apply (probably flawed, and certainly imprecise) physical models of how we understand light to behave, and then try to guess how different the image would have turned out in the presence of a theoretical lens affecting the incoming rays. In other words, there IS missing information, a LOT of it. And approximating that information to any degree of accuracy after the fact is intractable, if not impossible. So the fact that we can train a machine to generate a result this believable is just...unfathomably amazing. A human brain, with it's inherent understanding of how the physical world behaves, really is the only other thing that could feasibly accomplish a remotely similar task.

jhayward 8y ago

> Well, no, I mean, a photo which has been selectively blurred like this has _less information_ than we started with, so it's objectively telling us less, and that's why it's funny, that we are spending so much time removing information.* You are confusing entropy with information. You can increase the information in an image by selectively removing everything that isn't relevant to some aspect of its viewing. By focusing on a subject our visual and cognitive systems don't have to do that work.

nl 8y ago

Now that they have shipped products that do that I'd be interested to see what the folks at Apple & Android do next. Have you seen Night Sight on Pixel phones? It’s pretty amazing.

visarga 8y ago

> What else could they do, maybe a little more creatively, to make portraits shine? Neural nets can also colorise or even transform photos into the style of paintings. They can detect the better image of a series of shots. They can remove noise, watermarks and even reconstitute arbitrary missing pieces of an image. They can remove whole sections of an image bringing closer far away objects or shortening empty spaces between objects, in a seamless fashion. They can imagine things that are not in the training set (GANs). They can take your face and transfer it (animate it) to another person in a video. They can make summer into winter and vice-versa, in a realistic way. Really it's an explosion of things we can do now that before were impossible.