4 ms·

I disagree with the conclusion. It looks like two images—one with the face obscured, one with the face visible—were combined. Notice that both the leaf and th

I disagree with the conclusion. It looks like two images—one with the face obscured, one with the face visible—were combined. Notice that both the leaf and the person are sharply in focus, and there’s a distinct circular, blurry outline around the face where a transition occurs between the two images. No, the leaf in the photo isn’t fabricated by iOS. No, iOS didn’t copy a leaf from elsewhere in the image to continue the pattern. But the conclusion, “iPhone Camera app did not replace person’s head with a leaf” is probably incorrect: it looks to me like it did, in fact, replace the person’s head with a leaf. It just so happens that the leaf was really there from the perspective of one lens or photo. Wasn’t this documented as intended behavior on newer iPhones? Images from different lenses will be combined and faces replaced with clearer versions in some scenarios. In this particular instance, the “clearer” version was a leaf, as there was a leaf in the way. This isn’t really bad thing; it’s an understandable bug in a piece of technology that usually works amazingly well. However, it does mean there’s room for improvement.

karlshea 5y ago
> No, the leaf in the photo isn’t fabricated by iOS. No, iOS didn’t copy a leaf from elsewhere in the image to continue the pattern. Those two things are exactly what everyone was assuming happened. So while you're right about what is going on with the camera behavior, I think you're disagreeing about a completely separate conclusion than everyone else arrived at. > It just so happens that the leaf was really there from the perspective of one lens or photo. In the original thread the photographer was fairly certain there weren't leaves in front, which is what started all of this.
- thaumasiotes 5y ago
  >> No, the leaf in the photo isn’t fabricated by iOS. No, iOS didn’t copy a leaf from elsewhere in the image to continue the pattern. > Those two things are exactly what everyone was assuming happened. What? Look at the other discussion thread: > The cause of this is image-stacking. > The phone takes ~20 frames, over 0.2 seconds. In that time, lots of people and things in the frame move. > Optical flow is used to track all moving parts of the image, and then 'undo' any movement, aligning all parts of the image. > Then the frames are combined, usually by, for each pixel, taking something like the median or throwing out outliers and using the average. > When the optical flow fails to track an object in more than half the frames, the 'outliers' that are thrown out can in fact be the image content you wanted. > It happens with leaves a lot because they can flutter fast from one frame to the next, so tracking each individual leaf is hard. A few bad tracking results on more than half the frames, and all you end up seeing is leaves where there should be a face.. ( https://news.ycombinator.com/item?id=29742539 https://news.ycombinator.com/item?id=29742539 ) There was nobody arguing that the phone hallucinated the leaf. The problem is that the phone removed the person's head so that it wouldn't pull focus from the leaf.
  - karlshea 5y ago
    I meant the original Twitter thread. But even here everyone was wondering where the leaf came from because the original photographer was insistent there weren't any in the foreground. The general consensus was that yes, it was image stacking, but perhaps it was moving a leaf that was in the background over her face. That turned out not to be the case after the updated tweet mentioned there was a tree in the foreground.

karlshea 5y ago

> No, the leaf in the photo isn’t fabricated by iOS. No, iOS didn’t copy a leaf from elsewhere in the image to continue the pattern. Those two things are exactly what everyone was assuming happened. So while you're right about what is going on with the camera behavior, I think you're disagreeing about a completely separate conclusion than everyone else arrived at. > It just so happens that the leaf was really there from the perspective of one lens or photo. In the original thread the photographer was fairly certain there weren't leaves in front, which is what started all of this.

thaumasiotes 5y ago
>> No, the leaf in the photo isn’t fabricated by iOS. No, iOS didn’t copy a leaf from elsewhere in the image to continue the pattern. > Those two things are exactly what everyone was assuming happened. What? Look at the other discussion thread: > The cause of this is image-stacking. > The phone takes ~20 frames, over 0.2 seconds. In that time, lots of people and things in the frame move. > Optical flow is used to track all moving parts of the image, and then 'undo' any movement, aligning all parts of the image. > Then the frames are combined, usually by, for each pixel, taking something like the median or throwing out outliers and using the average. > When the optical flow fails to track an object in more than half the frames, the 'outliers' that are thrown out can in fact be the image content you wanted. > It happens with leaves a lot because they can flutter fast from one frame to the next, so tracking each individual leaf is hard. A few bad tracking results on more than half the frames, and all you end up seeing is leaves where there should be a face.. ( https://news.ycombinator.com/item?id=29742539 https://news.ycombinator.com/item?id=29742539 ) There was nobody arguing that the phone hallucinated the leaf. The problem is that the phone removed the person's head so that it wouldn't pull focus from the leaf.
- karlshea 5y ago
  I meant the original Twitter thread. But even here everyone was wondering where the leaf came from because the original photographer was insistent there weren't any in the foreground. The general consensus was that yes, it was image stacking, but perhaps it was moving a leaf that was in the background over her face. That turned out not to be the case after the updated tweet mentioned there was a tree in the foreground.