3 ms·

Different purposes of computational capabilities in modern cameras vs. phones place them reliably in different categories. The former aim to computationally ad

Different purposes of computational capabilities in modern cameras vs. phones place them reliably in different categories. The former aim to computationally adjust settings and make it easier to capture raw scene data the way you intend. The “magic” happens on lower levels (e.g., tracking subject), and you generally feel in control of how it’s applied. The latter focus on computationally interpreting raw data and generating a display-referred image. The “magic” happens throughout the process, and is mostly a black box (recall that case where iPhone camera replaced human head with a leaf on a group photo). If you use phone camera to capture raw image, phone software may still be able to assist to a degree, but challenges arising from sensor size and lens limitations would become more obvious. (I would, however, watch out for the line becoming blurred if/when phones—hopefully not cameras!—start to capture computational raws. Potential improvements offered by applying extensive magic at unmosaiced raw signal level would for me be killed by that human-head-turned-to-leaf possibility, which would ruin phone photography for me.)

orbital-decay 5y ago
1. GCam, MotionCam etc were always able to output stacked raws. In case of raws, "computational" is pretty much a marketing buzzword that means very little, it's just good old exposure fusion and stacking in bayer domain, with in-camera alignment robust enough to shoot a long stack handheld. They pretty much have the same exposure/iso priority and controls you expect to simplify capturing the most of the scene data unchanged. There's even experimental RAW video recording in MotionCam, however ridiculous that might sound. 2. RAW is also pretty much meaningless with modern mirrorless since it's not literally raw. They also do a lot of stuff under the hood for years, including "computational" stuff like noise suppression or pixel shift/superresolution in bayer domain. So the line isn't just blurred, it doesn't exist at all. All this "computational" stuff is here precisely to capture more of the real data, not to draw something that never existed.
- goblin89 5y ago
  I don’t know what are things you reference in (1), some apps? In (2), either you are speaking about consumer point-and-shoots, or you are speaking of features that can be turned off, or more typically have to be turned on. (Have you ever tried to use pixel shift, by the way? What kind of magic do you think does it do, except for capturing multiple pixel-shifted exposures?) Mirrorless cameras I use get me a dumb array of raw light values, and I am free to apply (or not) noise reduction as I interpret the data. And if they were applying noise suppression, I would not put it anywhere in the same domain as computationally detecting and smoothening human skin or the like. The line is there, and at this time it’s pretty sharp. If you don’t believe, try some of the latest “dumb” pro mirrorless cameras with the latest iPhone and compare their respective raw captures.
  - orbital-decay 5y ago
    GCam is Google Camera, the app that started all this. Funny how everybody praises iPhone when Apple actually have been very slow to notice this trend, and their camera apps really sucked for many years. I hate GCam specifically because it requires lots of fiddling with mods and ad-hoc setups if you want full potential, but it doesn't change the fact that a "computational" camera outputs real scene data. >computationally detecting and smoothening human skin or the like Seems like there's some misunderstanding. I'm not talking about stuff like face detection or beautifying, computational bokeh, one-size-fits-all tonemapping etc. I'm talking about raw capture, which simply aligns and stacks many frames (discarding bogus ones) without demosaicing. There's really nothing more to it. Not much different than, say, Pentax' pixel shift. Similar process, same limitations and issues, no opaque magic that tries to think for you. I've used similar process for years with HDRMerge to shoot landscapes with my DSLRs. >If you don’t believe, try some of the latest “dumb” pro mirrorless cameras with the latest iPhone and compare their respective raw captures. I currently use the a7S III, but I fail to see how it's relevant. The difference is not due to them being dumb but because of having better sensors and optics, variable aperture, mechanical shutter etc. There's nothing smart or special in smartphone raw capture.
    - goblin89 5y ago
      We seem to mostly agree. Here’s the line—on one side of it are phones, able to produce decent images despite hardware limitations by applying black-box magic when interpreting raw captures (but not faring that well when used to shoot raw); on the other side are cameras, employing advanced tech (and sometimes, indeed, magic) to assist you with capturing raw signal of much higher fidelity and yielding full control as to its further interpretation. That’s what I meant by cameras being in a qualitatively different category, and why I wrote my comment arguing against the next iPhone somehow rendering mirrorless cameras useless. (The only point where it might become more of a debate in my eyes is if phones start applying similar extensive processing that they do, but at raw levels. There could be genuine gains there—but also unwelcome unpredictable adulteration of DNG captures.)
      - 5y ago
Xylakant 5y ago
> (recall that case where iPhone camera replaced human head with a leaf on a group photo) Though that leaf was just a leaf, hanging in the wrong place - no magic involved, not the iPhones fault. https://petapixel.com/2022/01/03/the-mystery-of-an-iphone-replacing-a-womans-head-with-a-leaf/ https://petapixel.com/2022/01/03/the-mystery-of-an-iphone-re...
- goblin89 5y ago
  Didn’t know this was debunked, my faith in iphoneography was restored a little! Still, I occasionally notice artifacts of computational processing—most recently, wrong color of a building (it somehow decided it’s yellow, instead of reddish beige), other times some skin improvements, eye sharpening and so on. (I don’t shoot in raw on my phone though.)

orbital-decay 5y ago

1. GCam, MotionCam etc were always able to output stacked raws. In case of raws, "computational" is pretty much a marketing buzzword that means very little, it's just good old exposure fusion and stacking in bayer domain, with in-camera alignment robust enough to shoot a long stack handheld. They pretty much have the same exposure/iso priority and controls you expect to simplify capturing the most of the scene data unchanged. There's even experimental RAW video recording in MotionCam, however ridiculous that might sound. 2. RAW is also pretty much meaningless with modern mirrorless since it's not literally raw. They also do a lot of stuff under the hood for years, including "computational" stuff like noise suppression or pixel shift/superresolution in bayer domain. So the line isn't just blurred, it doesn't exist at all. All this "computational" stuff is here precisely to capture more of the real data, not to draw something that never existed.

goblin89 5y ago
I don’t know what are things you reference in (1), some apps? In (2), either you are speaking about consumer point-and-shoots, or you are speaking of features that can be turned off, or more typically have to be turned on. (Have you ever tried to use pixel shift, by the way? What kind of magic do you think does it do, except for capturing multiple pixel-shifted exposures?) Mirrorless cameras I use get me a dumb array of raw light values, and I am free to apply (or not) noise reduction as I interpret the data. And if they were applying noise suppression, I would not put it anywhere in the same domain as computationally detecting and smoothening human skin or the like. The line is there, and at this time it’s pretty sharp. If you don’t believe, try some of the latest “dumb” pro mirrorless cameras with the latest iPhone and compare their respective raw captures.
- orbital-decay 5y ago
  GCam is Google Camera, the app that started all this. Funny how everybody praises iPhone when Apple actually have been very slow to notice this trend, and their camera apps really sucked for many years. I hate GCam specifically because it requires lots of fiddling with mods and ad-hoc setups if you want full potential, but it doesn't change the fact that a "computational" camera outputs real scene data. >computationally detecting and smoothening human skin or the like Seems like there's some misunderstanding. I'm not talking about stuff like face detection or beautifying, computational bokeh, one-size-fits-all tonemapping etc. I'm talking about raw capture, which simply aligns and stacks many frames (discarding bogus ones) without demosaicing. There's really nothing more to it. Not much different than, say, Pentax' pixel shift. Similar process, same limitations and issues, no opaque magic that tries to think for you. I've used similar process for years with HDRMerge to shoot landscapes with my DSLRs. >If you don’t believe, try some of the latest “dumb” pro mirrorless cameras with the latest iPhone and compare their respective raw captures. I currently use the a7S III, but I fail to see how it's relevant. The difference is not due to them being dumb but because of having better sensors and optics, variable aperture, mechanical shutter etc. There's nothing smart or special in smartphone raw capture.
  - goblin89 5y ago
    We seem to mostly agree. Here’s the line—on one side of it are phones, able to produce decent images despite hardware limitations by applying black-box magic when interpreting raw captures (but not faring that well when used to shoot raw); on the other side are cameras, employing advanced tech (and sometimes, indeed, magic) to assist you with capturing raw signal of much higher fidelity and yielding full control as to its further interpretation. That’s what I meant by cameras being in a qualitatively different category, and why I wrote my comment arguing against the next iPhone somehow rendering mirrorless cameras useless. (The only point where it might become more of a debate in my eyes is if phones start applying similar extensive processing that they do, but at raw levels. There could be genuine gains there—but also unwelcome unpredictable adulteration of DNG captures.)
    - 5y ago

Xylakant 5y ago

> (recall that case where iPhone camera replaced human head with a leaf on a group photo) Though that leaf was just a leaf, hanging in the wrong place - no magic involved, not the iPhones fault. https://petapixel.com/2022/01/03/the-mystery-of-an-iphone-replacing-a-womans-head-with-a-leaf/ https://petapixel.com/2022/01/03/the-mystery-of-an-iphone-re...

goblin89 5y ago
Didn’t know this was debunked, my faith in iphoneography was restored a little! Still, I occasionally notice artifacts of computational processing—most recently, wrong color of a building (it somehow decided it’s yellow, instead of reddish beige), other times some skin improvements, eye sharpening and so on. (I don’t shoot in raw on my phone though.)