3 ms·

Sure, some of the black box processing that phone camera does relies on known algorithms. The difference is that it also makes use of ML, which IMO either play

Sure, some of the black box processing that phone camera does relies on known algorithms. The difference is that it also makes use of ML, which IMO either plays a significant role or will do so in near term. > auto white balance, tonemapping, skin smoothing, other features This, for example, I see as an opaque step with nondeterministic outcomes, which if you take away from phones and shoot raw reveals how far they are away from mirrorless cameras. (Cameras that shoot raw do not do this all that well, nor are they expected to.) > There's nothing sacred or qualitatively different about single-shot capture I stack exposures too. Whatever process works, works. The difference is that my flow is A -> N -> X, my camera’s built-in processing is A -> M -> Y, while ML-powered black-box compute in a phone does A -> ? -> R today or A -> ? -> Z tomorrow. Due to its automatic nature, I see this processing as taking more of the creative part of photographer’s role, unlike compute in cameras where it is a capture-time assistant. It will have to play a bigger and bigger role in how phone camera output improves in near future, as there is not that much space for enlarging the sensor or the lens; and as a user you either let it do the magic or you end up with a very mediocre camera. It is not inherently better or worse, but it relates to photography in the sense that I appreciate it in kind of as Nvidia Canvas et al. relate to hand-created art: different tools, different goals.

orbital-decay 5y ago
It still seems we're talking about completely different things, or maybe you're trying to derail the conversation... Both ML-based processing and tonemapping take place after demosaicing to produce the final JPEG. RAW capture is just your typical laplacian pyramid stacking/exposure fusion with motion compensation and sensor-specific calibration. That's the main contributing factor that somewhat compensates for the sensor size, not some kind of ML processing. If you think that ML is somehow involved here, I have zero idea where you are getting it from. The entire point of pre-demosaicing pipeline is to robustly converge the captured stack into a neutral scene-referred baseline with provably better SNR and precise photometric parameters. And that's what you get in a resulting DNG, not some neural tonemapped RGB image (which is not even created at this point). Perhaps you should look at the pipeline yourself, here's the source code for a typical "computational imaging" camera app that produces stacked RAWs, for example. [1] And here's what Google Camera does [2] [3] (you can ignore everything that comes after demosaicing). Google's particular method is also partially implemented for arbitrary RAWs as third-party software, so you can stack any burst capture from your camera with the same method smartphones do. [4] (there's much less benefit for normal sized sensors, for obvious reasons) [1] https://github.com/mirsadm/motioncam/ https://github.com/mirsadm/motioncam/ [2] https://static.googleusercontent.com/media/hdrplusdata.org/en//hdrplus.pdf https://static.googleusercontent.com/media/hdrplusdata.org/e... (PDF) [3] https://static.googleusercontent.com/media/hdrplusdata.org/en//hdrplus_supp.pdf https://static.googleusercontent.com/media/hdrplusdata.org/e... (PDF) [4] https://github.com/martin-marek/hdr-plus-swift https://github.com/martin-marek/hdr-plus-swift
- deleted 5y ago
  [deleted]
- goblin89 5y ago
  > It still seems we're talking about completely different things, or maybe you're trying to derail the conversation... I am making literally the same point I made when replying to jhoechtl: phones and cameras are different beasts, and the next iPhone is unlikely to obsolete semi-pro mirrorless cameras. Judging by your tone, you are dead set at proving me wrong, but I can’t see a meaningful argument. Why do you keep reverting to ML involvement at pre-demosaicing level? “Magic” computed raws is not something I claimed to happen now, nor am I certain it ever will—I just speculated we are logically headed there and if we do get there it may affect my view at phones vs. mirrorless. So no, not interested in a hypothetical typical “dumb” camera app or raw capture from a phone. I mean the computationally more advanced capture (like the one done by bog standard Camera) that produces good images, which is what makes people think iPhone can rival a mirrorless. If you take away (by shooting raw or using a dumber app) semantic masks and all the magic it uses to obtain display-referred data for you, you just get a mediocre camera compared to a mirrorless, and it will remain the case for the next few iPhone releases.

orbital-decay 5y ago

It still seems we're talking about completely different things, or maybe you're trying to derail the conversation... Both ML-based processing and tonemapping take place after demosaicing to produce the final JPEG. RAW capture is just your typical laplacian pyramid stacking/exposure fusion with motion compensation and sensor-specific calibration. That's the main contributing factor that somewhat compensates for the sensor size, not some kind of ML processing. If you think that ML is somehow involved here, I have zero idea where you are getting it from. The entire point of pre-demosaicing pipeline is to robustly converge the captured stack into a neutral scene-referred baseline with provably better SNR and precise photometric parameters. And that's what you get in a resulting DNG, not some neural tonemapped RGB image (which is not even created at this point). Perhaps you should look at the pipeline yourself, here's the source code for a typical "computational imaging" camera app that produces stacked RAWs, for example. [1] And here's what Google Camera does [2] [3] (you can ignore everything that comes after demosaicing). Google's particular method is also partially implemented for arbitrary RAWs as third-party software, so you can stack any burst capture from your camera with the same method smartphones do. [4] (there's much less benefit for normal sized sensors, for obvious reasons) [1] https://github.com/mirsadm/motioncam/ https://github.com/mirsadm/motioncam/ [2] https://static.googleusercontent.com/media/hdrplusdata.org/en//hdrplus.pdf https://static.googleusercontent.com/media/hdrplusdata.org/e... (PDF) [3] https://static.googleusercontent.com/media/hdrplusdata.org/en//hdrplus_supp.pdf https://static.googleusercontent.com/media/hdrplusdata.org/e... (PDF) [4] https://github.com/martin-marek/hdr-plus-swift https://github.com/martin-marek/hdr-plus-swift

deleted 5y ago
[deleted]
goblin89 5y ago
> It still seems we're talking about completely different things, or maybe you're trying to derail the conversation... I am making literally the same point I made when replying to jhoechtl: phones and cameras are different beasts, and the next iPhone is unlikely to obsolete semi-pro mirrorless cameras. Judging by your tone, you are dead set at proving me wrong, but I can’t see a meaningful argument. Why do you keep reverting to ML involvement at pre-demosaicing level? “Magic” computed raws is not something I claimed to happen now, nor am I certain it ever will—I just speculated we are logically headed there and if we do get there it may affect my view at phones vs. mirrorless. So no, not interested in a hypothetical typical “dumb” camera app or raw capture from a phone. I mean the computationally more advanced capture (like the one done by bog standard Camera) that produces good images, which is what makes people think iPhone can rival a mirrorless. If you take away (by shooting raw or using a dumber app) semantic masks and all the magic it uses to obtain display-referred data for you, you just get a mediocre camera compared to a mirrorless, and it will remain the case for the next few iPhone releases.