8 ms·

I would like to see computational photography applied to raw images from DSLRs and MILCs with APS-C and larger sensors. Perhaps Canon, Nikon, Sony, and Fujifilm

I would like to see computational photography applied to raw images from DSLRs and MILCs with APS-C and larger sensors. Perhaps Canon, Nikon, Sony, and Fujifilm could have built-in options in their cameras for ‘social media mode’, with a modicum of noise reduction (honestly unnecessary at ISOs lower than about 1600 for modern cameras), but drastically improved HDR and white balance. Many of these cameras are able to take bracketed[1] exposures, and the SNR in even just one image from such sensors is immense compared to the tiny sensors in phones. Surely with this much more data to work with, HDR is much nicer and without the edge brightening typically seen in phone HDR images. [1]: https://www.nikonusa.com/en/learn-and-explore/a/tips-and-techniques/exposure-bracketing-the-creative-insurance-policy.html https://www.nikonusa.com/en/learn-and-explore/a/tips-and-tec...

haswell 4y ago
I’d love to see this on “adventures” cameras like the Olympus OM-D line. This would also pair well with Fujifilm’s lineup which already includes camera features focused on in-camera processing.
- rimliu 4y ago
  I think Olympus was a pioneer at least in some aspects in computation photography. At least "live composite" and "focus stacking" were not very common at the time Olympus introduced them.
jlarocco 4y ago
I'm not exactly sure what you're asking for here. In my experience (and as seen in the article) the image processing in most digital cameras will already blow an iPhone out of the water. As far as I know, iPhone and Android aren't doing anything that isn't already done by digital cameras. They ramp up the settings on things like noise reduction and sharpness to balance out their tiny sensors, but it's more or less the same algorithms that the cameras are using. Good cameras even allow you to tweak the settings and control RAW conversion right on the camera. The author could have botched the noise reduction on his Fujifilm to match the iPhone if he wanted to. [0] [0] https://www.jmpeltier.com/fujifilm-in-camera-raw-converter/ https://www.jmpeltier.com/fujifilm-in-camera-raw-converter/
- jack1243star 4y ago
  > As far as I know, iPhone and Android aren't doing anything that isn't already done by digital cameras. It's the opposite, most digital cameras aren't doing enough compared to smartphones, which have minimum control and look very good at 2M pixels. A proper camera with default settings looks bland in the hands of an amateur (too little saturation) and handles dynamic range poorly (without bracketing or post-processing). Plus it's never going to beat smartphones at connectivity (SNS), or portability. Most people aren't going to spend time processing RAW files, or try to get in-camera picture control/film simulation etc. to look pleasing like photographers do.
  - jlarocco 4y ago
    I don't buy it. Which cameras have worse "Auto" settings than a phone? I have two Fujifilms, a Panasonic, and a Nikon, and they all take better photos on "Auto" than my iPhone or any of my friends' Androids. And who buys a camera in this day and age but doesn't want to play with the settings? Point and shoot users stick with their phones now.
    - jsmith99 4y ago
      It's not about better or worse, it's about trade offs. At small social media sizes pictures will look better from a phone than from an enthusiast camera. Sharpening is a destructive process and the appropriate amount depends on how and what size you are using the image. It can also be easily applied anytime later. So enthusiast camera apply minimal sharpening and look under sharpened at small sizes, and phone cameras apply heavy sharpening but have artefacts if you zoom into them. Same with saturation and 'ai' tricks like skin smoothing: users of enthusiast cameras often have different preferences.
      - astrange 4y ago
        > Same with saturation and 'ai' tricks like skin smoothing: users of enthusiast cameras often have different preferences. Note, phone cameras don't do skin smoothing, people just think they do. (Unless you get a beauty camera app.) The issue is that almost any processing has the effect of removing noise, which is the same as skin smoothing. If you're taking 2-3 images and merging them, it'll remove the noise even if you do something smarter than averaging the pixels - so you actually have to add steps to measure the noise level and add it back. The same issue comes up in video compression, where there's dedicated film grain metadata since almost any compression kills the real thing.
dusted 4y ago
They already are. However, most photographers do not appreciate this type of distortion being applied to their images. At a glance, my samsung note 22 ultra takes better picture than my nikon d7500. At a glance. However, as soon as you want to actually DO anything to it, like, view it in any real detail, or on anything but a tiny screen, reality returns.. While the phone is absolutely fantastic for a quick snapshot, it just does not come close to the definition of the older camera with the bigger sensor.
- prox 4y ago
  This! My close to 10 year old DSLR still outperforms the latest and greatest on any phone for this reason.
- psychphysic 4y ago
  Similarly sometimes I just want to apply whatever ai magic elixir my phone does to the raw image from my DSLR. Messing around in lightroom etc is just not worth it for hundreds of shots I take per day sometimes.
  - matwood 4y ago
    So true. Not only that, but pulling out my phone and quickly snapping a shot is often better than quickly pulling out my non-phone cameras - especially in challenging lighting conditions. I absolutely can get better shots with my non-phone cameras, but they take more work.
    - prmoustache 4y ago
      I understand carrying and taking shot with a DSLR is much more involved than pulling out a phone from a pocket, but have you ever used compact cameras with 1" censor like a Sony RX100 or Canon G9x ? They both fit in your pocket and startup as fast as you can activate the camera on your phone. Out of habit I usually reach the power button inconsciently when grabbing it from my pocket and the lense is usually extending while my arm is moving in position so it is usually immediately ready to snap a picture. If you are in a setting where you are almost certain to take picture, a micro four third format also works great and depending on the lense are usually small enough if you are the kind of guy who wears small shoulder or sling bags.
buildbot 4y ago
There is no reason to do this. If you want to apply the algorithms and Iphone uses, simply take a burst and post process later for HDR or whatever. I remember being in middle school and messing around with Hugin and my Minolta bridge camera… People value quick shots/edits and don’t care about quality or editing things later don’t mind an iPhone doing all this behind the scene - but it is irreversible. The sort of error in the article would drive myself and other photographers up the wall. Also, an Iphone has a CPU and ISP that outclass desktops from only a few years ago - camera manufacturers simply don’t have the same compute available. On the other hand, some brands do provide interesting computational photography in their cameras at the very high end. Panasonic mirrorless full frame cameras have a pixel shift mode for super res/no bayer interpolation, with some ability to fix motion between steps. Phase One has frame averaging and dual exposure in their IQ4 digital backs, for sequential capture into a single frame and super high dynamic range respectively.
- SSLy 4y ago
  >camera manufacturers simply don’t have the same compute available. what forbids them from buying the competitive SoCs from Qualcomm? Pride?
  - scriptproof 4y ago
    They would have to rewrite entirely their software. There had been some attemps to use these Qualcomm processors in cameras (the Yi M1 for example) and the result was terrible, the autofocus specially.
  - _visgean 4y ago
    i think the battery consumption is also quite bad.
  - zirgs 4y ago
    Power consumption is the main issue. Software bloat is another. Current digital cameras start up and are ready to shoot almost instantly. But it takes a while for a phone to boot up. Photographers don't want that.
  - 4y ago
account42 4y ago
Pretty much all RAW processing includes computational photography - at the very least debayering, but likely more.
- astrange 4y ago
  Taking the RAW involves it too - autofocus and auto ISO. And developing the raw involves picking a white balance and several kinds of lens correction.
deleted 4y ago
[deleted]
throwanem 4y ago
> I would like to see computational photography applied to raw images from DSLRs and MILCs with APS-C and larger sensors. Wash your mouth out with soap! I did not spend five thousand dollars on a D850-based macro rig to have it produce results no better in quality than what I can get from my phone.
irthomasthomas 4y ago
Most already offer that in their various automatic modes. We don't like it because whatever can be done computationally on a camera, can be done 10x better on a real computer with more hardware and finer controls.
mikewarot 4y ago
Long ago I did some experiments with SuperResolution[1] which is a technique of aligning images to sub-pixel resolution and stacking them to increase the amount of information with the square root of the number of exposures. I was using Hugin to align the images from my Nikon DSLR. I found that you can get to at least double the resolution in both dimensions fairly quickly, but you'll never get to "enhance" like in TV shows. [1] https://en.wikipedia.org/wiki/Super-resolution_imaging https://en.wikipedia.org/wiki/Super-resolution_imaging
- kuschku 4y ago
  Modern cameras use the optical image stabilization for pixel shift photography, where you can get up to 9× the normal sensor resolution (by moving the sensor slightly).
  - astrange 4y ago
    Of course, for this to work nothing else about the camera or the scene can move, or else you need a lot of smarts to realign the image and ignore the parts of the image that aren't registering anymore. This is typically pretty difficult for an outdoor photo.
cultofmetatron 4y ago
Its definitely done! and whats better, you can do far more with a proper raw file from a large sensor. the trick is the workflows aren't as simple as having it happen automatically. in my workflow I use either dx0 photolab which has excellent facilities for really bringing out a single image. sometimes I wanna go hardcore with a landscape and take multiple shots manuallly and blend them together using software like aurora HDR (example: https://www.flickr.com/photos/193526747@N04/52219385902/ https://www.flickr.com/photos/193526747@N04/52219385902/ ) That image is 5 stacked images combined together using a bit of computational photography and adjusted for saturation. if you want somethign taht will get you descent results fast and work with raws. you can also go with luminar https://skylum.com/luminar https://skylum.com/luminar an iphone image looks better at first snap but my z5's images blow them out of the water once I give them some love in the edit room.
NegativeK 4y ago
It's going to take ILCs switching to sensors like the Z9, without a physical shutter, doing very high frame rates at full resolution. I'd be interested in computational photography on ILCs if they allowed tuning it -- with phones, it comes with a bunch of other stylistic choices, and I want control over that stuff.
pkulak 4y ago
I've got an Olympus M43 camera, which has an image sensor that's both significantly smaller than full frame, and significantly larger than a phone. I've found that I can do absolutely amazing things to the RAW image later. There are these processes now that will use ML to remove noise, and it blows my mind every time. I can shoot at ISO 6400 now without even thinking about it. I used to cringe when I had to hit 1600. Olympus (OM Systems?) should build this stuff into their cameras. I used to have a bit of inkling to some day "upgrade" to full frame, but not any more.
- Tarrosion 4y ago
  I'm also shooting on an Olympus M43, several years old. Any suggestions for software tools you've found helpful?
  - pkulak 4y ago
    Yeah, didn't want to sound like a shill, but Dx0 is so good I run it in a virtual machine on my Linux box. It's "Deep Prime" noise reduction thing is what I was talking about. There are others, even some stand alone, but it's nice having it all in one place, and Deep Prime does seem to be the best.

haswell 4y ago

I’d love to see this on “adventures” cameras like the Olympus OM-D line. This would also pair well with Fujifilm’s lineup which already includes camera features focused on in-camera processing.

rimliu 4y ago
I think Olympus was a pioneer at least in some aspects in computation photography. At least "live composite" and "focus stacking" were not very common at the time Olympus introduced them.

jlarocco 4y ago

I'm not exactly sure what you're asking for here. In my experience (and as seen in the article) the image processing in most digital cameras will already blow an iPhone out of the water. As far as I know, iPhone and Android aren't doing anything that isn't already done by digital cameras. They ramp up the settings on things like noise reduction and sharpness to balance out their tiny sensors, but it's more or less the same algorithms that the cameras are using. Good cameras even allow you to tweak the settings and control RAW conversion right on the camera. The author could have botched the noise reduction on his Fujifilm to match the iPhone if he wanted to. [0] [0] https://www.jmpeltier.com/fujifilm-in-camera-raw-converter/ https://www.jmpeltier.com/fujifilm-in-camera-raw-converter/

jack1243star 4y ago
> As far as I know, iPhone and Android aren't doing anything that isn't already done by digital cameras. It's the opposite, most digital cameras aren't doing enough compared to smartphones, which have minimum control and look very good at 2M pixels. A proper camera with default settings looks bland in the hands of an amateur (too little saturation) and handles dynamic range poorly (without bracketing or post-processing). Plus it's never going to beat smartphones at connectivity (SNS), or portability. Most people aren't going to spend time processing RAW files, or try to get in-camera picture control/film simulation etc. to look pleasing like photographers do.
- jlarocco 4y ago
  I don't buy it. Which cameras have worse "Auto" settings than a phone? I have two Fujifilms, a Panasonic, and a Nikon, and they all take better photos on "Auto" than my iPhone or any of my friends' Androids. And who buys a camera in this day and age but doesn't want to play with the settings? Point and shoot users stick with their phones now.
  - jsmith99 4y ago
    It's not about better or worse, it's about trade offs. At small social media sizes pictures will look better from a phone than from an enthusiast camera. Sharpening is a destructive process and the appropriate amount depends on how and what size you are using the image. It can also be easily applied anytime later. So enthusiast camera apply minimal sharpening and look under sharpened at small sizes, and phone cameras apply heavy sharpening but have artefacts if you zoom into them. Same with saturation and 'ai' tricks like skin smoothing: users of enthusiast cameras often have different preferences.
    - astrange 4y ago
      > Same with saturation and 'ai' tricks like skin smoothing: users of enthusiast cameras often have different preferences. Note, phone cameras don't do skin smoothing, people just think they do. (Unless you get a beauty camera app.) The issue is that almost any processing has the effect of removing noise, which is the same as skin smoothing. If you're taking 2-3 images and merging them, it'll remove the noise even if you do something smarter than averaging the pixels - so you actually have to add steps to measure the noise level and add it back. The same issue comes up in video compression, where there's dedicated film grain metadata since almost any compression kills the real thing.

dusted 4y ago

They already are. However, most photographers do not appreciate this type of distortion being applied to their images. At a glance, my samsung note 22 ultra takes better picture than my nikon d7500. At a glance. However, as soon as you want to actually DO anything to it, like, view it in any real detail, or on anything but a tiny screen, reality returns.. While the phone is absolutely fantastic for a quick snapshot, it just does not come close to the definition of the older camera with the bigger sensor.

prox 4y ago
This! My close to 10 year old DSLR still outperforms the latest and greatest on any phone for this reason.
psychphysic 4y ago
Similarly sometimes I just want to apply whatever ai magic elixir my phone does to the raw image from my DSLR. Messing around in lightroom etc is just not worth it for hundreds of shots I take per day sometimes.
- matwood 4y ago
  So true. Not only that, but pulling out my phone and quickly snapping a shot is often better than quickly pulling out my non-phone cameras - especially in challenging lighting conditions. I absolutely can get better shots with my non-phone cameras, but they take more work.
  - prmoustache 4y ago
    I understand carrying and taking shot with a DSLR is much more involved than pulling out a phone from a pocket, but have you ever used compact cameras with 1" censor like a Sony RX100 or Canon G9x ? They both fit in your pocket and startup as fast as you can activate the camera on your phone. Out of habit I usually reach the power button inconsciently when grabbing it from my pocket and the lense is usually extending while my arm is moving in position so it is usually immediately ready to snap a picture. If you are in a setting where you are almost certain to take picture, a micro four third format also works great and depending on the lense are usually small enough if you are the kind of guy who wears small shoulder or sling bags.

buildbot 4y ago

There is no reason to do this. If you want to apply the algorithms and Iphone uses, simply take a burst and post process later for HDR or whatever. I remember being in middle school and messing around with Hugin and my Minolta bridge camera… People value quick shots/edits and don’t care about quality or editing things later don’t mind an iPhone doing all this behind the scene - but it is irreversible. The sort of error in the article would drive myself and other photographers up the wall. Also, an Iphone has a CPU and ISP that outclass desktops from only a few years ago - camera manufacturers simply don’t have the same compute available. On the other hand, some brands do provide interesting computational photography in their cameras at the very high end. Panasonic mirrorless full frame cameras have a pixel shift mode for super res/no bayer interpolation, with some ability to fix motion between steps. Phase One has frame averaging and dual exposure in their IQ4 digital backs, for sequential capture into a single frame and super high dynamic range respectively.

SSLy 4y ago
>camera manufacturers simply don’t have the same compute available. what forbids them from buying the competitive SoCs from Qualcomm? Pride?
- scriptproof 4y ago
  They would have to rewrite entirely their software. There had been some attemps to use these Qualcomm processors in cameras (the Yi M1 for example) and the result was terrible, the autofocus specially.
- _visgean 4y ago
  i think the battery consumption is also quite bad.
- zirgs 4y ago
  Power consumption is the main issue. Software bloat is another. Current digital cameras start up and are ready to shoot almost instantly. But it takes a while for a phone to boot up. Photographers don't want that.
- 4y ago

account42 4y ago

Pretty much all RAW processing includes computational photography - at the very least debayering, but likely more.

astrange 4y ago
Taking the RAW involves it too - autofocus and auto ISO. And developing the raw involves picking a white balance and several kinds of lens correction.

deleted 4y ago

[deleted]

throwanem 4y ago

> I would like to see computational photography applied to raw images from DSLRs and MILCs with APS-C and larger sensors. Wash your mouth out with soap! I did not spend five thousand dollars on a D850-based macro rig to have it produce results no better in quality than what I can get from my phone.

irthomasthomas 4y ago

Most already offer that in their various automatic modes. We don't like it because whatever can be done computationally on a camera, can be done 10x better on a real computer with more hardware and finer controls.

mikewarot 4y ago

Long ago I did some experiments with SuperResolution[1] which is a technique of aligning images to sub-pixel resolution and stacking them to increase the amount of information with the square root of the number of exposures. I was using Hugin to align the images from my Nikon DSLR. I found that you can get to at least double the resolution in both dimensions fairly quickly, but you'll never get to "enhance" like in TV shows. [1] https://en.wikipedia.org/wiki/Super-resolution_imaging https://en.wikipedia.org/wiki/Super-resolution_imaging

kuschku 4y ago
Modern cameras use the optical image stabilization for pixel shift photography, where you can get up to 9× the normal sensor resolution (by moving the sensor slightly).
- astrange 4y ago
  Of course, for this to work nothing else about the camera or the scene can move, or else you need a lot of smarts to realign the image and ignore the parts of the image that aren't registering anymore. This is typically pretty difficult for an outdoor photo.

cultofmetatron 4y ago

Its definitely done! and whats better, you can do far more with a proper raw file from a large sensor. the trick is the workflows aren't as simple as having it happen automatically. in my workflow I use either dx0 photolab which has excellent facilities for really bringing out a single image. sometimes I wanna go hardcore with a landscape and take multiple shots manuallly and blend them together using software like aurora HDR (example: https://www.flickr.com/photos/193526747@N04/52219385902/ https://www.flickr.com/photos/193526747@N04/52219385902/ ) That image is 5 stacked images combined together using a bit of computational photography and adjusted for saturation. if you want somethign taht will get you descent results fast and work with raws. you can also go with luminar https://skylum.com/luminar https://skylum.com/luminar an iphone image looks better at first snap but my z5's images blow them out of the water once I give them some love in the edit room.

NegativeK 4y ago

It's going to take ILCs switching to sensors like the Z9, without a physical shutter, doing very high frame rates at full resolution. I'd be interested in computational photography on ILCs if they allowed tuning it -- with phones, it comes with a bunch of other stylistic choices, and I want control over that stuff.

pkulak 4y ago

I've got an Olympus M43 camera, which has an image sensor that's both significantly smaller than full frame, and significantly larger than a phone. I've found that I can do absolutely amazing things to the RAW image later. There are these processes now that will use ML to remove noise, and it blows my mind every time. I can shoot at ISO 6400 now without even thinking about it. I used to cringe when I had to hit 1600. Olympus (OM Systems?) should build this stuff into their cameras. I used to have a bit of inkling to some day "upgrade" to full frame, but not any more.

Tarrosion 4y ago
I'm also shooting on an Olympus M43, several years old. Any suggestions for software tools you've found helpful?
- pkulak 4y ago
  Yeah, didn't want to sound like a shill, but Dx0 is so good I run it in a virtual machine on my Linux box. It's "Deep Prime" noise reduction thing is what I was talking about. There are others, even some stand alone, but it's nice having it all in one place, and Deep Prime does seem to be the best.