9 ms·

What does an unprocessed RAW file look like?

244 points by snailletters 7y ago

dlivingston 7y ago
The author notes that 'But, there is no such standard...[a]ll real-world RAW processing programs have their own ideas of a basic default state to apply to a fresh RAW file on load'. While I can accept that there isn't a universal RAW -> RGB standard, it seems strange to me that 'compute how this photo should appear' is left as an exercise as a reader. Photographers often view their work as a form of art, and artists are very particular about even the smallest details of their work. Why, then, would Nikon, Canon, and especially Leica, not have their own definable standards of how to process RAW photos for their particular cameras?
- tinus_hn 7y ago
  A RAW file is a description of the light captured by the sensor. What good would such a standard do? The RAW file is not supposed to be a finished work of art, it’s designed to be processed.
  - cmurf 7y ago
    The RAW file is analogous to the latent image on film. It's before most all developing and processing. A big difference is a metric f ton of non-linearity in film, and for practical purposes the RAW file contains linear data.
  - theoh 7y ago
    Interestingly, some major photo competitions require 'original' RAW files to be submitted along with processed jpegs. The idea is that they can check that the image hasn't been modified in a way that breaks the rules. Of course, software people know that it would be tedious but technically straightforward to concoct a fake RAW file from another 'raw' format such as a PPM image. The stakes in these competitions can be quite high so it's a bit disconcerting that they rely on a presumption that converting RAW files to editable images is a one-way process.
    - jacobush 7y ago
      No - it's not straightforward. How the light strikes, sensor imperfections, patterns unique to each camera body even, thermal noise. It's very very hard actually. It's one thing to doctor a finished image - it's another, in a forensic level really, to doctor a RAW file. You can make one which on its own is syntactically correct but which does not show all the telltale signs of being shot in actual hardware, no patterns or imperfections etc. This is trivial. Next level is to make it look plausible on its own, with plausible imperfections. This is a bit more involved and tedious. The really hard part is to make it look like it came from the exact same individual purported camera body the photographer allegedly used. This is really hard. An analogy would perhaps be taking a picture of a room: Now, build another room that would make the same picture. Make sure everything in the room is plausible yet renders exactly the same picture as the first room. (The room is the RAW file.) Edit: (I'm not saying all the competitions have the time and know how to verify a RAW file.)
      - theoh 7y ago
        The scenario I'm talking about is not synthesising a RAW file from scratch, but being able to freely edit it and save the result back into a RAW file that looks "original". I don't think you've made any statements that indicate that doing that would be complex. Again, this is not about faking a RAW file from scratch, just being able to edit it in place.
        jacobush 7y ago
        Oh, that I have too little insight in. I could speculate that going either way. Would be very interesting to know for sure!
        jacobush 7y ago
        One potential complication just occurred to me. If, (IFF!) each sensor site has a somewhat unique non-linearity in any way, you can't just go around in the image and change "pixels". You have to adjust them in a way that would not change the patterns normal to that camera for that sensor site and for neighboring sensor sites which could have been affected by the same light.
- etrevino 7y ago
  > Why, then, would Nikon, Canon, and especially Leica, not have their own definable standards of how to process RAW photos for their particular cameras? Pretty sure they do. Aren't those the manufacturer-specific import profiles in programs like Lightroom?
  - colejohnson66 7y ago
    But are those written by the camera manufacturers?
  - petepete 7y ago
    Nikon and Canon always try to push you to use their own software. Phase One, Adobe, Photo Mechanic etc all reverse engineer the manufacturers formats, which is why the results differ. Anecdotally, Capture One is (for me) way more responsive when working with Nikon .nef files than with Fuji .raf.
- m463 7y ago
  I believe nikon actually encrypts their raw file format. (or parts of the format)
- klank 7y ago
  > and artists are very particular about even the smallest details of their work. That's not necessarily true and paints way too broad a brush of artists and art. Art and artists need not necessarily be concerned about the smallest details. Particularly if it is an intermediate medium which never is shown to the art viewer/participant.
- toomanybeersies 7y ago
  They do to an extent. It's just that for some camera manufacturers, it's only accessible in-camera (i.e. shooting in JPG mode), for others, they do have a separate RAW converter you can use on your computer (Fujifilm does, unsure for other manufacturers). Canon can't make Adobe or Phase One use their algorithm for decoding RAWs though.
  - dylan604 7y ago
    They could not license the decoder for them to use in their respective software.
- brudgers 7y ago
  Adobe Digital Negative (DNG) [1] is that sort of standard (sort of) and there are manufacturers who use DNG as the RAW format in their cameras. Leica tends to use DNG as it's RAW file format. [2] [1]: https://en.wikipedia.org/wiki/Digital_Negative https://en.wikipedia.org/wiki/Digital_Negative [2]: https://www.bhphotovideo.com/c/product/1082939-REG/leica_10804_leica_s_medium_format.html/specs https://www.bhphotovideo.com/c/product/1082939-REG/leica_108...
- stephen_g 7y ago
  I think your premise is a bit faulty. A photographer basically never gives anybody a raw image unless they’re giving up control of how the final image looks to somebody else (as some other said, there are some special circumstances, such as competitions - but this is not for anybody to use for presentation, just ensuring that the image hasn’t been altered in ways not allowed - i.e compositing multiple photos to fake some amazing occurrence). Processing the raw file is almost as much a creative art as composition and photography itself! So if a application processes raw images in a way the photographer likes better than the algorithms another app uses, then all the better. They’re going to export it in the end anyway. The camera manufacturers of course do release specifications and probably proprietary code for processing raw images from their cameras to companies like Adobe to use in Lightroom under NDA, which I assume they use as a basis, but all these apps have their own ways of doing adjustment and extra processing too.
ggm 7y ago
This reminded me of the autochrome process. I believe they did something with coloured starch grains and lamp black.
bradknowles 7y ago
A raw file? Well, it's really more a lump of metal, until you start putting some teeth on it and you're ready for the tempering process. /s
JoshMcguigan 7y ago
This was really neat, but I wish the author went into similar detail for the auto white balance step.
- vortico 7y ago
  White-balancing is a function f(R, G, B, T_from, T_to) -> (R, G, B) which relies on the model of blackbody radiation and a model of the wavelength distribution definitions behind the (R, G, B) values, i.e. the particular color space you're working in. When working with a manufacturer's RAW format, Auto white-balancing is finding `T_to` so that the color values are "most balanced". It is up to the manufacturer how this is decided (and might be proprietary secret sauce). Custom white-balancing is taking a known white or grayscale object and finding `T_to` which would yield something close to (1, 1, 1) for pixels of that object.
  - vortico 7y ago
    I think I accidentally erased a sentence in an edit... When working with a manufacturer's RAW format, there is a defined way to get the values (R, G, B, T_from) as a result of demosaicing. For example, in the author's photo post-demosaicing, the manufacturer calibrates the final data to obtain (I'm completely making this up) T_from = 2700K. Then using that information, you can correctly adjust the temperature to any other temperature using `f` above.
3JPLW 7y ago
Really cool article. I'm left with just one question: Why are the 16-bit RAW values so limited in their dynamic range? Wouldn't sensor manufacturers want to have their pixels able to return values that range the whole way from 0x0000 to 0xffff?
- fake-name 7y ago
  Very few (if any) digital cameras have 16-bit ADCs. Most cameras use 12 or 14 bit ADCs, so they only use 2^12 or 2^14 of the available values. Generally, the RAW files aren't actually 16 bit, but rather a packed structure of the native ADC size. In this case, they're using 16-bit intermediate bitmaps. They're not 16 bit from the RAW. ------ Fujifilm actually has a few cameras with a sensor that can be put in 16-bit mode, but IIRC it's not enabled in software.
  - buildbot 7y ago
    Essentially only very high end digital backs from hasselblad/phase one have 16 bit ADCs, but people disagree if it’s even required, the last two bits are basically noise.
    - kurthr 7y ago
      If the noise is actually white, that may not be a bad thing for filtering.
      - kartickv 7y ago
        How is that?
        kortex 7y ago
        Additive white gaussian noise is quite well studied and pretty easy to remove with processing. So even if those last few bits are "just noise", if that noise is well-behaved, on the whole image you still have more information. There's also the effect of pixel well depth. I forget the exact interplay, but I believe this usually ends up being the limiting factor before ADC resolution.
Causality1 7y ago
Something that's fascinating to me is that at 1:1 display size, the output of modern cameras doesn't look that much better than the pictures out of old 0.3Mpixel still cameras of nearly twenty years ago. The dynamic range is better and the colors are more vibrant, and the noise floor is lower for dark scenes, but on the whole it still looks pretty crappy at full resolution. Why is that? Could we fix it by using larger CCD/CMOS sensor pixels and sticking to lower total pixel counts?
- pvg 7y ago
  What images are you comparing? 20 year old digital cameras were around a Mpixel. A modern camera with a decent lense produces much better images. In both cases you'd have to do some work to make sure the thing you're blowing up is exactly in focus.
  - deleted 7y ago
    [deleted]
- ryacko 7y ago
  Lens quality, focusing, time to compress the image, etc. People want to take a few photos in quick succession, which has it’s own issues. Film is superior to digital for those issues, for now.
- petascale 7y ago
  With higher resolution sensors, small imperfections are more visible. To extract the most from a high-megapixel camera, you need spot-on focusing, a higher quality (more expensive) lens, and more care (e.g. higher shutter speed or tripod) to avoid camera shake during exposure. If you nail all these, I wouldn't call it crappy even at 1:1. But there is a residual imperfection from the Bayer pattern on the sensor, since each pixel records only one color (R, G, or B) and the other two values for that pixel have to be guesstimated from neighboring pixels, so the de-bayering process isn't perfect. One way to fix it is to use a monochrome sensor and color filters, taking 3-4 exposures (luminance/mono plus R/G/B) and stack them. A few cameras on the market have a pixel shift feature that can do something similar - multiple exposures, shifting the sensor one pixel between exposures so each pixel get a true R/G/B sample, and stack them in camera or in post. Edit: Forgot to mention the anti-aliasing filter. It sits in front of the sensor and deliberately blurs the image at the pixel level. This is done to avoid aliasing and moiré artifacts, but obviously has the side effect of not-so-great image quality at the pixel peeping level. The fix for this is to get a camera without an AA filter, many modern high-resolution cameras don't have them. https://www.outdoorphotographer.com/photography-gear/cameras/can-you-go-no-low-pass/ https://www.outdoorphotographer.com/photography-gear/cameras...
arriu 7y ago
If you want to extract the actual raw data, dcraw doesn't really advertise the way to do it. Many of the options for extracting the data still apply some processing/conversion to the underlying data. It is a pretty big mess actually. Each manufacturer has different constants that need to be applied to the data, so the raw data from one device is not comparable to another. Convincing dcraw that you actually want the unaltered data is not straight forward. Anyways, if you really do want to see the raw image, there are a few undocumented flags you can use. > dcraw -E -4 -T *.CR2 This will give you an unprocessed 16-bit tiff file containing the "raw" data. What is interesting is that some camera sensors capture data slightly beyond the image you are presented with. The camera will crop in and debayer the image for you, as will your image editor. See this thread for hilarious lengths people go to in order to get unaltered data from their sensors (monochorme converting a dslr): https://stargazerslounge.com/topic/166334-debayering-a-dslrs-bayer-matrix/page/94/ https://stargazerslounge.com/topic/166334-debayering-a-dslrs...
- bufferoverflow 7y ago
  That thread is more about removing the physical filters from the DSLR sensors - RGB, UV, infrared to achieve higher resolution and much higher sensitivity.
cfstras 7y ago
Try https://github.com/anuejn/batic https://github.com/anuejn/batic - a shadertoy-like experiment where you can to implement a shader to convert the bayer-pattern.

244 points by snailletters 7y ago

The author notes that 'But, there is no such standard...[a]ll real-world RAW processing programs have their own ideas of a basic default state to apply to a fresh RAW file on load'. While I can accept that there isn't a universal RAW -> RGB standard, it seems strange to me that 'compute how this photo should appear' is left as an exercise as a reader. Photographers often view their work as a form of art, and artists are very particular about even the smallest details of their work. Why, then, would Nikon, Canon, and especially Leica, not have their own definable standards of how to process RAW photos for their particular cameras?

tinus_hn 7y ago
A RAW file is a description of the light captured by the sensor. What good would such a standard do? The RAW file is not supposed to be a finished work of art, it’s designed to be processed.
- cmurf 7y ago
  The RAW file is analogous to the latent image on film. It's before most all developing and processing. A big difference is a metric f ton of non-linearity in film, and for practical purposes the RAW file contains linear data.
- theoh 7y ago
  Interestingly, some major photo competitions require 'original' RAW files to be submitted along with processed jpegs. The idea is that they can check that the image hasn't been modified in a way that breaks the rules. Of course, software people know that it would be tedious but technically straightforward to concoct a fake RAW file from another 'raw' format such as a PPM image. The stakes in these competitions can be quite high so it's a bit disconcerting that they rely on a presumption that converting RAW files to editable images is a one-way process.
  - jacobush 7y ago
    No - it's not straightforward. How the light strikes, sensor imperfections, patterns unique to each camera body even, thermal noise. It's very very hard actually. It's one thing to doctor a finished image - it's another, in a forensic level really, to doctor a RAW file. You can make one which on its own is syntactically correct but which does not show all the telltale signs of being shot in actual hardware, no patterns or imperfections etc. This is trivial. Next level is to make it look plausible on its own, with plausible imperfections. This is a bit more involved and tedious. The really hard part is to make it look like it came from the exact same individual purported camera body the photographer allegedly used. This is really hard. An analogy would perhaps be taking a picture of a room: Now, build another room that would make the same picture. Make sure everything in the room is plausible yet renders exactly the same picture as the first room. (The room is the RAW file.) Edit: (I'm not saying all the competitions have the time and know how to verify a RAW file.)
    - theoh 7y ago
      The scenario I'm talking about is not synthesising a RAW file from scratch, but being able to freely edit it and save the result back into a RAW file that looks "original". I don't think you've made any statements that indicate that doing that would be complex. Again, this is not about faking a RAW file from scratch, just being able to edit it in place.
      - jacobush 7y ago
        Oh, that I have too little insight in. I could speculate that going either way. Would be very interesting to know for sure!
      - jacobush 7y ago
        One potential complication just occurred to me. If, (IFF!) each sensor site has a somewhat unique non-linearity in any way, you can't just go around in the image and change "pixels". You have to adjust them in a way that would not change the patterns normal to that camera for that sensor site and for neighboring sensor sites which could have been affected by the same light.
etrevino 7y ago
> Why, then, would Nikon, Canon, and especially Leica, not have their own definable standards of how to process RAW photos for their particular cameras? Pretty sure they do. Aren't those the manufacturer-specific import profiles in programs like Lightroom?
- colejohnson66 7y ago
  But are those written by the camera manufacturers?
- petepete 7y ago
  Nikon and Canon always try to push you to use their own software. Phase One, Adobe, Photo Mechanic etc all reverse engineer the manufacturers formats, which is why the results differ. Anecdotally, Capture One is (for me) way more responsive when working with Nikon .nef files than with Fuji .raf.
m463 7y ago
I believe nikon actually encrypts their raw file format. (or parts of the format)
klank 7y ago
> and artists are very particular about even the smallest details of their work. That's not necessarily true and paints way too broad a brush of artists and art. Art and artists need not necessarily be concerned about the smallest details. Particularly if it is an intermediate medium which never is shown to the art viewer/participant.
toomanybeersies 7y ago
They do to an extent. It's just that for some camera manufacturers, it's only accessible in-camera (i.e. shooting in JPG mode), for others, they do have a separate RAW converter you can use on your computer (Fujifilm does, unsure for other manufacturers). Canon can't make Adobe or Phase One use their algorithm for decoding RAWs though.
- dylan604 7y ago
  They could not license the decoder for them to use in their respective software.
brudgers 7y ago
Adobe Digital Negative (DNG) [1] is that sort of standard (sort of) and there are manufacturers who use DNG as the RAW format in their cameras. Leica tends to use DNG as it's RAW file format. [2] [1]: https://en.wikipedia.org/wiki/Digital_Negative https://en.wikipedia.org/wiki/Digital_Negative [2]: https://www.bhphotovideo.com/c/product/1082939-REG/leica_10804_leica_s_medium_format.html/specs https://www.bhphotovideo.com/c/product/1082939-REG/leica_108...
stephen_g 7y ago
I think your premise is a bit faulty. A photographer basically never gives anybody a raw image unless they’re giving up control of how the final image looks to somebody else (as some other said, there are some special circumstances, such as competitions - but this is not for anybody to use for presentation, just ensuring that the image hasn’t been altered in ways not allowed - i.e compositing multiple photos to fake some amazing occurrence). Processing the raw file is almost as much a creative art as composition and photography itself! So if a application processes raw images in a way the photographer likes better than the algorithms another app uses, then all the better. They’re going to export it in the end anyway. The camera manufacturers of course do release specifications and probably proprietary code for processing raw images from their cameras to companies like Adobe to use in Lightroom under NDA, which I assume they use as a basis, but all these apps have their own ways of doing adjustment and extra processing too.

ggm 7y ago

This reminded me of the autochrome process. I believe they did something with coloured starch grains and lamp black.

bradknowles 7y ago

A raw file? Well, it's really more a lump of metal, until you start putting some teeth on it and you're ready for the tempering process. /s

JoshMcguigan 7y ago

This was really neat, but I wish the author went into similar detail for the auto white balance step.

vortico 7y ago
White-balancing is a function f(R, G, B, T_from, T_to) -> (R, G, B) which relies on the model of blackbody radiation and a model of the wavelength distribution definitions behind the (R, G, B) values, i.e. the particular color space you're working in. When working with a manufacturer's RAW format, Auto white-balancing is finding `T_to` so that the color values are "most balanced". It is up to the manufacturer how this is decided (and might be proprietary secret sauce). Custom white-balancing is taking a known white or grayscale object and finding `T_to` which would yield something close to (1, 1, 1) for pixels of that object.
- vortico 7y ago
  I think I accidentally erased a sentence in an edit... When working with a manufacturer's RAW format, there is a defined way to get the values (R, G, B, T_from) as a result of demosaicing. For example, in the author's photo post-demosaicing, the manufacturer calibrates the final data to obtain (I'm completely making this up) T_from = 2700K. Then using that information, you can correctly adjust the temperature to any other temperature using `f` above.

3JPLW 7y ago

Really cool article. I'm left with just one question: Why are the 16-bit RAW values so limited in their dynamic range? Wouldn't sensor manufacturers want to have their pixels able to return values that range the whole way from 0x0000 to 0xffff?

fake-name 7y ago
Very few (if any) digital cameras have 16-bit ADCs. Most cameras use 12 or 14 bit ADCs, so they only use 2^12 or 2^14 of the available values. Generally, the RAW files aren't actually 16 bit, but rather a packed structure of the native ADC size. In this case, they're using 16-bit intermediate bitmaps. They're not 16 bit from the RAW. ------ Fujifilm actually has a few cameras with a sensor that can be put in 16-bit mode, but IIRC it's not enabled in software.
- buildbot 7y ago
  Essentially only very high end digital backs from hasselblad/phase one have 16 bit ADCs, but people disagree if it’s even required, the last two bits are basically noise.
  - kurthr 7y ago
    If the noise is actually white, that may not be a bad thing for filtering.
    - kartickv 7y ago
      How is that?
      - kortex 7y ago
        Additive white gaussian noise is quite well studied and pretty easy to remove with processing. So even if those last few bits are "just noise", if that noise is well-behaved, on the whole image you still have more information. There's also the effect of pixel well depth. I forget the exact interplay, but I believe this usually ends up being the limiting factor before ADC resolution.

Causality1 7y ago

Something that's fascinating to me is that at 1:1 display size, the output of modern cameras doesn't look that much better than the pictures out of old 0.3Mpixel still cameras of nearly twenty years ago. The dynamic range is better and the colors are more vibrant, and the noise floor is lower for dark scenes, but on the whole it still looks pretty crappy at full resolution. Why is that? Could we fix it by using larger CCD/CMOS sensor pixels and sticking to lower total pixel counts?

pvg 7y ago
What images are you comparing? 20 year old digital cameras were around a Mpixel. A modern camera with a decent lense produces much better images. In both cases you'd have to do some work to make sure the thing you're blowing up is exactly in focus.
- deleted 7y ago
  [deleted]
ryacko 7y ago
Lens quality, focusing, time to compress the image, etc. People want to take a few photos in quick succession, which has it’s own issues. Film is superior to digital for those issues, for now.
petascale 7y ago
With higher resolution sensors, small imperfections are more visible. To extract the most from a high-megapixel camera, you need spot-on focusing, a higher quality (more expensive) lens, and more care (e.g. higher shutter speed or tripod) to avoid camera shake during exposure. If you nail all these, I wouldn't call it crappy even at 1:1. But there is a residual imperfection from the Bayer pattern on the sensor, since each pixel records only one color (R, G, or B) and the other two values for that pixel have to be guesstimated from neighboring pixels, so the de-bayering process isn't perfect. One way to fix it is to use a monochrome sensor and color filters, taking 3-4 exposures (luminance/mono plus R/G/B) and stack them. A few cameras on the market have a pixel shift feature that can do something similar - multiple exposures, shifting the sensor one pixel between exposures so each pixel get a true R/G/B sample, and stack them in camera or in post. Edit: Forgot to mention the anti-aliasing filter. It sits in front of the sensor and deliberately blurs the image at the pixel level. This is done to avoid aliasing and moiré artifacts, but obviously has the side effect of not-so-great image quality at the pixel peeping level. The fix for this is to get a camera without an AA filter, many modern high-resolution cameras don't have them. https://www.outdoorphotographer.com/photography-gear/cameras/can-you-go-no-low-pass/ https://www.outdoorphotographer.com/photography-gear/cameras...

arriu 7y ago

If you want to extract the actual raw data, dcraw doesn't really advertise the way to do it. Many of the options for extracting the data still apply some processing/conversion to the underlying data. It is a pretty big mess actually. Each manufacturer has different constants that need to be applied to the data, so the raw data from one device is not comparable to another. Convincing dcraw that you actually want the unaltered data is not straight forward. Anyways, if you really do want to see the raw image, there are a few undocumented flags you can use. > dcraw -E -4 -T *.CR2 This will give you an unprocessed 16-bit tiff file containing the "raw" data. What is interesting is that some camera sensors capture data slightly beyond the image you are presented with. The camera will crop in and debayer the image for you, as will your image editor. See this thread for hilarious lengths people go to in order to get unaltered data from their sensors (monochorme converting a dslr): https://stargazerslounge.com/topic/166334-debayering-a-dslrs-bayer-matrix/page/94/ https://stargazerslounge.com/topic/166334-debayering-a-dslrs...

bufferoverflow 7y ago
That thread is more about removing the physical filters from the DSLR sensors - RGB, UV, infrared to achieve higher resolution and much higher sensitivity.

cfstras 7y ago

Try https://github.com/anuejn/batic https://github.com/anuejn/batic - a shadertoy-like experiment where you can to implement a shader to convert the bayer-pattern.