5 ms·

> Small neural networks I believe are the current state of the art (e.g. train to reverse a 16x16 color filter pattern for the given camera). What is currently

by kuschku 10mo ago

> Small neural networks I believe are the current state of the art (e.g. train to reverse a 16x16 color filter pattern for the given camera). What is currently in use by modern digital cameras is all trade secret stuff. Considering you usually shoot RAW, and debayer and process in post, the camera hasn't done any of that. It's only smartphones that might be doing internal AI Debayering, but they're already hallucinating most of the image anyway.

15155 10mo ago
Yes, people usually shoot RAW (anyone spending this much on a camera knows better) - but these cameras default to JPEG and often have dual-capture (RAW+JPEG) modes.
- qubitcoder 10mo ago
  To be clear, they default to JPEG for the image preview on the monitor (LCD screen). Whenever viewing an image on a professional camera, you’re always seeing the resulting JPEG image. The underlying data is always captured as a RAW file, and only discarded if you’ve configured the camera to only store the JPEG image (discarding the original RAW file after processing).
  - 15155 10mo ago
    > Whenever viewing an image on a professional camera Viewing any preview image on any camera implies a debayered version: who says is it JPEG-encoded - why would it need to be? Every time I browse my SD card full of persisted RAWs, is the camera unnecessarily converting to JPEG just to convert it back to bitmap display data? > The underlying data is always captured as a RAW file, and only discarded if you’ve configured the camera to only store the JPEG image (discarding the original RAW file after processing). Retaining only JPEG is the default configuration on all current-generation Sony and Canon mirrorless cameras: you have to go out of your way to persist RAW.
    - account42 9mo ago
      The cameras typically store a camera display sized preview JPEG in the raw files.
liampulles 10mo ago
Sure - if you don't want to do demosaicing on the camera, that's fine. It doesn't mean there is not an algorithm there as an option. If you care about trying to get an image that is as accurate as possible to the scene, then it is well within your interest to use a Convolutional Neural Network based algorithm, since these are amongst the highest performing in terms of measured PSNR (which is what nearly all demosaicing algorithms in academia are measured on). You are maybe thinking of generative AI?
- deleted 10mo ago
  [deleted]
- kuschku 10mo ago
  At least in broadcast/cinema, no one uses CNN for debayering, because why would you? In cinema, you just use a 6K sensor and use conventional debayering for a perfect 4K image. Even the $2000 Sony FX-30 ships with that feature nowadays. Combined with a good optical low pass filter, that'll also avoid any and all moiré noise. In broadcast, if you worry about moiré noise or debayering quality, you just buy a Sony Z750 with a three-chip prism design, which avoids the problem entirely by just having three separate full-resolution sensors.

by kuschku 10mo ago

15155 10mo ago

Yes, people usually shoot RAW (anyone spending this much on a camera knows better) - but these cameras default to JPEG and often have dual-capture (RAW+JPEG) modes.

qubitcoder 10mo ago
To be clear, they default to JPEG for the image preview on the monitor (LCD screen). Whenever viewing an image on a professional camera, you’re always seeing the resulting JPEG image. The underlying data is always captured as a RAW file, and only discarded if you’ve configured the camera to only store the JPEG image (discarding the original RAW file after processing).
- 15155 10mo ago
  > Whenever viewing an image on a professional camera Viewing any preview image on any camera implies a debayered version: who says is it JPEG-encoded - why would it need to be? Every time I browse my SD card full of persisted RAWs, is the camera unnecessarily converting to JPEG just to convert it back to bitmap display data? > The underlying data is always captured as a RAW file, and only discarded if you’ve configured the camera to only store the JPEG image (discarding the original RAW file after processing). Retaining only JPEG is the default configuration on all current-generation Sony and Canon mirrorless cameras: you have to go out of your way to persist RAW.
  - account42 9mo ago
    The cameras typically store a camera display sized preview JPEG in the raw files.

liampulles 10mo ago

Sure - if you don't want to do demosaicing on the camera, that's fine. It doesn't mean there is not an algorithm there as an option. If you care about trying to get an image that is as accurate as possible to the scene, then it is well within your interest to use a Convolutional Neural Network based algorithm, since these are amongst the highest performing in terms of measured PSNR (which is what nearly all demosaicing algorithms in academia are measured on). You are maybe thinking of generative AI?

deleted 10mo ago
[deleted]
kuschku 10mo ago
At least in broadcast/cinema, no one uses CNN for debayering, because why would you? In cinema, you just use a 6K sensor and use conventional debayering for a perfect 4K image. Even the $2000 Sony FX-30 ships with that feature nowadays. Combined with a good optical low pass filter, that'll also avoid any and all moiré noise. In broadcast, if you worry about moiré noise or debayering quality, you just buy a Sony Z750 with a three-chip prism design, which avoids the problem entirely by just having three separate full-resolution sensors.