5 ms·

The stupid thing is that any device with an orientation sensor is still writing images the wrong way and then setting a flag, expecting every viewing applicatio

The stupid thing is that any device with an orientation sensor is still writing images the wrong way and then setting a flag, expecting every viewing application to rotate the image. The camera knows which way it's oriented, so it should just write the pixels out in the correct order. Write the upper-left pixel first. Then the next one. And so on. WTF.

mavhc 1y ago
Because your non-smartphone camera doesn't have enough ram/speed to do that I assume (when in burst mode) If a smartphone camera is doing it, then bad camera app!
- Joel_Mckay 1y ago
  Most modern camera modules have built in hardware codecs like mjpeg, region of interest selection, and frame mirror/flip options. This is particularly important on smartphones and battery operated devices. However, most smartphone devices simply save the photo the same way regardless of orientation, and simply add a display-rotated flag to the metadata. It can be super annoying sometimes, as one can't really disable the feature on many devices. =3
- Aardwolf 1y ago
  Rotation for speed/efficiency/compression reasons (indeed with PNG's horizontal line filters it can have a compression reason too) should have been a flag part of the compressed image data format and for use by the encoder/decoder only (which does have caveats for renderers to handle partial decoding though... but the point is to have the behavior rigorously specified and encoded in the image format itself and handled by exactly one known place namely the decoder), not part of metadata It's basically a shame that the exif metadata contains things that affect the rendering
- joking 1y ago
  the main reason is probably that the chip is already outputting the image in a lossy format, and if you reorder the pixels you must reencode the image which means degrading the image, so it's much better to just change the exif orientation.
  - lsaferite 1y ago
    > the chip is already outputting the image in a lossy format Could you explain this one?
  - DidYaWipe 1y ago
    Image sensors don't "output images in a lossy format" as far as I know.
  - spider-mario 1y ago
    JPEG can be rotated losslessly. `jpegtran` can do it, for example (and comes with a script called `exifautotran` to automatically normalise the orientation of a bunch of JPEG files at once).
- account42 1y ago
  Burst mode in cameras means the sensor is readout is buffered in RAM while the encoding and writing to persistent storage catches up. Rotating the buffer would be part of the latter and not affect burst speed - and is an insignificant cost anyway.
ralferoo 1y ago
One interesting thing about JPEG is that you can rotate an image with no quality loss. You don't need to convert each 8x8 square to pixels, rotate and convert back, instead you can transform them in the encoded form. So, rotating each 8x8 square is easy, and then rotating the image is just re-ordering the rotated squares.
- pwdisswordfishz 1y ago
  That doesn't seem to apply to images that aren't multiples of 8 in size, does it?
  - justincormack 1y ago
    the stored image is always a multiple of 8, with padding that is ignored (and heavily compressed).
    - pwdisswordfishz 1y ago
      But can this lossless rotation process account for padding not being in the usual place (lower right corner presumably)?
      - mort96 1y ago
        I'm not sure if this is how JPEG implements it, but in H.264, you just have metadata which specifies a crop (since H.264 also encodes in blocks). From some quick Googling, it seems like JPEG also has EXIF data for cropping, so if that's the mechanism that's used to crop off the bottom and right portions today, there's no reason it couldn't also be used to crop off the top and left portions when losslessly rotating an image's blocks.
  - hidroto 1y ago
    are there any cameras that take pictures that are not a multiple of 8 in width and height?
klabb3 1y ago
TIL, and hard agree (on face value). I’ve been struck by this with arbitrary rotation of images depending on application, very annoying. What are the arguments for this? It would seem easier for everyone to rotate and then store exif for the original rotation if necessary.
- kllrnohj 1y ago
  > What are the arguments for this? It would seem easier for everyone to rotate and then store exif for the original rotation if necessary. Performance. Rotation during rendering is often free, whereas the camera would need an intermediate buffer + copy if it's unable to change the way it samples from the sensor itself.
  - airstrike 1y ago
    How is rotation during rendering free?
    - chainingsolid 1y ago
      Pretty much every pixel rendered these days was generated by a shader so gpu side you probably already have way more translation options than just a 90° rotation (likely already being used for a rotation of 0°). You'd likely have to write more code cpu side to handle the case of tell the gpu to rotate this please and handle the UI layout diffrence. Honestly not a lot of code.
    - kllrnohj 1y ago
      For anything GPU-rendered, applying a rotation matrix to a texture sample and/or frame-buffer write is trivially cheap (see also why Vulkan prerotation exists on Android). Even ignoring GPU-rendering, you always are doing a copy as part of rendering and often have some sort of matrix operation anyway at which point concatenating a rotation matrix often doesn't change much of anything.
      - account42 1y ago
Someone 1y ago
> The camera knows which way it's oriented, so it should just write the pixels out in the correct order. Write the upper-left pixel first. Then the next one. And so on. WTF. The hardware likely is optimized for the common case, so I would think that can be a lot slower. It wouldn’t surprise me, for example, if there are image sensors out there that can only be read out in top to bottom, left to right order. Also, with RAW images and sensors that aren’t rectangular grids, I think that would complicate RAW images parsing. Code for that could have to support up to four different formats, depending on how the sensor is designed,
- DidYaWipe 1y ago
  At this point I expect any camera ASICs to be able to incorporate this logic for plenty-fast processing. Or to do it when writing out the image file, after acquiring it to a buffer. Your raw-image idea is interesting. I'm curious as to how photosites' arrangement would play into this.
- account42 1y ago
  Sensors are not read out as JPEG but into intermediate memory. The encoding step can then deal with the needed rotation. RAW images aren't JPEGs so not relevant to the discussion.

mavhc 1y ago

Because your non-smartphone camera doesn't have enough ram/speed to do that I assume (when in burst mode) If a smartphone camera is doing it, then bad camera app!

Joel_Mckay 1y ago
Most modern camera modules have built in hardware codecs like mjpeg, region of interest selection, and frame mirror/flip options. This is particularly important on smartphones and battery operated devices. However, most smartphone devices simply save the photo the same way regardless of orientation, and simply add a display-rotated flag to the metadata. It can be super annoying sometimes, as one can't really disable the feature on many devices. =3
Aardwolf 1y ago
Rotation for speed/efficiency/compression reasons (indeed with PNG's horizontal line filters it can have a compression reason too) should have been a flag part of the compressed image data format and for use by the encoder/decoder only (which does have caveats for renderers to handle partial decoding though... but the point is to have the behavior rigorously specified and encoded in the image format itself and handled by exactly one known place namely the decoder), not part of metadata It's basically a shame that the exif metadata contains things that affect the rendering
joking 1y ago
the main reason is probably that the chip is already outputting the image in a lossy format, and if you reorder the pixels you must reencode the image which means degrading the image, so it's much better to just change the exif orientation.
- lsaferite 1y ago
  > the chip is already outputting the image in a lossy format Could you explain this one?
- DidYaWipe 1y ago
  Image sensors don't "output images in a lossy format" as far as I know.
- spider-mario 1y ago
  JPEG can be rotated losslessly. `jpegtran` can do it, for example (and comes with a script called `exifautotran` to automatically normalise the orientation of a bunch of JPEG files at once).
account42 1y ago
Burst mode in cameras means the sensor is readout is buffered in RAM while the encoding and writing to persistent storage catches up. Rotating the buffer would be part of the latter and not affect burst speed - and is an insignificant cost anyway.

ralferoo 1y ago

One interesting thing about JPEG is that you can rotate an image with no quality loss. You don't need to convert each 8x8 square to pixels, rotate and convert back, instead you can transform them in the encoded form. So, rotating each 8x8 square is easy, and then rotating the image is just re-ordering the rotated squares.

pwdisswordfishz 1y ago
That doesn't seem to apply to images that aren't multiples of 8 in size, does it?
- justincormack 1y ago
  the stored image is always a multiple of 8, with padding that is ignored (and heavily compressed).
  - pwdisswordfishz 1y ago
    But can this lossless rotation process account for padding not being in the usual place (lower right corner presumably)?
    - mort96 1y ago
      I'm not sure if this is how JPEG implements it, but in H.264, you just have metadata which specifies a crop (since H.264 also encodes in blocks). From some quick Googling, it seems like JPEG also has EXIF data for cropping, so if that's the mechanism that's used to crop off the bottom and right portions today, there's no reason it couldn't also be used to crop off the top and left portions when losslessly rotating an image's blocks.
- hidroto 1y ago
  are there any cameras that take pictures that are not a multiple of 8 in width and height?

klabb3 1y ago

TIL, and hard agree (on face value). I’ve been struck by this with arbitrary rotation of images depending on application, very annoying. What are the arguments for this? It would seem easier for everyone to rotate and then store exif for the original rotation if necessary.

kllrnohj 1y ago
> What are the arguments for this? It would seem easier for everyone to rotate and then store exif for the original rotation if necessary. Performance. Rotation during rendering is often free, whereas the camera would need an intermediate buffer + copy if it's unable to change the way it samples from the sensor itself.
- airstrike 1y ago
  How is rotation during rendering free?
  - chainingsolid 1y ago
    Pretty much every pixel rendered these days was generated by a shader so gpu side you probably already have way more translation options than just a 90° rotation (likely already being used for a rotation of 0°). You'd likely have to write more code cpu side to handle the case of tell the gpu to rotate this please and handle the UI layout diffrence. Honestly not a lot of code.
  - kllrnohj 1y ago
    For anything GPU-rendered, applying a rotation matrix to a texture sample and/or frame-buffer write is trivially cheap (see also why Vulkan prerotation exists on Android). Even ignoring GPU-rendering, you always are doing a copy as part of rendering and often have some sort of matrix operation anyway at which point concatenating a rotation matrix often doesn't change much of anything.
    - account42 1y ago

Someone 1y ago

> The camera knows which way it's oriented, so it should just write the pixels out in the correct order. Write the upper-left pixel first. Then the next one. And so on. WTF. The hardware likely is optimized for the common case, so I would think that can be a lot slower. It wouldn’t surprise me, for example, if there are image sensors out there that can only be read out in top to bottom, left to right order. Also, with RAW images and sensors that aren’t rectangular grids, I think that would complicate RAW images parsing. Code for that could have to support up to four different formats, depending on how the sensor is designed,

DidYaWipe 1y ago
At this point I expect any camera ASICs to be able to incorporate this logic for plenty-fast processing. Or to do it when writing out the image file, after acquiring it to a buffer. Your raw-image idea is interesting. I'm curious as to how photosites' arrangement would play into this.
account42 1y ago
Sensors are not read out as JPEG but into intermediate memory. The encoding step can then deal with the needed rotation. RAW images aren't JPEGs so not relevant to the discussion.