5 ms·

It's always surprised me that there's not more interest in log-scale/floating-point ADCs built directly into camera sensors. Both humans and algorithms care a l

It's always surprised me that there's not more interest in log-scale/floating-point ADCs built directly into camera sensors. Both humans and algorithms care a lot more about a couple-bit difference in dark areas than light, and we happily use floating point numbers to represent high-range values elsewhere in CS.

morcheeba 3y ago
There was a company that did this circa 2003 - SMaL. Their "autobrite" sensor is built to capture log-scale natively. They've switched owners twice since then, but it seems like they're getting more traction in car vision systems than in professional video. https://www.vision-systems.com/cameras-accessories/article/16743047/smal-camera-technologies-releases-cameras-for-security-and-surveillance https://www.vision-systems.com/cameras-accessories/article/1...
- jeffreygoesto 3y ago
  Blast from the past. You could do things like that [0] in 2005... Autobrite was the first thing to go out the window, once we knew how the chip worked we did or own exposure control... [0] https://youtu.be/0UaTYX-ygG8?feature=shared https://youtu.be/0UaTYX-ygG8?feature=shared
verall 3y ago
A lot of the processing steps expect linearity and would have to be reworked for floating point or log scale data. Most HDR sensors are using some kind of logarithmic compression for sensor readout, but I've never really heard of a floating point adc. Google seems to suggest tbey're not readily available.
- raverbashing 3y ago
  Yeah there isn't, it's not that simple, even at sensor level to have that much dynamic range A linear ADC with enough range is usually fine, you can do the math later. But maybe for this case it needs a non-linear element before the ADC? (no idea how log recording needs anything at the HW level)
fiedzia 3y ago
Most recently microphones/recorders started using it for recording sound
- _kb 3y ago
  From my understanding, the ADC's are still fixed point and linear. Two (or more) then run in parallel over different signal levels to produce the 32-bit float output. Encoding audio with different log-scale companding has been around for some time too (since the 1970's) with A-law and mu-law in G.711.
  - u320 3y ago
    It doesn't really matter HOW they do it, as long as you get the advantages of float encoding (practically infinite headroom). Of course if you zoom in enough there will be something in there that uses integers, but this would be true for e.g. a floating point adder as well.
    - numpad0 3y ago
      It should matter that "practiaclly infinite headroom" comes from the fact that the raw samples has 64 bit of dynamic range, than output format being float. (does that mean there is a crossover in the middle of amplitude range that shows LSBs of one of ADCs poking their heads, I mean, in a hypothetical ultra-naive implementation?)
    - whyoh 3y ago
      >the advantages of float encoding (practically infinite headroom) The best implementations have a dynamic range of about 140-150 dBA. Floating point is not needed to achieve that and it isn't always used (look at Stagetec products).
  - FunkyDuckling 3y ago
    I'm not entirely sure what you mean by floating point for an ADC. from a super high level all ADCS do is quantize an analog signal. They take in a voltage from say 0 - 1.8V and quantize that on a 12 bit range. Return a value from 0-4095. You could build one that scales this range with non-linear steps. But this doesn't add any value. We won't get more accuracy at smaller steps. Our noise and accuracy problems won't be solved by this as they are due to thermal noise or mismatch. quantization noise is not the problem. (We already build segmented ADCs to try and do this)
nvy 3y ago
I feel like professional/prosumer photogs, aka the kind of people who buy fancy SLR cameras and serious business lenses, probably already know this stuff. I also suspect that the vast, vast majority of phone users just want subjectively good-looking photos.
- dclowd9901 3y ago
  Yeah, 100%. About 99% of the customer base just wants to take a good photo with the smallest effort possible. Which makes it even more remarkable the company cares enough to include this kind of functionality in a consumer product.
  - LoganDark 3y ago
    > Which makes it even more remarkable the company cares enough to include this kind of functionality in a consumer product. Maybe people are just starting to get sick of this argument being used against everything.
    - robertlagrant 3y ago
      How is this an argument against anything?
      - LoganDark 3y ago
        Whenever someone wants a new feature or gets disrupted by the loss of a previous feature, people go, "you have to think of the consumers. Consumers don't use those features, because consumers are stupid. I can't see why anyone would ever add/keep a feature that's not going to be used by those consumers." The comment I replied to didn't go nearly that far, but it's an argument I've seen so often that I feel compelled to point out that "consumers" are not the only target audience for Apple - they are trying to also market themselves to creators, as well as professionals, and they absolutely know the difference and notice when things like these become available.
        3y ago
kqr 3y ago
Is the quantisation error on a modern 14-something bit sensor really that big of a problem compared to something like the inherent shot noise of dark areas?
numpad0 3y ago
There is no floating point ADC, just a stereo assigned to two volume levels to be stuffed into a float. hardware accelerated HDR on cameras are commonplace these days, especially in dashcams and CCTV cameras.
amaterasu 3y ago
Some sensors do this internally, unusual though. The rest of the high-end ones apply curves manually in software directly at the egress of the sensor. The reason they don't in all cases is that it complicates black level correction, gamut transforms and demosaic operations (without some assumptions).
audunw 3y ago
From an analog design perspective, I don't think that makes sense. Not that I'm an analog designer, but I worked closely with them as a digital designer on CMOS camera sensors. You're already extracting as much information you can from the analog signal on the least significant bits. It's not like designing a log-scale ADC lets you pull more information from the least significant bits. So you don't really have anything to gain. Why make a more complicated analog circuit to extract less information? It's generally better to let the digital side decide what to keep, how to compress the signal, etc. And I should mention that CMOS camera sensors can often do a lot of digital processing right there on the chip. So you can do log-scale conversion or whatever you want before you send the data out of the CMOS camera chip. It might be possible that you could reduce the power consumption of a SAR (successive approximation) ADC by skipping AD conversion of less significant bits if you have signal on the more significant bits. But I doubt the power savings would be very significant.
adql 3y ago
See but you don't necessarily want full range logarithmic. If your darkest pixel is halfway down the ADC range, on linear you're throwing away one bit, on logarithmic you're throwing out way more bits. Just using higher bit linear ADC then converting it to logarithmic in post-processing seems more sensible. Hell you could even go signal magic and merge few photos with different shutter speed together to get most detail. Also proper log-to-in converter like AD8307 costs like $20 so I'd assume doing that woudl bring the sensor price way up if you needed to have a bunch of them.

morcheeba 3y ago

There was a company that did this circa 2003 - SMaL. Their "autobrite" sensor is built to capture log-scale natively. They've switched owners twice since then, but it seems like they're getting more traction in car vision systems than in professional video. https://www.vision-systems.com/cameras-accessories/article/16743047/smal-camera-technologies-releases-cameras-for-security-and-surveillance https://www.vision-systems.com/cameras-accessories/article/1...

jeffreygoesto 3y ago
Blast from the past. You could do things like that [0] in 2005... Autobrite was the first thing to go out the window, once we knew how the chip worked we did or own exposure control... [0] https://youtu.be/0UaTYX-ygG8?feature=shared https://youtu.be/0UaTYX-ygG8?feature=shared

verall 3y ago

A lot of the processing steps expect linearity and would have to be reworked for floating point or log scale data. Most HDR sensors are using some kind of logarithmic compression for sensor readout, but I've never really heard of a floating point adc. Google seems to suggest tbey're not readily available.

raverbashing 3y ago
Yeah there isn't, it's not that simple, even at sensor level to have that much dynamic range A linear ADC with enough range is usually fine, you can do the math later. But maybe for this case it needs a non-linear element before the ADC? (no idea how log recording needs anything at the HW level)

fiedzia 3y ago

Most recently microphones/recorders started using it for recording sound

_kb 3y ago
From my understanding, the ADC's are still fixed point and linear. Two (or more) then run in parallel over different signal levels to produce the 32-bit float output. Encoding audio with different log-scale companding has been around for some time too (since the 1970's) with A-law and mu-law in G.711.
- u320 3y ago
  It doesn't really matter HOW they do it, as long as you get the advantages of float encoding (practically infinite headroom). Of course if you zoom in enough there will be something in there that uses integers, but this would be true for e.g. a floating point adder as well.
  - numpad0 3y ago
    It should matter that "practiaclly infinite headroom" comes from the fact that the raw samples has 64 bit of dynamic range, than output format being float. (does that mean there is a crossover in the middle of amplitude range that shows LSBs of one of ADCs poking their heads, I mean, in a hypothetical ultra-naive implementation?)
  - whyoh 3y ago
    >the advantages of float encoding (practically infinite headroom) The best implementations have a dynamic range of about 140-150 dBA. Floating point is not needed to achieve that and it isn't always used (look at Stagetec products).
- FunkyDuckling 3y ago
  I'm not entirely sure what you mean by floating point for an ADC. from a super high level all ADCS do is quantize an analog signal. They take in a voltage from say 0 - 1.8V and quantize that on a 12 bit range. Return a value from 0-4095. You could build one that scales this range with non-linear steps. But this doesn't add any value. We won't get more accuracy at smaller steps. Our noise and accuracy problems won't be solved by this as they are due to thermal noise or mismatch. quantization noise is not the problem. (We already build segmented ADCs to try and do this)

nvy 3y ago

I feel like professional/prosumer photogs, aka the kind of people who buy fancy SLR cameras and serious business lenses, probably already know this stuff. I also suspect that the vast, vast majority of phone users just want subjectively good-looking photos.

dclowd9901 3y ago
Yeah, 100%. About 99% of the customer base just wants to take a good photo with the smallest effort possible. Which makes it even more remarkable the company cares enough to include this kind of functionality in a consumer product.
- LoganDark 3y ago
  > Which makes it even more remarkable the company cares enough to include this kind of functionality in a consumer product. Maybe people are just starting to get sick of this argument being used against everything.
  - robertlagrant 3y ago
    How is this an argument against anything?
    - LoganDark 3y ago
      Whenever someone wants a new feature or gets disrupted by the loss of a previous feature, people go, "you have to think of the consumers. Consumers don't use those features, because consumers are stupid. I can't see why anyone would ever add/keep a feature that's not going to be used by those consumers." The comment I replied to didn't go nearly that far, but it's an argument I've seen so often that I feel compelled to point out that "consumers" are not the only target audience for Apple - they are trying to also market themselves to creators, as well as professionals, and they absolutely know the difference and notice when things like these become available.
      - 3y ago

kqr 3y ago

Is the quantisation error on a modern 14-something bit sensor really that big of a problem compared to something like the inherent shot noise of dark areas?

numpad0 3y ago

There is no floating point ADC, just a stereo assigned to two volume levels to be stuffed into a float. hardware accelerated HDR on cameras are commonplace these days, especially in dashcams and CCTV cameras.

amaterasu 3y ago

Some sensors do this internally, unusual though. The rest of the high-end ones apply curves manually in software directly at the egress of the sensor. The reason they don't in all cases is that it complicates black level correction, gamut transforms and demosaic operations (without some assumptions).

audunw 3y ago

From an analog design perspective, I don't think that makes sense. Not that I'm an analog designer, but I worked closely with them as a digital designer on CMOS camera sensors. You're already extracting as much information you can from the analog signal on the least significant bits. It's not like designing a log-scale ADC lets you pull more information from the least significant bits. So you don't really have anything to gain. Why make a more complicated analog circuit to extract less information? It's generally better to let the digital side decide what to keep, how to compress the signal, etc. And I should mention that CMOS camera sensors can often do a lot of digital processing right there on the chip. So you can do log-scale conversion or whatever you want before you send the data out of the CMOS camera chip. It might be possible that you could reduce the power consumption of a SAR (successive approximation) ADC by skipping AD conversion of less significant bits if you have signal on the more significant bits. But I doubt the power savings would be very significant.

adql 3y ago

See but you don't necessarily want full range logarithmic. If your darkest pixel is halfway down the ADC range, on linear you're throwing away one bit, on logarithmic you're throwing out way more bits. Just using higher bit linear ADC then converting it to logarithmic in post-processing seems more sensible. Hell you could even go signal magic and merge few photos with different shutter speed together to get most detail. Also proper log-to-in converter like AD8307 costs like $20 so I'd assume doing that woudl bring the sensor price way up if you needed to have a bunch of them.