4 ms·

What? What about dynamic range, exposure control, color information? What happens when the sun is on the horizon and then the car enters and exits a cool-white

by verall 7y ago

What? What about dynamic range, exposure control, color information? What happens when the sun is on the horizon and then the car enters and exits a cool-white lit led tunnel so the entire scene briefly goes black and everything changes colors? There are tons of huge problems in this space these benchmarks wouldn't be meaningful at all for. 1000fps and 4k might be some of the least meaningful benchmarks I have ever heard someone discuss in the realm of self-driving.

bitL 7y ago
Doesn't that depend on hardware? If you have a proper HDR camera that can handle overload of light and records enhanced spectrum without clipping information (those exist), I can't see how does it affect the underlying perceptual algorithms themselves and how would that make those problems suddenly unsolvable. Having a very high sampling frequency allows you to do reasonable decisions on millimeter scales instead of yard/meter ones with lower sampling frequency; if you have some clever voting/heatmap/etc. mechanism in place, you'd get higher certainty that likely scales well with further increase of sampling frequency. So the perception problem is "solved" because a simple increase of frequency eliminates the need of applying more sophisticated methods and gets you to required accuracy demanded by safety protocols.
- verall 7y ago
  > proper HDR camera that can handle overload of light and records enhanced spectrum without clipping information (those exist) _No_, they don't. CMOS sensors are consistently limited by readout speed (bits-per-second off of the sensor), so FPS, resolution, and dynamic range are fundamental tradeoffs. The bleeding edge of HDR, high-framerate, high-res sensor are full of dirty hacks here like PWL and hardware bracketing to achieve what they do, and it is nothing like your quoted figures. 1000 fps limits your exposure time to 1ms unless you are achieving that 1000fps with sensor arrays, which adds tons of complexity to still sensors, let alone trying to get a reasonable picture from a fast moving vehicle. There is a reason if you ever look at a high-framerate camera setup they are using like 3+ studio lamps to throw enough light on the subject. This "high sampling frequency allows you to do decisions on millimeter scales" is speculative nonsense, and does not even try to address camera spacial frequency limitations that would prevent this. I work on HDR CMOS cameras for the automotive market as my day job. I would love a link to one of these enhanced spectrum cameras you speak of.

by verall 7y ago

bitL 7y ago

Doesn't that depend on hardware? If you have a proper HDR camera that can handle overload of light and records enhanced spectrum without clipping information (those exist), I can't see how does it affect the underlying perceptual algorithms themselves and how would that make those problems suddenly unsolvable. Having a very high sampling frequency allows you to do reasonable decisions on millimeter scales instead of yard/meter ones with lower sampling frequency; if you have some clever voting/heatmap/etc. mechanism in place, you'd get higher certainty that likely scales well with further increase of sampling frequency. So the perception problem is "solved" because a simple increase of frequency eliminates the need of applying more sophisticated methods and gets you to required accuracy demanded by safety protocols.

verall 7y ago
> proper HDR camera that can handle overload of light and records enhanced spectrum without clipping information (those exist) _No_, they don't. CMOS sensors are consistently limited by readout speed (bits-per-second off of the sensor), so FPS, resolution, and dynamic range are fundamental tradeoffs. The bleeding edge of HDR, high-framerate, high-res sensor are full of dirty hacks here like PWL and hardware bracketing to achieve what they do, and it is nothing like your quoted figures. 1000 fps limits your exposure time to 1ms unless you are achieving that 1000fps with sensor arrays, which adds tons of complexity to still sensors, let alone trying to get a reasonable picture from a fast moving vehicle. There is a reason if you ever look at a high-framerate camera setup they are using like 3+ studio lamps to throw enough light on the subject. This "high sampling frequency allows you to do decisions on millimeter scales" is speculative nonsense, and does not even try to address camera spacial frequency limitations that would prevent this. I work on HDR CMOS cameras for the automotive market as my day job. I would love a link to one of these enhanced spectrum cameras you speak of.