4 ms·

Exactly. The dynamic range of the human eye is vastly better than a visible spectrum camera.

extralego 9y ago
It's not so simple. Technically, you are not wrong but a video feed should have been sufficient here. It should also be considered that digital video has improved drastically over the last decade. Even LIDAR aside, computer vision and a raw video feed should have been enough to have prevented this collision. When a digital camera records an image, a gamma curve is applied to it before display, which makes up for our bias against the darker portions which the digital equipment does not have. We are very capable of guessing the results of bright conditions but not dark conditions via compressed video. Moreso, these cars should not be using consumer CCDs with compression. They should be utilizing the full possible scope of video. See: https://en.wikipedia.org/wiki/Gamma_correction https://en.wikipedia.org/wiki/Gamma_correction
- taneq 9y ago
  > When a digital camera records an image, a gamma curve is applied to it before display, which makes up for our bias against the darker portions which the digital equipment does not have. Gamma correction makes up for a bias against darker portions in the display, not in our eyes. It's a holdover from the CRT days where the change in brightness between pixel values of, say, 10 and 11, was far less than the change between 250 and 251. Human eyes have excellent low-light discernment which is why 'black' doesn't really look black and you can make out blocky shapes during dark scenes on some DVDs.
  - extralego 9y ago
    Compressed video lacks information in the blacks and that is why we see blocks. The blocks are not there before compression, so it’s not simply a matter of detecting them. While we are good at seeing objects in blacks, your explanation alone doesn’t account for why compression algorithms reason to remove so much of that data. Maybe we are saying the same thing. It’s hard to tell. Your assertion about the origins, however, are at odds with what I have been taught, my understanding, and all the supporting info I am finding in a quick search. My understanding is that luminance values from a sensor have something of an empirical scale but I’m sure this no complete explanation. I am speaking from my working knowledge. I can’t find anything supporting that it is simply a fix for discrepancies between display types. Can you link to something or explain what I am missing?
heartbreak 9y ago
> The dynamic range of the human eye is vastly better than a visible spectrum camera. Certainly better than any camera mounted on a dashboard. It's honestly a bit surreal how the pedestrian appears out of the splotch of pure darkness in the frame. That's low dynamic range and resolution (or high compression) at work, not how light behaves in reality.
- flatline 9y ago
  This was my initial reaction as well, but am not 100% certain after thinking about it some more. Take this for example: https://i.imgur.com/AlO4h7p.gifv https://i.imgur.com/AlO4h7p.gifv I figured that light in front of the car was mostly just messing with the camera but that driver sure didn’t see that pedestrian either. I’m willing to give a human driver the benefit of the doubt here and say that even with eyes on the road and hands on the wheel the outcome would likely have been the same. The pedestrian was not highly visible - no reflectors, dark clothes, it’s really hard to see people like this.
  - dwighttk 9y ago
    >that driver sure didn’t see that pedestrian either unless their phone was showing video of the road in front of them, I don't know why they would have seen her.
URSpider94 9y ago
Not exactly. The dynamic range of the eye in a single scene is around 12-14 stops, about the same as a high-end digital sensor, like the one in the Canon EOS 5D or Sony Alpha. https://www.cambridgeincolour.com/tutorials/cameras-vs-human-eye.htm https://www.cambridgeincolour.com/tutorials/cameras-vs-human... The eye can gain a lot more stops through adaptation (irising, low-light rod-only vision), but those mechanisms dont come into play when viewing a single scene -- and cameras can also make adjustments, e.g. shutter speed and aperture - to gain as much, if not more, range.
- cmurf 9y ago
  A camera captures the entire scene in a frame with a fixed dynamic range. Human vision builds the scene with spatially variant mapping, the scene is made from many frames with different exposures stacked together in real time. I'm concerned about poor scotopic adaptation due to the rather bright light source inside the car - maybe it's the display he's looking at. I see a prominent amount of light on the ceiling all the way to the back of the car and right on his face. It's really straight forward to collect the actual scene luminances from this particular car interior and exterior in this location, but my estimation is the interior luminance is a bigger problem for adaptation than the street lights because the display he's presumably looking at has a much wider field of view, and he's looking directly at it for a prolonged period of time. It's possible he's not even scotopically adapted because of this. And also why is he even looking at the screen? He's obviously distracted by something. Is this required for testing? Ostensibly he's supposed to drive the car first. Is this display standard equipment? Or is it unique to it being an Uber? Or is it an entertainment device? Retest with an OEM lit interior whose driver is paying attention. We already know the autonomous setup failed. But barriers are in place than also increase the potential for the human backup driver to fail.
  - URSpider94 9y ago
    I agree, but I don’t think the eye can adapt beyond its inherent dynamic range over a matter of milliseconds - the iris is not opening or closing over that timescale, so you’re relying on the inherent dynamic range of the retina (which is pretty good). What the eye IS doing is some kind of HDR processing, which is much better than the gamma and levels applied to that video. I bet a professional colorist could grade that footage to make it a much better reflection of what the driver could see in the shadows - even with a crappy camera, you can usually pull out quite a bit of shadow detail.

It's not so simple. Technically, you are not wrong but a video feed should have been sufficient here. It should also be considered that digital video has improved drastically over the last decade. Even LIDAR aside, computer vision and a raw video feed should have been enough to have prevented this collision. When a digital camera records an image, a gamma curve is applied to it before display, which makes up for our bias against the darker portions which the digital equipment does not have. We are very capable of guessing the results of bright conditions but not dark conditions via compressed video. Moreso, these cars should not be using consumer CCDs with compression. They should be utilizing the full possible scope of video. See: https://en.wikipedia.org/wiki/Gamma_correction https://en.wikipedia.org/wiki/Gamma_correction

taneq 9y ago
> When a digital camera records an image, a gamma curve is applied to it before display, which makes up for our bias against the darker portions which the digital equipment does not have. Gamma correction makes up for a bias against darker portions in the display, not in our eyes. It's a holdover from the CRT days where the change in brightness between pixel values of, say, 10 and 11, was far less than the change between 250 and 251. Human eyes have excellent low-light discernment which is why 'black' doesn't really look black and you can make out blocky shapes during dark scenes on some DVDs.
- extralego 9y ago
  Compressed video lacks information in the blacks and that is why we see blocks. The blocks are not there before compression, so it’s not simply a matter of detecting them. While we are good at seeing objects in blacks, your explanation alone doesn’t account for why compression algorithms reason to remove so much of that data. Maybe we are saying the same thing. It’s hard to tell. Your assertion about the origins, however, are at odds with what I have been taught, my understanding, and all the supporting info I am finding in a quick search. My understanding is that luminance values from a sensor have something of an empirical scale but I’m sure this no complete explanation. I am speaking from my working knowledge. I can’t find anything supporting that it is simply a fix for discrepancies between display types. Can you link to something or explain what I am missing?

heartbreak 9y ago

> The dynamic range of the human eye is vastly better than a visible spectrum camera. Certainly better than any camera mounted on a dashboard. It's honestly a bit surreal how the pedestrian appears out of the splotch of pure darkness in the frame. That's low dynamic range and resolution (or high compression) at work, not how light behaves in reality.

flatline 9y ago
This was my initial reaction as well, but am not 100% certain after thinking about it some more. Take this for example: https://i.imgur.com/AlO4h7p.gifv https://i.imgur.com/AlO4h7p.gifv I figured that light in front of the car was mostly just messing with the camera but that driver sure didn’t see that pedestrian either. I’m willing to give a human driver the benefit of the doubt here and say that even with eyes on the road and hands on the wheel the outcome would likely have been the same. The pedestrian was not highly visible - no reflectors, dark clothes, it’s really hard to see people like this.
- dwighttk 9y ago
  >that driver sure didn’t see that pedestrian either unless their phone was showing video of the road in front of them, I don't know why they would have seen her.

URSpider94 9y ago

Not exactly. The dynamic range of the eye in a single scene is around 12-14 stops, about the same as a high-end digital sensor, like the one in the Canon EOS 5D or Sony Alpha. https://www.cambridgeincolour.com/tutorials/cameras-vs-human-eye.htm https://www.cambridgeincolour.com/tutorials/cameras-vs-human... The eye can gain a lot more stops through adaptation (irising, low-light rod-only vision), but those mechanisms dont come into play when viewing a single scene -- and cameras can also make adjustments, e.g. shutter speed and aperture - to gain as much, if not more, range.

cmurf 9y ago
A camera captures the entire scene in a frame with a fixed dynamic range. Human vision builds the scene with spatially variant mapping, the scene is made from many frames with different exposures stacked together in real time. I'm concerned about poor scotopic adaptation due to the rather bright light source inside the car - maybe it's the display he's looking at. I see a prominent amount of light on the ceiling all the way to the back of the car and right on his face. It's really straight forward to collect the actual scene luminances from this particular car interior and exterior in this location, but my estimation is the interior luminance is a bigger problem for adaptation than the street lights because the display he's presumably looking at has a much wider field of view, and he's looking directly at it for a prolonged period of time. It's possible he's not even scotopically adapted because of this. And also why is he even looking at the screen? He's obviously distracted by something. Is this required for testing? Ostensibly he's supposed to drive the car first. Is this display standard equipment? Or is it unique to it being an Uber? Or is it an entertainment device? Retest with an OEM lit interior whose driver is paying attention. We already know the autonomous setup failed. But barriers are in place than also increase the potential for the human backup driver to fail.
- URSpider94 9y ago
  I agree, but I don’t think the eye can adapt beyond its inherent dynamic range over a matter of milliseconds - the iris is not opening or closing over that timescale, so you’re relying on the inherent dynamic range of the retina (which is pretty good). What the eye IS doing is some kind of HDR processing, which is much better than the gamma and levels applied to that video. I bet a professional colorist could grade that footage to make it a much better reflection of what the driver could see in the shadows - even with a crappy camera, you can usually pull out quite a bit of shadow detail.