5 ms·

I fully agree with you, except to the Lidar argument. It holds currently, because Lidars are relatively new (compared to cameras). Considering the progress came

by mikejb 8y ago

I fully agree with you, except to the Lidar argument. It holds currently, because Lidars are relatively new (compared to cameras). Considering the progress cameras have made, and apply a similar progress to Lidar. It's ultimately a race between learning enough on mostly-visual input, and developing cheaper and better hardware. It's a race, and I think it's brave to make claims on what tech will develop faster. Also, I don't think a "restart from scratch" is required when moving away from Lidar - I don't have any hard evidence to make that claim and am happy to change my mind, but I haven't found any argument for doing so yet.

amluto 8y ago
> It's ultimately a race between learning enough on mostly-visual input, and developing cheaper and better hardware. I’m not convinced that the camera-based approach is just a learning problem. As far as I know, cameras still have awful dynamic range compared to humans. Remember the Uber crash video where it looked like the victim was only visible for a couple seconds before impact? Cameras need to work a lot better to be suitable for autonomous driving.
- Retric 8y ago
  This really depends on the Camera, some have beyond human level dynamic ranges which we could equip on cars. The cheap Go Pro sensor was likely much worse than what the car was actually using.
  - alexsb92 8y ago
    I don't believe that's actually the case that cameras have better dynamic range than humans. From what I've seen human vision is definitely in the 20+ stop range. It's especially doubtful that cameras that are cheap enough to put on an autonomous system would get anywhere near that.
    - Retric 8y ago
      The human eye uses a lot of tricks to be high effective dynamic range but the retina is a long way from 20 stops. https://en.m.wikipedia.org/wiki/High-dynamic-range_imaging https://en.m.wikipedia.org/wiki/High-dynamic-range_imaging So, in a direct comparison where cameras are allowed to use a similar set of tricks tricks including a fair amount of post processing they actually win. Aka, Retna vs CCD the CCD wins, iris vs iris camera wins etc.
- kwhitefoot 8y ago
  The video that uber released was from a dashcam.
- ricw 8y ago
  I’m unsure about the cameras in the Tesla’s, but modern cameras (Sony A7M3 being the current leader) have a way better view than the average human. So much so that it can take sharp and accurate pictures in the dark while a normal human eye wouldn’t even see anything / it would seem pitch black. As for dynamic range: HDR and the like can easily take care of that. The proliferation of phone cameras has thankfully taken care of this. I don’t think cameras are even remotely a bottleneck here.
  - planteen 8y ago
    How well does the Sony A7M3 work at night in video mode from inside a moving car with oncoming traffic headlights in the FOV? That is really the case that needs to be supported. My guess is that LIDAR is superior in that scenario, but I've never tried a Sony A7M3 either.
    - solarkraft 8y ago
      I'm guessing this is about the A7s - the sensor really is phenomenal. But it's also large and way too expensive for a vehicle (although the dynamic range problem would be somewhat solved with good hdr)
    - amluto 8y ago
      Also, you need to consider an A7M3 with a somewhat dirty lens.
- Gustomaximus 8y ago
  > Remember the Uber crash video I always had the feeling they softened the image. Or at the least it was camera with low quality/settings giving a false impression of what the view was. If you pause the video right before impact the detail is still much poorer than you would expect.
  - Balgair 8y ago
    Oh, without really much doubt, Uber modified the picture. Some person on reddit went out there and took a picture with their phone camera a few days afterwards. The Uber image is pretty clearly modified to seem like the woman 'came out of nowhere' (in terms of bit-depth). https://www.reddit.com/r/SelfDrivingCars/comments/869olw/hdr_photo_from_the_site_of_the_uber_accident/ https://www.reddit.com/r/SelfDrivingCars/comments/869olw/hdr...
    - gamblor956 8y ago
      An HDR picture requires, among other things, sufficient stability and time to capture all of the light, shades, tones, etc. It's more likely that the sensors in the Uber setup are set for fast processing, i.e., less light per frame, rather than for visual fidelity, i.e., more light per frame. After all, faster processing is more important to a vehicle traveling 60 mph than sharp borders. If the shutter time is set low enough, you would get something similar to what Uber released even on a DSLR. Also, as others have pointed out the Uber video was from a dashcam. The point of the dashcam was to record events in the event of an accident, not be a high-fidelity visual record.
      - Balgair 8y ago
        Still, that means they put out their most favorable image, one where the woman just appears from the inky darkness, not the ones that the car was actually 'seeing'. If anything, this further demonizes Uber (past Greyball, the rampant sexual harassment, and the other crumminess)
    - stordoff 8y ago
      A HDR still and a normal video are going to look very different, especially when you consider the video was taking from a moving car. It proves that Uber's footage was of low quality, but not that it was modified. A single image is always going to be a poor way of judging the light levels, because the exposure (shutter speed, aperture, ISO) can make it look drastically different.

by mikejb 8y ago

amluto 8y ago

> It's ultimately a race between learning enough on mostly-visual input, and developing cheaper and better hardware. I’m not convinced that the camera-based approach is just a learning problem. As far as I know, cameras still have awful dynamic range compared to humans. Remember the Uber crash video where it looked like the victim was only visible for a couple seconds before impact? Cameras need to work a lot better to be suitable for autonomous driving.

Retric 8y ago
This really depends on the Camera, some have beyond human level dynamic ranges which we could equip on cars. The cheap Go Pro sensor was likely much worse than what the car was actually using.
- alexsb92 8y ago
  I don't believe that's actually the case that cameras have better dynamic range than humans. From what I've seen human vision is definitely in the 20+ stop range. It's especially doubtful that cameras that are cheap enough to put on an autonomous system would get anywhere near that.
  - Retric 8y ago
    The human eye uses a lot of tricks to be high effective dynamic range but the retina is a long way from 20 stops. https://en.m.wikipedia.org/wiki/High-dynamic-range_imaging https://en.m.wikipedia.org/wiki/High-dynamic-range_imaging So, in a direct comparison where cameras are allowed to use a similar set of tricks tricks including a fair amount of post processing they actually win. Aka, Retna vs CCD the CCD wins, iris vs iris camera wins etc.
kwhitefoot 8y ago
The video that uber released was from a dashcam.
ricw 8y ago
I’m unsure about the cameras in the Tesla’s, but modern cameras (Sony A7M3 being the current leader) have a way better view than the average human. So much so that it can take sharp and accurate pictures in the dark while a normal human eye wouldn’t even see anything / it would seem pitch black. As for dynamic range: HDR and the like can easily take care of that. The proliferation of phone cameras has thankfully taken care of this. I don’t think cameras are even remotely a bottleneck here.
- planteen 8y ago
  How well does the Sony A7M3 work at night in video mode from inside a moving car with oncoming traffic headlights in the FOV? That is really the case that needs to be supported. My guess is that LIDAR is superior in that scenario, but I've never tried a Sony A7M3 either.
  - solarkraft 8y ago
    I'm guessing this is about the A7s - the sensor really is phenomenal. But it's also large and way too expensive for a vehicle (although the dynamic range problem would be somewhat solved with good hdr)
  - amluto 8y ago
    Also, you need to consider an A7M3 with a somewhat dirty lens.
Gustomaximus 8y ago
> Remember the Uber crash video I always had the feeling they softened the image. Or at the least it was camera with low quality/settings giving a false impression of what the view was. If you pause the video right before impact the detail is still much poorer than you would expect.
- Balgair 8y ago
  Oh, without really much doubt, Uber modified the picture. Some person on reddit went out there and took a picture with their phone camera a few days afterwards. The Uber image is pretty clearly modified to seem like the woman 'came out of nowhere' (in terms of bit-depth). https://www.reddit.com/r/SelfDrivingCars/comments/869olw/hdr_photo_from_the_site_of_the_uber_accident/ https://www.reddit.com/r/SelfDrivingCars/comments/869olw/hdr...
  - gamblor956 8y ago
    An HDR picture requires, among other things, sufficient stability and time to capture all of the light, shades, tones, etc. It's more likely that the sensors in the Uber setup are set for fast processing, i.e., less light per frame, rather than for visual fidelity, i.e., more light per frame. After all, faster processing is more important to a vehicle traveling 60 mph than sharp borders. If the shutter time is set low enough, you would get something similar to what Uber released even on a DSLR. Also, as others have pointed out the Uber video was from a dashcam. The point of the dashcam was to record events in the event of an accident, not be a high-fidelity visual record.
    - Balgair 8y ago
      Still, that means they put out their most favorable image, one where the woman just appears from the inky darkness, not the ones that the car was actually 'seeing'. If anything, this further demonizes Uber (past Greyball, the rampant sexual harassment, and the other crumminess)
  - stordoff 8y ago
    A HDR still and a normal video are going to look very different, especially when you consider the video was taking from a moving car. It proves that Uber's footage was of low quality, but not that it was modified. A single image is always going to be a poor way of judging the light levels, because the exposure (shutter speed, aperture, ISO) can make it look drastically different.