6 ms·

LIDAR also has limitations on angular resolution just as a function of how the sensor works. It's entirely possible that the size of the person/bike on LIDAR wa

LIDAR also has limitations on angular resolution just as a function of how the sensor works. It's entirely possible that the size of the person/bike on LIDAR was just too small until it was too late to stop. Why it didn't even appear to try to stop? You got me, refresh rate on the LIDAR? LIDAR flat out being mounted to high and relying on optical sensors instead for collision avoidance of small targets (like a human head)? I'm guessing, I'd love to see an NTSB report on this.

vowelless 9y ago
> LIDAR also has limitations on angular resolution just as a function of how the sensor works. It's entirely possible that the size of the person/bike on LIDAR was just too small until it was too late to stop. I highly doubt this is the issue. I am not sure what Ubers setup is, but even a standard velodyne should have been able to pick that up based on angular resolution.
- schreiaj 9y ago
  > Realizing that, for a LIDAR with a 400' range at 15hz moving at 40mph you get ~7 samples of a point before you're at it... For at least the first 3 frames that person is going to look like sensor noise. At 110 feet that person (which I'm calling a 2' wide target) is 1 degree of your sensor measurement. Updating this with math not done at midnight: Frame Distance Angular Size of Target Pixel Size 0 400 0.286481285 7.162032123 1 396.0888889 0.289310143 7.232753564 2 388.2666667 0.295138837 7.378470932 3 376.5333333 0.304335969 7.608399215 4 360.8888889 0.317529122 7.93822806 5 341.3333333 0.3357213 8.393032503 6 317.8666667 0.360506726 9.012668144 7 290.4888889 0.394484519 9.862112983 8 259.2 0.442105838 11.05264595 9 224 0.511583054 12.78957636 10 184.8888889 0.619810252 15.4952563 11 141.8666667 0.807794769 20.19486922 12 94.93333333 1.207252619 30.18131548 13 44.08888889 2.600887175 65.02217938 This is based on the velodyne LIDAR specs I could find last night with some quick googling: - 400' range - .04 degree angular resolution - 15hz max update rate If you have more accurate real world experience with these sensors and can share more accurate performance characteristics I can update. These calculations were done assuming a vehicle moving at 40 mph. The stopping distance at that speed is about 110ft. I computed the pixel size by assuming 1 measurement = 1 pixel giving me 9000 pixels per 360 degrees.
  - rightbyte 9y ago
    I think you are using the wrong angular resolution, though. http://velodynelidar.com/docs/datasheet/63-9194_Rev-G_HDL-64E_S3_Spec%20Sheet_Web.pdf http://velodynelidar.com/docs/datasheet/63-9194_Rev-G_HDL-64... Thats the one LIDAR Uber seems to have matching pictures. 5Hz - 20Hz full round sampling rate, lets assume 15 Hz. The resolution in the horizontal plane is dependent on rotational speed, so at 15 Hz it should be 0,26 degrees. (0,35/20*15 = 0.26) For the woman height the angular resolution is 0.4 degrees no matter the rotation speed. Id est, she would have been atleast one pixel wide from 400 feet and about 2 pixels high and growing in size if we assume 2' wide. (Not counting bike). I really see no exuse for Uber messing this up that bad. The LIDAR can't have missed a potential "obstacle" when it got closer, even if the car wouldn't classify it as a human.
    - schreiaj 9y ago
      I was using Rev E because it's the data sheet I had handy. Mostly I was trying to point out that LIDAR is not some magic thing that always sees everything and there's limitations. Frame Distance Angular Size of Target Pixel Size 0.00 400.00 0.29 1.10 1.00 396.09 0.29 1.11 2.00 388.27 0.30 1.14 3.00 376.53 0.30 1.17 4.00 360.89 0.32 1.22 5.00 341.33 0.34 1.29 6.00 317.87 0.36 1.39 7.00 290.49 0.39 1.52 8.00 259.20 0.44 1.70 9.00 224.00 0.51 1.97 10.00 184.89 0.62 2.38 11.00 141.87 0.81 3.11 12.00 94.93 1.21 4.64 13.00 44.09 2.60 10.00 There's with your .26 angular resolution @ 15hz. (I just have a spreadsheet that spits all these out for me.) These are NOT big targets, they could easily have been mistaken for noise and filtered out. All of the LIDAR data I've ever seen has been fairly noisy and did require filtering to get usable information from it. And given the number of frames they get maybe their filtering was just too aggressive.
      - rightbyte 9y ago
        Yes, I agree with you that we can't assume that the car could have noticed the woman from 120 meters from LIDAR data alone. Maybe with some kind of sensor fusion with IR-cameras. But, as it got closer and what the computer though was noise was on about the same place a sane obstacle finder should have given a posetive match. Maybe at 30 - 40 m worst case? At 142 feet the woman probably had (assuming she was 5.5'): asind(5.5/142) = 2.21* => 2.21/0.4 = 5.5 So between 5 and 6 "scanlines" going from left to right over her. Assuming she was 2' wide that's 0.8 degrees which would be 2 to 3 pixels in breadth according to your spread sheet. That's between 10 and 18 pixels (voxels?) that stand out clearly from the flat road around it, exluding the bike. If you wan't to get an idea of how LIDAR data looks Velodyne has free samples and a viewer for less resolution models. http://velodynelidar.com/downloads.html http://velodynelidar.com/downloads.html It pretty hard to identify obstacles far off, but you will still see there is something there. It's especially easy to identify obstacles that are vertical. As she got closer, she would eventually show up clearly on the LIDAR data. But since the car never slowed down or went left, it didn't notice her at all even at point blanc (or did see her but failed to do anything about it).
Ajedi32 9y ago
Why even bother having a LIDAR system on your self driving car if it doesn't have sufficient resolution to detect a person standing right in front of it? This doesn't seem like an edge case at all. Pedestrian crossing the road at a normal walking pace, and no obstructions in the way which would block the car's vision. The fact that it's dark out should be irrelevant to every sensor on that car other than the cameras. Something obviously went terribly wrong here; either with the sensors themselves or the software. Probably both.
- schreiaj 9y ago
  For detecting larger obstacles like buildings or other vehicles would be my guess. Realistically faster sensors should be used to detect obstacles. LIDARs I could find with some cursory googling can run up to 15hz. Computer vision systems can run much faster (I have a little JeVois camera that'll do eyeball tracking at 120hz onboard, I assume something that costs more can do better). But more importantly, you're vastly trivializing the problem - Standing right in front of it, sure the LIDAR will see the person no problem. Standing 110 feet away (which would be min stopping distance at that speed)? Realizing that, for a LIDAR with a 400' range at 15hz moving at 40mph you get ~7 samples of a point before you're at it... For at least the first 3 frames that person is going to look like sensor noise. At 110 feet that person (which I'm calling a 2' wide target) is 1 degree of your sensor measurement. It's not that it's useless or broken, more just this a seriously bad case where optical tracking couldn't work and where LIDAR is particularly ineffective at seeing the person because of how it works. More effective might be dedicated time of flight sensors in the front bumpers, unsure how long a range those can get, but they are also relatively "slow" sensors.
  - sandos 9y ago
    Is 360 degree lidar really needed? A smaller FOV and higher resolution for 120-180 degrees pointed forward seems a better bet.
    - milesokeefe 9y ago
      It’s not mutually exclusive either. You can have lower frequency, lower angular res 360 spinning LIDAR for low granularity general perception, and also have much higher frequency, brighter, and lower FOV (~90-120deg) solid state lidar mounted at the very least on the front corners of the car. We should be absolutely littering these vehicles with sensors, there’s no reason to be conservative at this stage.
Animats 9y ago
Velodyne's current product has more than enough horizontal resolution for that.[1] Look at the pictures. [1] http://velodynelidar.com/blog/128-lasers-car-go-round-round-david-hall-velodynes-new-sensor/ http://velodynelidar.com/blog/128-lasers-car-go-round-round-...
- Retric 9y ago
  The person in that last picture is something like 5 feet from the car which is far to close to be useful at 40MPH. At those speeds what's important is what it sees at 150 and 200 feet and how fast it can refresh.
MertsA 9y ago
When your resolution is low enough to not see this they stop calling it LIDAR and start calling it a rangefinder. If this was actually a fundamental limitation of the sensor then that's the crappiest LIDAR unit I've ever heard of. When I first heard about the accident my initial reaction was "this is why vision only systems are inadequate". The fact that it didn't even detect the object at all before it collided with it with lidar, radar, and vision is inexcusable. This could set fully autonomous cars back enough that forget the cost of one life, this delay could kill tens of thousands because of a preventable accident.

vowelless 9y ago

> LIDAR also has limitations on angular resolution just as a function of how the sensor works. It's entirely possible that the size of the person/bike on LIDAR was just too small until it was too late to stop. I highly doubt this is the issue. I am not sure what Ubers setup is, but even a standard velodyne should have been able to pick that up based on angular resolution.

schreiaj 9y ago
> Realizing that, for a LIDAR with a 400' range at 15hz moving at 40mph you get ~7 samples of a point before you're at it... For at least the first 3 frames that person is going to look like sensor noise. At 110 feet that person (which I'm calling a 2' wide target) is 1 degree of your sensor measurement. Updating this with math not done at midnight: Frame Distance Angular Size of Target Pixel Size 0 400 0.286481285 7.162032123 1 396.0888889 0.289310143 7.232753564 2 388.2666667 0.295138837 7.378470932 3 376.5333333 0.304335969 7.608399215 4 360.8888889 0.317529122 7.93822806 5 341.3333333 0.3357213 8.393032503 6 317.8666667 0.360506726 9.012668144 7 290.4888889 0.394484519 9.862112983 8 259.2 0.442105838 11.05264595 9 224 0.511583054 12.78957636 10 184.8888889 0.619810252 15.4952563 11 141.8666667 0.807794769 20.19486922 12 94.93333333 1.207252619 30.18131548 13 44.08888889 2.600887175 65.02217938 This is based on the velodyne LIDAR specs I could find last night with some quick googling: - 400' range - .04 degree angular resolution - 15hz max update rate If you have more accurate real world experience with these sensors and can share more accurate performance characteristics I can update. These calculations were done assuming a vehicle moving at 40 mph. The stopping distance at that speed is about 110ft. I computed the pixel size by assuming 1 measurement = 1 pixel giving me 9000 pixels per 360 degrees.
- rightbyte 9y ago
  I think you are using the wrong angular resolution, though. http://velodynelidar.com/docs/datasheet/63-9194_Rev-G_HDL-64E_S3_Spec%20Sheet_Web.pdf http://velodynelidar.com/docs/datasheet/63-9194_Rev-G_HDL-64... Thats the one LIDAR Uber seems to have matching pictures. 5Hz - 20Hz full round sampling rate, lets assume 15 Hz. The resolution in the horizontal plane is dependent on rotational speed, so at 15 Hz it should be 0,26 degrees. (0,35/20*15 = 0.26) For the woman height the angular resolution is 0.4 degrees no matter the rotation speed. Id est, she would have been atleast one pixel wide from 400 feet and about 2 pixels high and growing in size if we assume 2' wide. (Not counting bike). I really see no exuse for Uber messing this up that bad. The LIDAR can't have missed a potential "obstacle" when it got closer, even if the car wouldn't classify it as a human.
  - schreiaj 9y ago
    I was using Rev E because it's the data sheet I had handy. Mostly I was trying to point out that LIDAR is not some magic thing that always sees everything and there's limitations. Frame Distance Angular Size of Target Pixel Size 0.00 400.00 0.29 1.10 1.00 396.09 0.29 1.11 2.00 388.27 0.30 1.14 3.00 376.53 0.30 1.17 4.00 360.89 0.32 1.22 5.00 341.33 0.34 1.29 6.00 317.87 0.36 1.39 7.00 290.49 0.39 1.52 8.00 259.20 0.44 1.70 9.00 224.00 0.51 1.97 10.00 184.89 0.62 2.38 11.00 141.87 0.81 3.11 12.00 94.93 1.21 4.64 13.00 44.09 2.60 10.00 There's with your .26 angular resolution @ 15hz. (I just have a spreadsheet that spits all these out for me.) These are NOT big targets, they could easily have been mistaken for noise and filtered out. All of the LIDAR data I've ever seen has been fairly noisy and did require filtering to get usable information from it. And given the number of frames they get maybe their filtering was just too aggressive.
    - rightbyte 9y ago
      Yes, I agree with you that we can't assume that the car could have noticed the woman from 120 meters from LIDAR data alone. Maybe with some kind of sensor fusion with IR-cameras. But, as it got closer and what the computer though was noise was on about the same place a sane obstacle finder should have given a posetive match. Maybe at 30 - 40 m worst case? At 142 feet the woman probably had (assuming she was 5.5'): asind(5.5/142) = 2.21* => 2.21/0.4 = 5.5 So between 5 and 6 "scanlines" going from left to right over her. Assuming she was 2' wide that's 0.8 degrees which would be 2 to 3 pixels in breadth according to your spread sheet. That's between 10 and 18 pixels (voxels?) that stand out clearly from the flat road around it, exluding the bike. If you wan't to get an idea of how LIDAR data looks Velodyne has free samples and a viewer for less resolution models. http://velodynelidar.com/downloads.html http://velodynelidar.com/downloads.html It pretty hard to identify obstacles far off, but you will still see there is something there. It's especially easy to identify obstacles that are vertical. As she got closer, she would eventually show up clearly on the LIDAR data. But since the car never slowed down or went left, it didn't notice her at all even at point blanc (or did see her but failed to do anything about it).

Ajedi32 9y ago

Why even bother having a LIDAR system on your self driving car if it doesn't have sufficient resolution to detect a person standing right in front of it? This doesn't seem like an edge case at all. Pedestrian crossing the road at a normal walking pace, and no obstructions in the way which would block the car's vision. The fact that it's dark out should be irrelevant to every sensor on that car other than the cameras. Something obviously went terribly wrong here; either with the sensors themselves or the software. Probably both.

schreiaj 9y ago
For detecting larger obstacles like buildings or other vehicles would be my guess. Realistically faster sensors should be used to detect obstacles. LIDARs I could find with some cursory googling can run up to 15hz. Computer vision systems can run much faster (I have a little JeVois camera that'll do eyeball tracking at 120hz onboard, I assume something that costs more can do better). But more importantly, you're vastly trivializing the problem - Standing right in front of it, sure the LIDAR will see the person no problem. Standing 110 feet away (which would be min stopping distance at that speed)? Realizing that, for a LIDAR with a 400' range at 15hz moving at 40mph you get ~7 samples of a point before you're at it... For at least the first 3 frames that person is going to look like sensor noise. At 110 feet that person (which I'm calling a 2' wide target) is 1 degree of your sensor measurement. It's not that it's useless or broken, more just this a seriously bad case where optical tracking couldn't work and where LIDAR is particularly ineffective at seeing the person because of how it works. More effective might be dedicated time of flight sensors in the front bumpers, unsure how long a range those can get, but they are also relatively "slow" sensors.
- sandos 9y ago
  Is 360 degree lidar really needed? A smaller FOV and higher resolution for 120-180 degrees pointed forward seems a better bet.
  - milesokeefe 9y ago
    It’s not mutually exclusive either. You can have lower frequency, lower angular res 360 spinning LIDAR for low granularity general perception, and also have much higher frequency, brighter, and lower FOV (~90-120deg) solid state lidar mounted at the very least on the front corners of the car. We should be absolutely littering these vehicles with sensors, there’s no reason to be conservative at this stage.

Animats 9y ago

Velodyne's current product has more than enough horizontal resolution for that.[1] Look at the pictures. [1] http://velodynelidar.com/blog/128-lasers-car-go-round-round-david-hall-velodynes-new-sensor/ http://velodynelidar.com/blog/128-lasers-car-go-round-round-...

Retric 9y ago
The person in that last picture is something like 5 feet from the car which is far to close to be useful at 40MPH. At those speeds what's important is what it sees at 150 and 200 feet and how fast it can refresh.

MertsA 9y ago

When your resolution is low enough to not see this they stop calling it LIDAR and start calling it a rangefinder. If this was actually a fundamental limitation of the sensor then that's the crappiest LIDAR unit I've ever heard of. When I first heard about the accident my initial reaction was "this is why vision only systems are inadequate". The fact that it didn't even detect the object at all before it collided with it with lidar, radar, and vision is inexcusable. This could set fully autonomous cars back enough that forget the cost of one life, this delay could kill tens of thousands because of a preventable accident.