3 ms·

I lean towards what Velodyne is saying in this situation. I have been working with LiDAR systems for over 4 years of which the last 1.5 years have been towards

by aecs99 9y ago

I lean towards what Velodyne is saying in this situation. I have been working with LiDAR systems for over 4 years of which the last 1.5 years have been towards building autonomous driving vehicles. When I saw the videos, I was truly baffled by how a LiDAR can miss that. I worked with different types of LiDARs (from different manufacturers) and there is a very high chance that the LiDAR point cloud contains all the information corresponding to the person and the bicycle to make a decision. What we need to keep in mind is that sensing an object is different from deciding whether or not to take an action (e.g., hitting brakes, raising alarms, swerving, etc.). Most LiDAR/RADAR/Camera manufacturers only provide input data. It's like saying "hey, I see this". It's up to the perception software to decide whether or not to make a decision. In most cars, relatively simpler decisions are made by the car's perception software (e.g., adaptive cruise control, lane change warning, automatic braking, etc.). Self-driving companies override such systems, and rewire the car such that it is their perception software that makes the decision. So the onus is completely on the self-driving company's software. In this case, it is the perception software developed by Uber to be critiqued - not Velodyne, not Volvo, not the camera manufacturer. It looks like the engineers at Velodyne feel confident that they should (and would have) sensed the person, and hence their statement. I wouldn't doubt them much as they have been in the LiDAR game since DARPA days when self driving was considered experimental. From a different angle, Velodyne may not have much to loose by throwing Uber under the bus - especially when compared to how much their reputation is at stake. This is because Velodyne has several big customers (e.g., Waymo, and almost every other self-driving, mapping company that is serious about getting big). NTSB should and will get access to the point clouds. Uber has a choice of releasing the point clouds to the public - but I highly doubt they will.

blensor 9y ago
I don't understand why everyone is focusing so heavily on LIDAR being at fault or not at fault. The car does have a RADAR as well, which would not help detecting the pedestrian but most certainly the bike she was moving along. I don't know the field of view of the radar but that should have caused an emergency brake as well, shouldn't it?
- mturmon 9y ago
  From a recent publication in an IEEE conference related to intelligent vehicles: "Radar is robust against bad weather, rain and fog; it can measure speed and distance of an object, but it does not provide enough data points to detect obstacle boundaries, and experimental results show that radar is not reliable to detect small obstacles like pedestrians." This would be because the wavelength of lidar is in the micron range while that of vehicle-detection radar is in the mm-cm range. You won't be able to reliably get radar reflections off of mm/cm-scale objects or object elements, or accurately (<1cm) localize object boundaries. Good navigation would require tighter localization. Radars are really good, though, for detection of objects, including identifying moving objects, close up -- canonical examples being walls and other vehicles. Radar sensors are rather cheap (having been in mass production for a long time) so it's common to have one on every bumper or corner.
  - Qworg 9y ago
    To be pithy - humans are big absorptive sacks of water, radar sees us poorly.
  - blensor 9y ago
    If it were "only" a human then the radar would have a hard time seeing the person. But the bicycle is a substantial chunk of metal which should give much stronger echo. And a 10 kg. metal object placed in the path of an autonomous car should cause all kinds of emergency measures to engage. Many high end cards have auto braking systems based on radar (and additional sensors). Mercedes even has a similar scenario on their product page [1] Volvo does so too, as far as I could find. So not only did they not detect the obstacle with two different sensor technologies they may have deactivated the already existing safety features in that car as well. [1] https://www.mercedes-benz.com/en/mercedes-benz/innovation/on-the-radar-screen-recognising-risks-automatically/ https://www.mercedes-benz.com/en/mercedes-benz/innovation/on...
- TillE 9y ago
  > an emergency break I think half the people commenting on this incident have misspelled "brake". It's odd, because that's not something I've observed as a common error before.
codedokode 9y ago
If you worked with LIDARs, maybe you know how much noise do they give in the output? Could not it be that Uber software filtered pedestrian image out as a noise, for example because there was no matching object on a camera or because reflections from the bike looked like a random noise?
- mturmon 9y ago
  Both effects you mention (sensor fusion problem between camera/lidar; spotty lidar reflections from bike) are possible. These problems probably should not have prevented detecting this obstacle, though. But, a lot depends on factors like the range of the pedestrian/bike, the particular Velodyne unit used, and the mode it was used in. One key thing is that lidar reflections off the bike would have been spotty, but lidar off the pedestrian's body should have been pretty good. That's a perhaps 50-cm wide solid object, which is pretty large by these standards. But the number of lidar "footprints" on the target depends on range. You'd have to estimate the range of the target (15m?) and compute the angle subtended by the target (0.5m/15m ~= 0.03 radian ~= 2 degrees), and then compare this to the angular resolution of the Velodyne unit to get a number of footprints-on-target. Perhaps a half dozen, across a couple of left-to-right scan lines. Again, depending on the scan pattern of the particular Velodyne unit in use. The unit should make more than one pass in the time it took to intersect the pedestrian. This should be enough to detect something, if the world-modeling and decision-making software was operating correctly, hence the puzzlement.
- laythea 9y ago
  It is not possible for the algorithm looking at the LIDAR input data to have the same level of discrimination as humans, so this would be a possibly in my opinion.
- blensor 9y ago
  They do have noise, but we are talking about milimeter to centimeter scale (accuracy is < 2cm). So a grown up person is roughly 2 orders of magnitude bigger than the accuracy of the scanner. To give an example how big (or small) this noise would have been in this situation I did a very simple virtual scan of a person with a bicycle at a distance of 15 meters [1] It was scanned with a virtual scanner inside our sensor simulation software, so this is not the real data and should be taken with a grain of salt. [1] http://www.blensor.org/blog_entry_20180323.html http://www.blensor.org/blog_entry_20180323.html
omgwtfbyobbq 9y ago
Would a bunch of plastic bags filled with stuff (plastic bottles, clothes, etc..) tied to a bike be something the LiDAR would see as part of the roadway rather than a set of distinct objects?

by aecs99 9y ago

blensor 9y ago

I don't understand why everyone is focusing so heavily on LIDAR being at fault or not at fault. The car does have a RADAR as well, which would not help detecting the pedestrian but most certainly the bike she was moving along. I don't know the field of view of the radar but that should have caused an emergency brake as well, shouldn't it?

mturmon 9y ago
From a recent publication in an IEEE conference related to intelligent vehicles: "Radar is robust against bad weather, rain and fog; it can measure speed and distance of an object, but it does not provide enough data points to detect obstacle boundaries, and experimental results show that radar is not reliable to detect small obstacles like pedestrians." This would be because the wavelength of lidar is in the micron range while that of vehicle-detection radar is in the mm-cm range. You won't be able to reliably get radar reflections off of mm/cm-scale objects or object elements, or accurately (<1cm) localize object boundaries. Good navigation would require tighter localization. Radars are really good, though, for detection of objects, including identifying moving objects, close up -- canonical examples being walls and other vehicles. Radar sensors are rather cheap (having been in mass production for a long time) so it's common to have one on every bumper or corner.
- Qworg 9y ago
  To be pithy - humans are big absorptive sacks of water, radar sees us poorly.
- blensor 9y ago
  If it were "only" a human then the radar would have a hard time seeing the person. But the bicycle is a substantial chunk of metal which should give much stronger echo. And a 10 kg. metal object placed in the path of an autonomous car should cause all kinds of emergency measures to engage. Many high end cards have auto braking systems based on radar (and additional sensors). Mercedes even has a similar scenario on their product page [1] Volvo does so too, as far as I could find. So not only did they not detect the obstacle with two different sensor technologies they may have deactivated the already existing safety features in that car as well. [1] https://www.mercedes-benz.com/en/mercedes-benz/innovation/on-the-radar-screen-recognising-risks-automatically/ https://www.mercedes-benz.com/en/mercedes-benz/innovation/on...
TillE 9y ago
> an emergency break I think half the people commenting on this incident have misspelled "brake". It's odd, because that's not something I've observed as a common error before.

codedokode 9y ago

If you worked with LIDARs, maybe you know how much noise do they give in the output? Could not it be that Uber software filtered pedestrian image out as a noise, for example because there was no matching object on a camera or because reflections from the bike looked like a random noise?

mturmon 9y ago
Both effects you mention (sensor fusion problem between camera/lidar; spotty lidar reflections from bike) are possible. These problems probably should not have prevented detecting this obstacle, though. But, a lot depends on factors like the range of the pedestrian/bike, the particular Velodyne unit used, and the mode it was used in. One key thing is that lidar reflections off the bike would have been spotty, but lidar off the pedestrian's body should have been pretty good. That's a perhaps 50-cm wide solid object, which is pretty large by these standards. But the number of lidar "footprints" on the target depends on range. You'd have to estimate the range of the target (15m?) and compute the angle subtended by the target (0.5m/15m ~= 0.03 radian ~= 2 degrees), and then compare this to the angular resolution of the Velodyne unit to get a number of footprints-on-target. Perhaps a half dozen, across a couple of left-to-right scan lines. Again, depending on the scan pattern of the particular Velodyne unit in use. The unit should make more than one pass in the time it took to intersect the pedestrian. This should be enough to detect something, if the world-modeling and decision-making software was operating correctly, hence the puzzlement.
laythea 9y ago
It is not possible for the algorithm looking at the LIDAR input data to have the same level of discrimination as humans, so this would be a possibly in my opinion.
blensor 9y ago
They do have noise, but we are talking about milimeter to centimeter scale (accuracy is < 2cm). So a grown up person is roughly 2 orders of magnitude bigger than the accuracy of the scanner. To give an example how big (or small) this noise would have been in this situation I did a very simple virtual scan of a person with a bicycle at a distance of 15 meters [1] It was scanned with a virtual scanner inside our sensor simulation software, so this is not the real data and should be taken with a grain of salt. [1] http://www.blensor.org/blog_entry_20180323.html http://www.blensor.org/blog_entry_20180323.html

omgwtfbyobbq 9y ago

Would a bunch of plastic bags filled with stuff (plastic bottles, clothes, etc..) tied to a bike be something the LiDAR would see as part of the roadway rather than a set of distinct objects?