3 ms·

So if the LIDAR did not fail, it was probably the neural network that takes in LIDAR data and makes the decision to brake. Would love to see NTSB releasing the

So if the LIDAR did not fail, it was probably the neural network that takes in LIDAR data and makes the decision to brake. Would love to see NTSB releasing the data for us to analyze.

jefft255 9y ago
Just to be clear, neural networks typically aren't used to work with lidar in self driving cars. NN for point clouds are still at the research stage.
- stevew20 9y ago
  I was really hoping I wouldn't need to be the first person to point this out! Thanks Jeff
  - abakker 9y ago
    My understanding is that you don’t do analysis or training inside the control loop at all. Basically, build a model and then it becomes a go/no-go check when you really want to use it.
- adrianmonk 9y ago
  Does this mean this is just a research project that Uber doesn't actually use in their cars yet? From https://eng.uber.com/sbnet/ https://eng.uber.com/sbnet/ "By applying convolutional neural networks (CNNs) and other deep learning techniques, researchers at Uber ATG Toronto are committed to developing technologies that power safer and more reliable transportation solutions." "CNNs are widely used for analyzing visual imagery and data from LiDAR sensors. In autonomous driving, CNNs allow self-driving vehicles to see other cars and pedestrians"
  - jefft255 9y ago
    There are many projects using deep learning with lidar. Google PointNet, PointNet++, also https://www.youtube.com/watch?v=UXHX9kFGXfg https://www.youtube.com/watch?v=UXHX9kFGXfg . These are all much newer than 2D CNNs and I don't know if it works well enough to actually be used in SDCs. Also, using CNNs on point clouds comes with all sorts of problems.
blensor 9y ago
I don't know how the system is integrated into the self driving logic as a whole. But a few years ago when we were working on a self driving train, LIDAR was used as a system that reports back a list of obstacles (position, size, maybe a simple shape descriptor) or as an obstacle map that is laid out in a grid and shows which cells are occupied and how high the thi g occupying the cell is. If your system that processes the pointcloud and creates this data does not detect faulty (or missing) data from the sensor the higher level logic will happily hum along believing that nothing is amiss. And some LIDARs even have the option to do the processing on the device itself (i.e. IBEO). In which case you can theoretically work with only a list of obstacles reported back by the sensor.

So if the LIDAR did not fail, it was probably the neural network that takes in LIDAR data and makes the decision to brake. Would love to see NTSB releasing the data for us to analyze.

jefft255 9y ago

Just to be clear, neural networks typically aren't used to work with lidar in self driving cars. NN for point clouds are still at the research stage.

stevew20 9y ago
I was really hoping I wouldn't need to be the first person to point this out! Thanks Jeff
- abakker 9y ago
  My understanding is that you don’t do analysis or training inside the control loop at all. Basically, build a model and then it becomes a go/no-go check when you really want to use it.
adrianmonk 9y ago
Does this mean this is just a research project that Uber doesn't actually use in their cars yet? From https://eng.uber.com/sbnet/ https://eng.uber.com/sbnet/ "By applying convolutional neural networks (CNNs) and other deep learning techniques, researchers at Uber ATG Toronto are committed to developing technologies that power safer and more reliable transportation solutions." "CNNs are widely used for analyzing visual imagery and data from LiDAR sensors. In autonomous driving, CNNs allow self-driving vehicles to see other cars and pedestrians"
- jefft255 9y ago
  There are many projects using deep learning with lidar. Google PointNet, PointNet++, also https://www.youtube.com/watch?v=UXHX9kFGXfg https://www.youtube.com/watch?v=UXHX9kFGXfg . These are all much newer than 2D CNNs and I don't know if it works well enough to actually be used in SDCs. Also, using CNNs on point clouds comes with all sorts of problems.

blensor 9y ago

I don't know how the system is integrated into the self driving logic as a whole. But a few years ago when we were working on a self driving train, LIDAR was used as a system that reports back a list of obstacles (position, size, maybe a simple shape descriptor) or as an obstacle map that is laid out in a grid and shows which cells are occupied and how high the thi g occupying the cell is. If your system that processes the pointcloud and creates this data does not detect faulty (or missing) data from the sensor the higher level logic will happily hum along believing that nothing is amiss. And some LIDARs even have the option to do the processing on the device itself (i.e. IBEO). In which case you can theoretically work with only a list of obstacles reported back by the sensor.