4 ms·

"Yeah, we don't need Lidar at all!" except when it's going to kill you.

ajross 5y ago
People don't tend to discuss this much, but in fact Lidar has somewhat limited horizontal resolution and a ton of noise in the signal. Linear scan-aligned things like poles don't show up well until they're quite close. It's obviously likely it would have seen it before this car did, but Lidar cars would have the same kind of behavior of "Why didn't it see THAT?", just in a different regime. (The best theory I've seen presented for this particular bug, FWIW, is that it was the shadows and training data at fault. Teslas can, obviously, recognize street lights. But this one evaded detection at a distance because it was inside the dark shadow and presumably the contract detection failed to resolve it. And once it was in the near field, the recognizers don't have as much training data and failed to see it.)
- ra7 5y ago
  > People don't tend to discuss this much, but in fact Lidar has somewhat limited horizontal resolution and a ton of noise in the signal. This was probably true in 2016, but not anymore. Waymo’s 5th gen lidar and Argo AI’s lidar all show massive performance and resolution improvements. They have been game changers. Anyone not using lidar in 2021 is just handicapping themselves, as we are seeing with Tesla FSD performance.
  - kjksf 5y ago
    Can you quantify the improvements for us? As far as I know the specs for Waymo's 5th generation lidar are not publicly available. Can you link to them? How much work did you do with Waymo's 5th generation lidar? Or any lidar for that matter? Do you have any qualifications to claim that a lidar that isn't even available to you is capable of doing a very specific task?
    - ra7 5y ago
      The specs are not publicly available, but the comparison to their 4th gen lidar is. As you can see in [1], the resolution improvement is massive, which is what the GP was claiming lidars lack. I don't see a "ton of signal to noise ratio" in the sensor data either as claimed. Same with Argo AI's lidar [2]. If you want to hear from an expert (which I am not), here is Brad Templeton talking about Waymo and Argo's lidar improvements [3]. [1] https://www.youtube.com/watch?v=COgEQuqTAug&t=11601s https://www.youtube.com/watch?v=COgEQuqTAug&t=11601s [2] https://www.youtube.com/watch?v=I_tYxMh3ddA https://www.youtube.com/watch?v=I_tYxMh3ddA [3] https://www.forbes.com/sites/bradtempleton/2021/10/11/new-lidars-from-waymo-and-others-produce-amazing-results https://www.forbes.com/sites/bradtempleton/2021/10/11/new-li...
throwawaylinux 5y ago
Do lidar driving systems have objectively better performance? Genuine question (I don't keep up with driving "AI"). I'm skeptical if they would never kill anyone or cause any crashes. That said, I see sign poles flattened all the time. Paint and tire marks all over barriers, walls, and islands. People crash all the time. The old anecdote game isn't really useful, automatic driving could be 100x better than human drivers in all measurable metrics and you'd still be able to upload dashcam videos of it fucking up.
- stefan_ 5y ago
  You don't see the difference between the one guy willfully crashing his Airbus into a mountain and Boeing having MCAS that makes the control planes not work? The big fucking difference is that the latter was a system installed into all of them! The failure mode was systematic! A different car with the same software facing a similar situation would go straight for the pole again!
  - throwawaylinux 5y ago
    I do. I don't see how your comment addresses what I wrote though.
- ra7 5y ago
  > Do lidar driving systems have objectively better performance? Yes. CA DMV accident and disengagement data for self driving systems using Lidar show they are indeed better (and by miles). Note that Tesla refuses to share FSD beta data because they classify it as a level 2 system. I'm willing to bet their disengagement rate is in the single digits. For comparison, Waymo and Cruise both had around 30,000 miles per disengagement in 2020 according to CA DMV records.
  - kjksf 5y ago
    So you know that you don't have equivalent data for Tesla but that doesn't stop you from claiming that Waymo is better than Tesla? Based on data that neither you nor anyone else have?
bdamm 5y ago
Yeah, I gotta admit, it seems like it could be really useful but only for hitting the brakes when the vision system is driving towards something stationary.

People don't tend to discuss this much, but in fact Lidar has somewhat limited horizontal resolution and a ton of noise in the signal. Linear scan-aligned things like poles don't show up well until they're quite close. It's obviously likely it would have seen it before this car did, but Lidar cars would have the same kind of behavior of "Why didn't it see THAT?", just in a different regime. (The best theory I've seen presented for this particular bug, FWIW, is that it was the shadows and training data at fault. Teslas can, obviously, recognize street lights. But this one evaded detection at a distance because it was inside the dark shadow and presumably the contract detection failed to resolve it. And once it was in the near field, the recognizers don't have as much training data and failed to see it.)

ra7 5y ago
> People don't tend to discuss this much, but in fact Lidar has somewhat limited horizontal resolution and a ton of noise in the signal. This was probably true in 2016, but not anymore. Waymo’s 5th gen lidar and Argo AI’s lidar all show massive performance and resolution improvements. They have been game changers. Anyone not using lidar in 2021 is just handicapping themselves, as we are seeing with Tesla FSD performance.
- kjksf 5y ago
  Can you quantify the improvements for us? As far as I know the specs for Waymo's 5th generation lidar are not publicly available. Can you link to them? How much work did you do with Waymo's 5th generation lidar? Or any lidar for that matter? Do you have any qualifications to claim that a lidar that isn't even available to you is capable of doing a very specific task?
  - ra7 5y ago
    The specs are not publicly available, but the comparison to their 4th gen lidar is. As you can see in [1], the resolution improvement is massive, which is what the GP was claiming lidars lack. I don't see a "ton of signal to noise ratio" in the sensor data either as claimed. Same with Argo AI's lidar [2]. If you want to hear from an expert (which I am not), here is Brad Templeton talking about Waymo and Argo's lidar improvements [3]. [1] https://www.youtube.com/watch?v=COgEQuqTAug&t=11601s https://www.youtube.com/watch?v=COgEQuqTAug&t=11601s [2] https://www.youtube.com/watch?v=I_tYxMh3ddA https://www.youtube.com/watch?v=I_tYxMh3ddA [3] https://www.forbes.com/sites/bradtempleton/2021/10/11/new-lidars-from-waymo-and-others-produce-amazing-results https://www.forbes.com/sites/bradtempleton/2021/10/11/new-li...

throwawaylinux 5y ago

Do lidar driving systems have objectively better performance? Genuine question (I don't keep up with driving "AI"). I'm skeptical if they would never kill anyone or cause any crashes. That said, I see sign poles flattened all the time. Paint and tire marks all over barriers, walls, and islands. People crash all the time. The old anecdote game isn't really useful, automatic driving could be 100x better than human drivers in all measurable metrics and you'd still be able to upload dashcam videos of it fucking up.

stefan_ 5y ago
You don't see the difference between the one guy willfully crashing his Airbus into a mountain and Boeing having MCAS that makes the control planes not work? The big fucking difference is that the latter was a system installed into all of them! The failure mode was systematic! A different car with the same software facing a similar situation would go straight for the pole again!
- throwawaylinux 5y ago
  I do. I don't see how your comment addresses what I wrote though.
ra7 5y ago
> Do lidar driving systems have objectively better performance? Yes. CA DMV accident and disengagement data for self driving systems using Lidar show they are indeed better (and by miles). Note that Tesla refuses to share FSD beta data because they classify it as a level 2 system. I'm willing to bet their disengagement rate is in the single digits. For comparison, Waymo and Cruise both had around 30,000 miles per disengagement in 2020 according to CA DMV records.
- kjksf 5y ago
  So you know that you don't have equivalent data for Tesla but that doesn't stop you from claiming that Waymo is better than Tesla? Based on data that neither you nor anyone else have?

bdamm 5y ago

Yeah, I gotta admit, it seems like it could be really useful but only for hitting the brakes when the vision system is driving towards something stationary.