4 ms·

Do lidar driving systems have objectively better performance? Genuine question (I don't keep up with driving "AI"). I'm skeptical if they would never kill anyon

Do lidar driving systems have objectively better performance? Genuine question (I don't keep up with driving "AI"). I'm skeptical if they would never kill anyone or cause any crashes. That said, I see sign poles flattened all the time. Paint and tire marks all over barriers, walls, and islands. People crash all the time. The old anecdote game isn't really useful, automatic driving could be 100x better than human drivers in all measurable metrics and you'd still be able to upload dashcam videos of it fucking up.

stefan_ 5y ago
You don't see the difference between the one guy willfully crashing his Airbus into a mountain and Boeing having MCAS that makes the control planes not work? The big fucking difference is that the latter was a system installed into all of them! The failure mode was systematic! A different car with the same software facing a similar situation would go straight for the pole again!
- throwawaylinux 5y ago
  I do. I don't see how your comment addresses what I wrote though.
ra7 5y ago
> Do lidar driving systems have objectively better performance? Yes. CA DMV accident and disengagement data for self driving systems using Lidar show they are indeed better (and by miles). Note that Tesla refuses to share FSD beta data because they classify it as a level 2 system. I'm willing to bet their disengagement rate is in the single digits. For comparison, Waymo and Cruise both had around 30,000 miles per disengagement in 2020 according to CA DMV records.
- kjksf 5y ago
  So you know that you don't have equivalent data for Tesla but that doesn't stop you from claiming that Waymo is better than Tesla? Based on data that neither you nor anyone else have?
  - ra7 5y ago
    There’s no evidence for Tesla FSD having a better safety record. When they used to report to the DMV, they were still nowhere near Waymo and Cruise. Since then, we all know how “capable” FSD really is. There are tons of videos showing FSD screwing up the most basic of scenarios. So it’s a very reasonable conclusion.
  - sidibe 5y ago
    Almost all of the videos on Youtube show interventions every couple minutes at least, let's generously say 1 per 5 miles. Do you think off film they're doing 100k miles without intervention for each of those to make up for it?
    - throwawaylinux 5y ago
      Are you talking about entertainment videos where some random person tests there own car where the cut to the beginning the most difficult routes they have found where previous software and sometimes even human drivers have difficulty, with the uneventful parts often cut or sped up? If so, I've seen quite a few of those for the Tesla FSD beta, but none for any other cars. Not that I try to find them, the Tesla hype must just make those come up more. Do you have any of them with other cars showing them go through these same routes without problems, that FSD struggles with? That might be fun to watch, even if it's not significant actual data.
- throwawaylinux 5y ago
  > Yes. CA DMV accident and disengagement data for self driving systems using Lidar show they are indeed better (and by miles). This seems to be a self reporting scheme for companies testing self driving cars in the state. I could find little or nothing about the details of the testing being done or methodology used by each company. > Note that Tesla refuses to share FSD beta data because they classify it as a level 2 system. I'm willing to bet their disengagement rate is in the single digits. Hypothetically, could company A use company employees as human drivers driven specific routes chosen by the company, while company B used volunteers from the public in their own cars driving their own routes, and both data sets would go into the same CA DMV data? Could company A have any incentive to improve miles per disengagement to the point where they might intensively train their drivers, direct them about certain situations which don't require disengagement, etc., choose to drive on routes known to be less difficult? And could the drivers employed to do so much test driving rapidly learn more about the behavior and know when they can avoid disengagement? If so, then I'm a bit skeptical about the objectivity of that data. Given Tesla FSD numbers are not included, are you saying that data is showing lidar systems are better by showing numbers for e.g., Waymo and Cruise vs other non-Tesla companies which don't use lidar? Which companies are those? I'd like to understand how you're using this CA DMV data.
  - ra7 5y ago
    > This seems to be a self reporting scheme for companies testing self driving cars in the state. I could find little or nothing about the details of the testing being done or methodology used by each company. It is a self reporting scheme. Waymo publishes their methodology at [1], Aurora at [2]. I'm not aware who else publishes their methodology, so it's up to you how much you trust them. > Hypothetically, could company A use company employees as human drivers driven specific routes chosen by the company, while company B used volunteers from the public in their own cars driving their own routes, and both data sets would go into the same CA DMV data? Yes, why shouldn't it go to the same dataset? Company A tests in an area defined by their Operational Design Domain (ODD), company B's ODD is simply everywhere in CA (that's what they've committed to). > Could company A have any incentive to improve miles per disengagement to the point where they might intensively train their drivers, direct them about certain situations which don't require disengagement, etc., choose to drive on routes known to be less difficult? And could the drivers employed to do so much test driving rapidly learn more about the behavior and know when they can avoid disengagement? Sure, they can drive only in routes that are less difficult. But they have a product to develop (a public robotaxi service) and areas to expand. So they don't really have an incentive to exclusively work towards artificially reducing disengagement rate. > Given Tesla FSD numbers are not included, are you saying that data is showing lidar systems are better by showing numbers for e.g., Waymo and Cruise vs other non-Tesla companies which don't use lidar? Which companies are those? I'd like to understand how you're using this CA DMV data. Nope, I indeed mean they are better than Tesla. There are no other (aspiring or current) L4+ systems that don't use lidar. When Tesla used to report disengagement rate in CA, they were still worse than their competitors. That and the tons of YouTube videos showing FSD being unable to handle basic scenarios tells us their current disengagement rate is far worse than competitors. What they should be really doing is start being transparent and report safety data like everyone else. [1] https://waymo.com/safety https://waymo.com/safety [2] https://safetycaseframework.aurora.tech/gsn https://safetycaseframework.aurora.tech/gsn
derstander 5y ago
> Do lidar driving systems have objectively better performance? In one sense yes: accurate ranging is an inherent capability of a lidar system that a passive optical system needs additional algorithms to achieve (e.g., simultaneous localization and mapping (SLAM)). In an active system like lidar you know when you’ve transmitted energy and you can measure the delay until you receive an attenuated version of it scattered from objects in the environment. And you know the velocity of the energy so you can solve for distance.

stefan_ 5y ago

You don't see the difference between the one guy willfully crashing his Airbus into a mountain and Boeing having MCAS that makes the control planes not work? The big fucking difference is that the latter was a system installed into all of them! The failure mode was systematic! A different car with the same software facing a similar situation would go straight for the pole again!

throwawaylinux 5y ago
I do. I don't see how your comment addresses what I wrote though.

ra7 5y ago

> Do lidar driving systems have objectively better performance? Yes. CA DMV accident and disengagement data for self driving systems using Lidar show they are indeed better (and by miles). Note that Tesla refuses to share FSD beta data because they classify it as a level 2 system. I'm willing to bet their disengagement rate is in the single digits. For comparison, Waymo and Cruise both had around 30,000 miles per disengagement in 2020 according to CA DMV records.

kjksf 5y ago
So you know that you don't have equivalent data for Tesla but that doesn't stop you from claiming that Waymo is better than Tesla? Based on data that neither you nor anyone else have?
- ra7 5y ago
  There’s no evidence for Tesla FSD having a better safety record. When they used to report to the DMV, they were still nowhere near Waymo and Cruise. Since then, we all know how “capable” FSD really is. There are tons of videos showing FSD screwing up the most basic of scenarios. So it’s a very reasonable conclusion.
- sidibe 5y ago
  Almost all of the videos on Youtube show interventions every couple minutes at least, let's generously say 1 per 5 miles. Do you think off film they're doing 100k miles without intervention for each of those to make up for it?
  - throwawaylinux 5y ago
    Are you talking about entertainment videos where some random person tests there own car where the cut to the beginning the most difficult routes they have found where previous software and sometimes even human drivers have difficulty, with the uneventful parts often cut or sped up? If so, I've seen quite a few of those for the Tesla FSD beta, but none for any other cars. Not that I try to find them, the Tesla hype must just make those come up more. Do you have any of them with other cars showing them go through these same routes without problems, that FSD struggles with? That might be fun to watch, even if it's not significant actual data.
throwawaylinux 5y ago
> Yes. CA DMV accident and disengagement data for self driving systems using Lidar show they are indeed better (and by miles). This seems to be a self reporting scheme for companies testing self driving cars in the state. I could find little or nothing about the details of the testing being done or methodology used by each company. > Note that Tesla refuses to share FSD beta data because they classify it as a level 2 system. I'm willing to bet their disengagement rate is in the single digits. Hypothetically, could company A use company employees as human drivers driven specific routes chosen by the company, while company B used volunteers from the public in their own cars driving their own routes, and both data sets would go into the same CA DMV data? Could company A have any incentive to improve miles per disengagement to the point where they might intensively train their drivers, direct them about certain situations which don't require disengagement, etc., choose to drive on routes known to be less difficult? And could the drivers employed to do so much test driving rapidly learn more about the behavior and know when they can avoid disengagement? If so, then I'm a bit skeptical about the objectivity of that data. Given Tesla FSD numbers are not included, are you saying that data is showing lidar systems are better by showing numbers for e.g., Waymo and Cruise vs other non-Tesla companies which don't use lidar? Which companies are those? I'd like to understand how you're using this CA DMV data.
- ra7 5y ago
  > This seems to be a self reporting scheme for companies testing self driving cars in the state. I could find little or nothing about the details of the testing being done or methodology used by each company. It is a self reporting scheme. Waymo publishes their methodology at [1], Aurora at [2]. I'm not aware who else publishes their methodology, so it's up to you how much you trust them. > Hypothetically, could company A use company employees as human drivers driven specific routes chosen by the company, while company B used volunteers from the public in their own cars driving their own routes, and both data sets would go into the same CA DMV data? Yes, why shouldn't it go to the same dataset? Company A tests in an area defined by their Operational Design Domain (ODD), company B's ODD is simply everywhere in CA (that's what they've committed to). > Could company A have any incentive to improve miles per disengagement to the point where they might intensively train their drivers, direct them about certain situations which don't require disengagement, etc., choose to drive on routes known to be less difficult? And could the drivers employed to do so much test driving rapidly learn more about the behavior and know when they can avoid disengagement? Sure, they can drive only in routes that are less difficult. But they have a product to develop (a public robotaxi service) and areas to expand. So they don't really have an incentive to exclusively work towards artificially reducing disengagement rate. > Given Tesla FSD numbers are not included, are you saying that data is showing lidar systems are better by showing numbers for e.g., Waymo and Cruise vs other non-Tesla companies which don't use lidar? Which companies are those? I'd like to understand how you're using this CA DMV data. Nope, I indeed mean they are better than Tesla. There are no other (aspiring or current) L4+ systems that don't use lidar. When Tesla used to report disengagement rate in CA, they were still worse than their competitors. That and the tons of YouTube videos showing FSD being unable to handle basic scenarios tells us their current disengagement rate is far worse than competitors. What they should be really doing is start being transparent and report safety data like everyone else. [1] https://waymo.com/safety https://waymo.com/safety [2] https://safetycaseframework.aurora.tech/gsn https://safetycaseframework.aurora.tech/gsn

derstander 5y ago

> Do lidar driving systems have objectively better performance? In one sense yes: accurate ranging is an inherent capability of a lidar system that a passive optical system needs additional algorithms to achieve (e.g., simultaneous localization and mapping (SLAM)). In an active system like lidar you know when you’ve transmitted energy and you can measure the delay until you receive an attenuated version of it scattered from objects in the environment. And you know the velocity of the energy so you can solve for distance.