3 ms·

> Yes. CA DMV accident and disengagement data for self driving systems using Lidar show they are indeed better (and by miles). This seems to be a self reportin

> Yes. CA DMV accident and disengagement data for self driving systems using Lidar show they are indeed better (and by miles). This seems to be a self reporting scheme for companies testing self driving cars in the state. I could find little or nothing about the details of the testing being done or methodology used by each company. > Note that Tesla refuses to share FSD beta data because they classify it as a level 2 system. I'm willing to bet their disengagement rate is in the single digits. Hypothetically, could company A use company employees as human drivers driven specific routes chosen by the company, while company B used volunteers from the public in their own cars driving their own routes, and both data sets would go into the same CA DMV data? Could company A have any incentive to improve miles per disengagement to the point where they might intensively train their drivers, direct them about certain situations which don't require disengagement, etc., choose to drive on routes known to be less difficult? And could the drivers employed to do so much test driving rapidly learn more about the behavior and know when they can avoid disengagement? If so, then I'm a bit skeptical about the objectivity of that data. Given Tesla FSD numbers are not included, are you saying that data is showing lidar systems are better by showing numbers for e.g., Waymo and Cruise vs other non-Tesla companies which don't use lidar? Which companies are those? I'd like to understand how you're using this CA DMV data.

ra7 5y ago
> This seems to be a self reporting scheme for companies testing self driving cars in the state. I could find little or nothing about the details of the testing being done or methodology used by each company. It is a self reporting scheme. Waymo publishes their methodology at [1], Aurora at [2]. I'm not aware who else publishes their methodology, so it's up to you how much you trust them. > Hypothetically, could company A use company employees as human drivers driven specific routes chosen by the company, while company B used volunteers from the public in their own cars driving their own routes, and both data sets would go into the same CA DMV data? Yes, why shouldn't it go to the same dataset? Company A tests in an area defined by their Operational Design Domain (ODD), company B's ODD is simply everywhere in CA (that's what they've committed to). > Could company A have any incentive to improve miles per disengagement to the point where they might intensively train their drivers, direct them about certain situations which don't require disengagement, etc., choose to drive on routes known to be less difficult? And could the drivers employed to do so much test driving rapidly learn more about the behavior and know when they can avoid disengagement? Sure, they can drive only in routes that are less difficult. But they have a product to develop (a public robotaxi service) and areas to expand. So they don't really have an incentive to exclusively work towards artificially reducing disengagement rate. > Given Tesla FSD numbers are not included, are you saying that data is showing lidar systems are better by showing numbers for e.g., Waymo and Cruise vs other non-Tesla companies which don't use lidar? Which companies are those? I'd like to understand how you're using this CA DMV data. Nope, I indeed mean they are better than Tesla. There are no other (aspiring or current) L4+ systems that don't use lidar. When Tesla used to report disengagement rate in CA, they were still worse than their competitors. That and the tons of YouTube videos showing FSD being unable to handle basic scenarios tells us their current disengagement rate is far worse than competitors. What they should be really doing is start being transparent and report safety data like everyone else. [1] https://waymo.com/safety https://waymo.com/safety [2] https://safetycaseframework.aurora.tech/gsn https://safetycaseframework.aurora.tech/gsn
- throwawaylinux 5y ago
  > Yes, why shouldn't it go to the same dataset? I didn't say it should or shouldn't, I was just asking about the data you are using. > Sure, they can drive only in routes that are less difficult. But they have a product to develop (a public robotaxi service) and areas to expand. So they don't really have an incentive to exclusively work towards artificially reducing disengagement rate. Oh they absolutely do if people buying their shares, their services, employing them, or writing legislation, or just general public attitudes are influenced by this data. The marginal cost of cars is so low given the costs involved that it would be trivial to add easy miles while still doing difficult routes to develop the product. Including doing most of the difficult work in other states. > Nope, I indeed mean they are better than Tesla. There are no other (aspiring or current) L4+ systems that don't use lidar. When Tesla used to report disengagement rate in CA, they were still worse than their competitors. That and the tons of YouTube videos showing FSD being unable to handle basic scenarios tells us their current disengagement rate is far worse than competitors. Okay, so not the objective data I wanted but thanks for helping. > What they should be really doing is start being transparent and report safety data like everyone else. I don't think this is really a matter of transparency given all the problems with the data and tests we've just identified. When the products and are made available to the public that would be a different matter.
  - ra7 5y ago
    > Oh they absolutely do if people buying their shares, their services, employing them, or writing legislation, or just general public attitudes are influenced by this data. None of this matters if they don't show product development. They have a market to serve and profitability to think of. There's only so much PR you gain out of artificially reducing disengagement rate. You can't seriously believe this is a real strategy employed by a dozen SDCs. > I don't think this is really a matter of transparency given all the problems with the data and tests we've just identified. When the products and are made available to the public that would be a different matter. Even if you think self reporting data is a problem, the solution is not to encourage companies like Tesla to continue to be opaque and not comply with rules. Since Tesla FSD is already in the public's hands (with more weekly rollouts planned), it is indeed a matter of transparency.

ra7 5y ago

> This seems to be a self reporting scheme for companies testing self driving cars in the state. I could find little or nothing about the details of the testing being done or methodology used by each company. It is a self reporting scheme. Waymo publishes their methodology at [1], Aurora at [2]. I'm not aware who else publishes their methodology, so it's up to you how much you trust them. > Hypothetically, could company A use company employees as human drivers driven specific routes chosen by the company, while company B used volunteers from the public in their own cars driving their own routes, and both data sets would go into the same CA DMV data? Yes, why shouldn't it go to the same dataset? Company A tests in an area defined by their Operational Design Domain (ODD), company B's ODD is simply everywhere in CA (that's what they've committed to). > Could company A have any incentive to improve miles per disengagement to the point where they might intensively train their drivers, direct them about certain situations which don't require disengagement, etc., choose to drive on routes known to be less difficult? And could the drivers employed to do so much test driving rapidly learn more about the behavior and know when they can avoid disengagement? Sure, they can drive only in routes that are less difficult. But they have a product to develop (a public robotaxi service) and areas to expand. So they don't really have an incentive to exclusively work towards artificially reducing disengagement rate. > Given Tesla FSD numbers are not included, are you saying that data is showing lidar systems are better by showing numbers for e.g., Waymo and Cruise vs other non-Tesla companies which don't use lidar? Which companies are those? I'd like to understand how you're using this CA DMV data. Nope, I indeed mean they are better than Tesla. There are no other (aspiring or current) L4+ systems that don't use lidar. When Tesla used to report disengagement rate in CA, they were still worse than their competitors. That and the tons of YouTube videos showing FSD being unable to handle basic scenarios tells us their current disengagement rate is far worse than competitors. What they should be really doing is start being transparent and report safety data like everyone else. [1] https://waymo.com/safety https://waymo.com/safety [2] https://safetycaseframework.aurora.tech/gsn https://safetycaseframework.aurora.tech/gsn

throwawaylinux 5y ago
> Yes, why shouldn't it go to the same dataset? I didn't say it should or shouldn't, I was just asking about the data you are using. > Sure, they can drive only in routes that are less difficult. But they have a product to develop (a public robotaxi service) and areas to expand. So they don't really have an incentive to exclusively work towards artificially reducing disengagement rate. Oh they absolutely do if people buying their shares, their services, employing them, or writing legislation, or just general public attitudes are influenced by this data. The marginal cost of cars is so low given the costs involved that it would be trivial to add easy miles while still doing difficult routes to develop the product. Including doing most of the difficult work in other states. > Nope, I indeed mean they are better than Tesla. There are no other (aspiring or current) L4+ systems that don't use lidar. When Tesla used to report disengagement rate in CA, they were still worse than their competitors. That and the tons of YouTube videos showing FSD being unable to handle basic scenarios tells us their current disengagement rate is far worse than competitors. Okay, so not the objective data I wanted but thanks for helping. > What they should be really doing is start being transparent and report safety data like everyone else. I don't think this is really a matter of transparency given all the problems with the data and tests we've just identified. When the products and are made available to the public that would be a different matter.
- ra7 5y ago
  > Oh they absolutely do if people buying their shares, their services, employing them, or writing legislation, or just general public attitudes are influenced by this data. None of this matters if they don't show product development. They have a market to serve and profitability to think of. There's only so much PR you gain out of artificially reducing disengagement rate. You can't seriously believe this is a real strategy employed by a dozen SDCs. > I don't think this is really a matter of transparency given all the problems with the data and tests we've just identified. When the products and are made available to the public that would be a different matter. Even if you think self reporting data is a problem, the solution is not to encourage companies like Tesla to continue to be opaque and not comply with rules. Since Tesla FSD is already in the public's hands (with more weekly rollouts planned), it is indeed a matter of transparency.