8 ms·
The standard for success is not zero deaths, but it should at least be a system no more dangerous than human drivers. Generally speaking, the average human dri
by Veserv 5y ago
The standard for success is not zero deaths, but it should at least be a system no more dangerous than human drivers.
Generally speaking, the average human driver causes a “ding” every 100k miles, a insurance worthy event every 250k miles, a police worthy event every 500k miles, and a fatality every 60M miles. Assuming an average driving speed of 60 mph, which is obviously an overestimation, those are 1.6k, 4k, 8k, and 1M hours respectively.
Evaluating a L2 autonomy system is somewhat challenging since in truth you are evaluating a human+autonomous system, so you should expect the overall safety to increase unless the autonomy system is actively making things more dangerous. However, for a L4/L5 autonomy system, which is Tesla’s goal, we can reasonably approximate the quality by miles per intervention as an intervention is a failure of the autonomy.
As a reasonable baseline based on video evidence of FSD, we can assume that each intervention prevents at least a “ding”. Therefore, we can evaluate the quality of FSD relative to a minimal L4 system as the ratio of time per disengagement versus 1.6k hours. Using those same videos as evidence, we see the FSD system requiring intervention every few minutes, i.e. <10 minutes, on average. That is 1/10,000 of the minimal quality expected from a fully functional L4 system.
So, for Tesla to create a minimal acceptable L4 autonomy solution that would actually usher in a age of safe self driving systems, they need to improve the core functionality of their beta product by 10,000x. It is that gap between what they are delivering and what they still need to achieve that is outrageous.
- jquery 5y agoVery well said. You’ve put into words what my intuition has been screaming at me but has been unable to properly formulate. Thank you.
- jimmydorry 5y agoIn this stage of the product development the rate of intervention is not, and likely won't be for a long time, a good measure of failure. We likely won't see full L4 systems for a very long time. In the mean time, we could start to get a better handle on the safety profile of these systems by better classifying the interventions. Something like these: * Intervention due to unreadable conditions (e.g. unpainted lines, a new area with no training data, failure of LIDAR / camera, w/e) * Human intervention due to not feeling safe * Intervention due to unsafe condition (e.g. imminent collision, or just hit something) * Human intervention for other reason (e.g. human is bored, or human changing destination, or human wanting to take another route) --- #1 could certainly lead to #2 and #3, but is more of an indicator that the car is in novel conditions not trained for, or has hardware failure. #1 is the low hanging fruit (relatively speaking), and #2 is where the majority of the work lies. #3 is where the system has actually failed. And #4 need to just be discarded.
- Smoosh 5y agoI agree with you overall, but would question two parts of your post. > we can assume that each intervention prevents at least a “ding”. I don't think we can be sure that the vehicle wouldn't have corrected itself in time, nor how many would have been much worse than a "ding". This is where some closed course testing with simulated pedestrians and bicycles etc may give more evidence. In evaluating self driving software, knowing what the consequences of a failure are is important. As you point out, safety drivers prevent this assessment. > based on video evidence of FSD I think that there is selection bias in the readily available footage. Firstly, the short clips are likely to be where there is an incident. The longer ones are usually beta testers deliberately choosing tricky routes to test the FSD. I don't see anyone posting videos of a 3 hour freeway drive where nothing happens. But my point is just that this is complex, and we should be much more careful than we are being.