In this stage of the product development the rate of intervention is not, and likely won't be for a long time, a good measure of failure. We likely won't see fu

In this stage of the product development the rate of intervention is not, and likely won't be for a long time, a good measure of failure. We likely won't see full L4 systems for a very long time. In the mean time, we could start to get a better handle on the safety profile of these systems by better classifying the interventions. Something like these: * Intervention due to unreadable conditions (e.g. unpainted lines, a new area with no training data, failure of LIDAR / camera, w/e) * Human intervention due to not feeling safe * Intervention due to unsafe condition (e.g. imminent collision, or just hit something) * Human intervention for other reason (e.g. human is bored, or human changing destination, or human wanting to take another route) --- #1 could certainly lead to #2 and #3, but is more of an indicator that the car is in novel conditions not trained for, or has hardware failure. #1 is the low hanging fruit (relatively speaking), and #2 is where the majority of the work lies. #3 is where the system has actually failed. And #4 need to just be discarded.