In many other industries, automation or alternate data/program paths are kept "on" but in a simulation system together with actual driving. So there should be a

In many other industries, automation or alternate data/program paths are kept "on" but in a simulation system together with actual driving. So there should be a period of 5 to 10 years where all tesla cars have the autopilot system active, but controlling a virtual car in a virtual environment. As far as autopilot is concerned it's doing stuff, but only the driver is in control of the vehicle (it's best to disconnect the autopilot system completely from any physical car control systems) Collect data and see where the two systems (one human, one automated) differ and where they react the same, and from there extrapolate how well the system works (vs a baseline of human drivers under all conditions) Do this for 10 years and you'll have a pretty good idea of how to improve autopilot. I'm surprised such a study hasn't been done inside telsa itself. The space shuttle if I remember correctly, does something similar to this with the redundant processors - all computers does all calculations simultaneously, and if there are discrepancies then either use consensus (2 out of 3) or sound the emergency alarm.