The ethics of this are heavily influenced by how much you believe a) this rollout will result in a net decrease in automobile accidents and b) how much we shoul

The ethics of this are heavily influenced by how much you believe a) this rollout will result in a net decrease in automobile accidents and b) how much we should assume drivers will both be informed and intervene if the system missteps. If you believe both are unlikely, or you believe engineers should have a "do no harm" based ethics similar to medicine, this would be unacceptable. But if you believe both are highly and/or think engineering ethics should focus on harm minimization, this makes sense to do and under certain assumptions it becomes unethical to not roll it out. As a concrete example: if you assume that 99% of accidents that would be caused by the system will be prevented by driver interventions, and you think overall the system will reduce the likelihood of an accident by 20%, the question you have to ask yourself is: if 1 in 5 people harmed over the next 6 months we could have saved, should we avoid doing so and wait until only 1 in 200 will not be harmed, in part due to their own negligence? As you can see, the assumed probabilities matter a lot, so I'd be curious how one can come up with good projections. The only case where these probabilities don't matter is if you believe that engineers ought to never create harm where none would have existed otherwise, but when it comes to self driving cars, this is an impossibility since it assumes perfect autonomy. In general, if you buy into the theory that "human + AI" is always going to be smarter than just AI, then arguably even in a world with great autonomous driving, the best system will be one that has both an AI and a human. And that is what we have here, so it's more a question of if AI quality is sufficient or if this is just a bad idea in general given the natural tendencies of humans to be poor co-pilots.