3 ms·

The ultimate realization of self driving will be successful, and will have profoundly positive impacts on society This is clearly possible in principle. After

The ultimate realization of self driving will be successful, and will have profoundly positive impacts on society This is clearly possible in principle. After all, fully autonomous vehicles can have much more capable sensor systems than any human driver, can process the input from those sensors faster, and can then apply any necessary adjustments to the vehicle's behaviour almost instantaneously, all without ever getting tired or distracted. No human driver will ever match that potential. The problem I have with this whole argument, and thus the reason I'm not sure the moral/ethical arguments stand up either, is that we don't know how to reliably write excellent software yet. The entire argument for autonomous driving rests on the premise that we will at some point in the usefully near future reach a situation where the cars drive themselves much better than we humans collectively do. But there is, to the best of my knowledge, no evidence so far that we as a society can actually create the necessary software yet. Until we can, any autonomous vehicle control system will be at risk of life-threatening bugs. In particular, it will be at risk of widespread failures of the same type that could cause much more harm than any single human driver's failure could, whether caused by defects in the software alone or by a malicious actor exploiting a security vulnerability. That is a huge risk that is getting far too little attention so far IMHO.

PopePompus 6y ago
But humans too have life-threatening bugs, especially when combined with an automobile. Car+driver combinations have blind spots, for example. Human drivers get drunk, get distracted, and spill coffee in their laps. I think autonomous cars will kill people, and they will kill people in situations where human drivers would not. But I'm finding it increasingly easy to imagine that autonomous driving will save lives overall.
- Silhouette 6y ago
  Yes, human driving is far from perfect and sometimes that has tragic consequences. But there are two issues at play here. The first is whether we expect the autonomous vehicles to become safer than the population of human drivers they would be replacing under normal circumstances. This is a population measure, and it's about overall safety and averages, and given enough time and data it seems a realistic outcome. The second is the risk of a single catastrophic failure, which is a risk that doesn't generally exist with today's human-controlled vehicles. It is no exaggeration to say that the kinds of autonomous vehicle technologies we are conceiving here have the potential, if they go wrong, to cause more harm than any WMD ever deployed in war. And not only the theoretical potential, but given the current state of our software development abilities, a non-trivial chance of it actually happening in practice. Until we've dealt with the latter problem, which I don't believe we currently know how to solve, any arguments based on the long-term superiority of autonomous vehicles over the capabilities of any human driver is going to be fundamentally flawed.

PopePompus 6y ago

But humans too have life-threatening bugs, especially when combined with an automobile. Car+driver combinations have blind spots, for example. Human drivers get drunk, get distracted, and spill coffee in their laps. I think autonomous cars will kill people, and they will kill people in situations where human drivers would not. But I'm finding it increasingly easy to imagine that autonomous driving will save lives overall.

Silhouette 6y ago
Yes, human driving is far from perfect and sometimes that has tragic consequences. But there are two issues at play here. The first is whether we expect the autonomous vehicles to become safer than the population of human drivers they would be replacing under normal circumstances. This is a population measure, and it's about overall safety and averages, and given enough time and data it seems a realistic outcome. The second is the risk of a single catastrophic failure, which is a risk that doesn't generally exist with today's human-controlled vehicles. It is no exaggeration to say that the kinds of autonomous vehicle technologies we are conceiving here have the potential, if they go wrong, to cause more harm than any WMD ever deployed in war. And not only the theoretical potential, but given the current state of our software development abilities, a non-trivial chance of it actually happening in practice. Until we've dealt with the latter problem, which I don't believe we currently know how to solve, any arguments based on the long-term superiority of autonomous vehicles over the capabilities of any human driver is going to be fundamentally flawed.