You could divide this into two kinds of failures: one local to a physical area (such as a traffic light that somehow always looks green to the self-driving came

You could divide this into two kinds of failures: one local to a physical area (such as a traffic light that somehow always looks green to the self-driving cameras) and global ones (such as virus, or a bug like a GPS time rollover causing all cars to do something crazy simultaneously.) Local ones seem unlikely to cause casualties larger than events that fool large numbers of human drivers, like sinkholes or malfunctioning traffic lights or black ice on a freeway. Some of these will probably happen, and the systems will be improved, and they'll happen less and less often. Global ones could be very large, and there are several instances of global, cascading failures in other systems like telephone or data networks. There are known ways to prevent many classes of failures, but getting to 100.000000% is hard.