Even in that hypothetical future the cars would still have to deal with pedestrians, road workers, construction, weather, road damage, flooding, and many other

Even in that hypothetical future the cars would still have to deal with pedestrians, road workers, construction, weather, road damage, flooding, and many other random situations that are much more complicated than just staying in your lane and maintaining a two car lengths gap from the car in front. How does a self driving car deal with a blizzard with white-out conditions or a complete loss of traction due to ice? Yes, humans get into a lot of accidents in these conditions but as a percentage of the time the rate is still remarkably low. If you want to eliminate large classes of those adverse situations then the best approach we have right now is to switch to rail transport. Public transit is truly the way to go!