This would be true if we could rely on radar being an idealized sensor in which its error follows a normal distribution and in which visions error also follows

This would be true if we could rely on radar being an idealized sensor in which its error follows a normal distribution and in which visions error also follows a normal distribution. Under that world we could just use sensor fusion techniques to combine the two readings. We don't have that sort of normally distributed error though. In practice there are scenarios where radar gets things very wrong, like with bridges. This is is a systematic error, not a normally distributed error. If you use the average rate of error and you combine it with the vision reading the sensor fusion is much worse, not much better, and in practice it resulted historically in break checks at underpasses, followed by drivers taking over to prevent themselves from getting rear ended. In theory you could absolutely do sensor fusion, but it isn't trivial. You need to have a network that can (1) understand the scene and (2) understand the error distributions as they relate to each particular scene. But notice this - vision is usually more reliable than radar, by a hell of a lot, so how exactly are you going to be determining the edge cases where you need to assume a different error distribution for radar? Seeing how tricky this gets? Combining with radar relies on vision being reliable enough to help inform you of how radar is going to fail. Ugly circular dependence right there. Lets say you can compensate for the potential for radar to error dramatically. In theory, this would allow you to have a better reading in some edge cases, but in practice those edge cases don't matter. What are the actual edge cases where it helps? 1. Driving in the dark or while blinded: radar isn't sufficient to drive safely. It can't see lane markings for example. Therefore this isn't a solution. The correct thing to do if you can't drive on vision alone is to not drive. Or - and I'm dead serious here (fully automated logistics is basically so ridiculous for enabling massive wealth that very unrealistic things are worth pursuing to attain it) - modify every road in the world so as to trivialize driving with sensors other than vision. 2. Seeing obstacles that are hidden from sight. This is an edge case that matters and is where you would get the big win, but it isn't actually just a win. It is also a huge complication. Uncertainty matters and you just made it very hard to project a cone of occlusion producing uncertainty because now we are claiming knowledge of occluded areas with an error prone sensor. You need some feedback mechanisms here or you are going to underestimate/overestimate your ignorance in situations involving uncertainty, potentially resulting in bad driving decisions as a consequence. 3. The more complicated you make this, the greater the potential latency to make the decision. Pursue the theoretical best and latency could increase enough so that you delay your decision. However, this is a real time system. It can't afford pure theoretical best, because latency matters a lot. That makes the sensor fusion more useful as something you use in the backend systems that aren't real time - training time and inference time have very different properties. I actually basically agree with your larger point that in theory these other sensors should be able to benefit. However, I disagree that getting them to the point where they do benefit is easy. I think it is actually pretty hard.