IIRC there was a presentation from Karpathy talking about the challenges with sensor fusion, particular in resolving divergence between e.g. the vision and the

IIRC there was a presentation from Karpathy talking about the challenges with sensor fusion, particular in resolving divergence between e.g. the vision and the radar stack: https://www.youtube.com/watch?v=NSDTZQdo6H8&t=1949s https://www.youtube.com/watch?v=NSDTZQdo6H8&t=1949s My background is in physics, but I find myself having a growing appreciate for the vision-only stack. It's really challenging building a formal understanding of the world that is robust to outliers that are so numerous as navigating in an urban environment. With vision, you have multiple kinds of information that are highly correlated (colour, spatial distribution, depth, etc) that are self-consistent. Whereas, fusing radar with vision, where object responses to radar are highly geometry & material dependent, is a much harder task. I'm really not an expert, so this reads more as an opinion than an experienced view, but I can see the merits in doubling down on vision.