I'm to presume that they aren't trying to do SLAM purely with classifiers, though with the evidence of Autopilot's performance thus far, I wouldn't put it past

I'm to presume that they aren't trying to do SLAM purely with classifiers, though with the evidence of Autopilot's performance thus far, I wouldn't put it past them. That aspect definitely isn't in Karpathy's wheelhouse. There's a great presentation from Gabe Sibley (now with Cruise) on camera based SLAM, or "Mobile Robot Perception for Long Term Perception in Novel Environments" that explains quite well the principles underlying how it works. Gabe concludes at the end that probabilistic perception and modelling using cameras isn't reliable enough and that Lidar is likely necessary for safety critical systems such as autonomous vehicles: https://vimeo.com/88273779 https://vimeo.com/88273779