We're not doing that while driving, though. If we're talking purely about going off memory, there's no reason why machines couldn't build up a similar catalog

We're not doing that while driving, though. If we're talking purely about going off memory, there's no reason why machines couldn't build up a similar catalog (which could be used by every self driving AI once learned). And human ability to judge distances varies significantly between drivers.