Really this is something that almost everyone who drives does subconsciously every time they drive. Driving in a new environment is stressful and more difficult

Really this is something that almost everyone who drives does subconsciously every time they drive. Driving in a new environment is stressful and more difficult because you haven't mapped out the terrain and so you're working entirely from what you see and not from a virtual model in your own head. Obviously the Google car's data collection is more extreme and precise than that which happens in your head, but that seems like it's mostly just because they don't yet entirely know what they need. And this, in the long run, will be part of the advantage of these cars once they get the process more automated. I expect that when you can finally buy one of these cars they'll have a (perhaps opt-out) system where terrain data is collected and merged into Google's own system to provide better data for future drives for not just you, but everyone else as well. And that'll be one of the reasons it'll be safer. If also a whole bunch more creepy.