The tricky part of streetview is that it also has accurate localization (damn good GPSes, probably with realtime kinematics) and also accelerometers / compasses

The tricky part of streetview is that it also has accurate localization (damn good GPSes, probably with realtime kinematics) and also accelerometers / compasses and stuff to make sure that when, say, you hit a bump in the road or your camera is angled somehow, the image is fixed to account for that. Maybe you could fix that with enough images and some intelligent matching based on overlaps, but I can forsee it being quite tricky.