Okay, step #1: go search the web for "web spidering" and "web crawling". Go read the relevant articles (and Wikipedia's, too; a good starting point IMHO). Step

Okay, step #1: go search the web for "web spidering" and "web crawling". Go read the relevant articles (and Wikipedia's, too; a good starting point IMHO). Step #2: update this question when you have a question that's answerable. See, entire books can be (and have been) written on the subjectS that you ask about (and that's even before getting started with the legal aspects). As for question number 4: Impossible unless you have strong AI (see also: How Apple automatically collated their map data from three different sources and what they got as a result). You see, fully automated data collection from multiple source will introduce more errors than manual methods (which are slow, OTOH); automated collection and manual verification is necessary.