Apart from the copyright issues, if you're going to scrape or crawl someone's site.. it's best to be polite about it. I wrote a web crawler a few years back an

Apart from the copyright issues, if you're going to scrape or crawl someone's site.. it's best to be polite about it. I wrote a web crawler a few years back and at the time I didn't really understand the implications of having a crawler grab 20+ pages concurrently from a site. I learned pretty fast when I found a few sites had banned my crawler.. Oops, sorry guys!