I know, it's odd how some people act like all web scraping is a terrible thing, when people have built such large businesses based on it. Companies try very har

I know, it's odd how some people act like all web scraping is a terrible thing, when people have built such large businesses based on it. Companies try very hard to get scraped by Google, and don't like being scraped by other random entities so much - I suppose it depends on your intent. Poorly written spiders are a problem for system admins. If a resource does have a public API, some very positive things about using that instead are that not only is it less resource intensive for them, it is much easier for you to create and maintain. If you're relying on scraping to get specific info from a site, it's a huge Odin when changes to their HTML break your code.