People should realize that search engines are essentially just huge, generic scrapers. I've written my share of scrapers and they are almost always a major PIT

People should realize that search engines are essentially just huge, generic scrapers. I've written my share of scrapers and they are almost always a major PITA unless you're just doing a drive-by for one or two DOM elements. Scraping often will end up to be extremely time consuming, although there is rarely significant challenge to it. Just a lot of monotonous, mind-numbing, tedious work. My advice to most freelancers is: avoid scrapers until you get some teammates to share in the agony.