I love scraping and hate it too. I have limited experience with it, but some clients want you to web scrape competitors sites to steal their customer list etc.

I love scraping and hate it too. I have limited experience with it, but some clients want you to web scrape competitors sites to steal their customer list etc. It is a matter of ethics. I tried C# and a few other languages and it didn't work because some Javascript on the site had anti-scraping script. I declined the offer, as I didn't want to get sued by their competitor they wanted me to web scrape. Web scraping is going the way webbots work now. Have to scrape data to automate stuff.