3 ms·

If the websites you scrape don't want you to do so, they will update their robots.txt and ban you. If you don't respect that, they will ban your IP and/or take

by ntoshev 18y ago

If the websites you scrape don't want you to do so, they will update their robots.txt and ban you. If you don't respect that, they will ban your IP and/or take legal action. So just try not to piss them off.

cmars232 18y ago
If the websites don't want you to scrape, use proxies and botnets!