3 ms·
Web-scraping can do harm to a business if the scraper does not implement things like timeouts or sleeps between processes. Websites and businesses do not facto
by notahaterbut 6y ago
Web-scraping can do harm to a business if the scraper does not implement things like timeouts or sleeps between processes.
Websites and businesses do not factor bandwidth for the rogue scraper.
- PestoDiRucola 6y agoYou can easily block a harmful scraper like that from the website owner's side.
- notahaterbut 6y agowhat if the scraper uses ip rotation or serverless workers ?
- kingdomcome50 6y agoYou make the target. I’ll make the bot. Let’s see who wins. In all seriousness, your suggestion is woefully naive in the face of any serious entity.
- kordlessagain 6y agoThis only applies to crawlers who are set to recurse into URLs.