7 ms·
This key point really hits home for us. We've had sites that have plenty of hardware behind it get slammed by crappy scrapers that just obliterate the site, eve
by johng 6mo ago
This key point really hits home for us. We've had sites that have plenty of hardware behind it get slammed by crappy scrapers that just obliterate the site, even with caching and behind CloudFlare.
So they steal your content, keep people from actually seeing it on your site and then obliterate your site which requires you to pay more for hosting, all while not making money anymore.
"Uncontrolled web scraping inflicts heavy technical and financial burdens on publishing infrastructure. Automated bots consume massive server and CDN resources without providing any audience engagement, which spikes operational costs while degrading site performance for real human users. One Akamai customer reclaimed 97% of their request volume by using “tarpitting,” which allows publishers to frustrate these bots."