3 ms·
In practice they're often using botnets. It's what makes this such a difficult problem to deal with. If they weren't it would be trivial to rate-limit or block
by marginalia_nu 3y ago
In practice they're often using botnets. It's what makes this such a difficult problem to deal with. If they weren't it would be trivial to rate-limit or block problematic IPs or subnets.
- RugnirViking 3y agoas I mentioned in the post: why would a legitimate scraper be using a botnet?
- marginalia_nu 3y agoIn most cases, they aren't legitimate scrapers. They have a complete disregard for the resources they use from the websites they scrape and use a botnet to hide their point of origin.