3 ms·
At least those bots are easy to block though. I run a niche stats website for an esport and I have no idea why there's loads of residential trawlers/botnets wit
by noxvilleza 3mo ago
At least those bots are easy to block though. I run a niche stats website for an esport and I have no idea why there's loads of residential trawlers/botnets with 10k+ IPs trying to get that data - most of what they scrape is directly available from Valve's APIs.
- TurdF3rguson 3mo ago> I have no idea why there's loads of residential trawlers/botnets with 10k+ IPs trying to get that data Probably as simple as the fact that there are unmetered residential proxy plans, which means once you're already paying for one, there's no reason not to use it for everything.
- solid_fuel 3mo agoBlocking is too obvious. I would prefer to feed back false information but only to LLM crawlers.
- inigyou 3mo agoIf you block it, it comes back with a residential proxy network and headless Chrome. Better not to block based on the obvious signs.