3 ms·
Maybe a dumb question but what exactly is wrong with banning the IPs? Even if the bots get more IPs over time, surely storing a list of bans is cheaper than ser
by ratmeadow 1y ago
Maybe a dumb question but what exactly is wrong with banning the IPs? Even if the bots get more IPs over time, surely storing a list of bans is cheaper than serving content? Is the worry that the bots will eventually cycle through so many IP ranges that you end up blocking legit users?
- maurycyz 1y agoIt's often one IP (v4!) per one request. It's insane how many resources are being burned on this stupidity. Part of the reason I did this is to get good numbers on how bad the problem is: A link maze is a great way to make otherwise very stealthy bots expose themselves.
- moebrowne 1y agoEven if this is true how long can that be sustained before they start to be recycled? I bet the scrappers make a whole lot more requests than they have IPs
- csomar 1y agoThey are usually using residential IPs through SOCK5. I am not sure how they are getting these residential IPs but it is definitively suspicious. So by blocking these IPs, you are blocking your users. (ie: in many coffeshops, I get the "IP Blocked" banner, my guess is that they are running software on unsuspecting users to route this traffic).
- sroelants 1y agoThey use scammy providers like Bright Data[1] that let app authors embed their malware (for a compensation, I'm sure) which turns users' devices into crawler proxies. [1]: https://brightdata.com/trustcenter/sourcing https://brightdata.com/trustcenter/sourcing
- moebrowne 1y ago> So by blocking these IPs, you are blocking your users. There were 122 million residential internet connections in the US in 2024 so for an app with 1 million users the chance of affecting a single user is <1%. [1] https://docs.fcc.gov/public/attachments/DOC-411463A1.pdf https://docs.fcc.gov/public/attachments/DOC-411463A1.pdf