4 ms·
When the bots come out of cloud services then the IPs are all over the place: it's much harder to do right these days.
by DamonHD 2y ago
When the bots come out of cloud services then the IPs are all over the place: it's much harder to do right these days.
- Joel_Mckay 2y agoRate limiting firewalls and spider traps also work well... There is page referral monitoring, context cookies, and dynamically created link chaff with depth-charge rules. One can dance all day friend, but a black-hole is coming for the entire IP block shortly. And unlike many people, some never remove a providers leased block and route until it is re-sold. =3
- immibis 2y agoI mean, blocking random IP ranges is your prerogative if you don't want to have customers. The scrapers will find ways around, while actual users will be unable to use your site. Residential proxies are something like $5 for 1000.
- Joel_Mckay 2y agoTrue, note domestic ISP IP ranges are published, and unless one deals internationally... don't bother serving people that will never buy anything from your firm anyways. Domestic "Users" functioning as proxies will be tripping usage limits, and getting temporarily banned. Google does this by the way, try hammering their services and find out what happens. Context cookies also immediately flag egregious multi-user routes, and if it is a ISP IP you know its a problem user. If it is over 15 users an hour per IP, than you can be 100% sure its a Tor proxy. we ban over 243000+ IPs, and saw zero impact to our bottom line. Have a nice day, =)
- DamonHD 2y agoThose of us running sites for public information rather than sales cannot make the simple cut-off that you do. And again, none of this is simple. It has taken me a few weeks to establish a usage mechanism that does catch the worst feed pullers, but it still can hurt legit new users. That is an opportunity cost.
- Joel_Mckay 2y agoOne must assume most user IP edge proxies are compromised hosts. If someone paid for that list they were almost certainly conned, as the black hats regularly publish that content on their forums. These folks want as many users as possible in order to hide their nuisance traffic origin in the traffic noise. Allowing users known to have an active RAT or their "proxy friends" on a commercial site is not helping anyone.... especially the victims. https://www.youtube.com/watch?v=aCbfMkh940Q https://www.youtube.com/watch?v=aCbfMkh940Q Worth studying the problem from time to time when you get bored of the antics. These folks are generally uninterested in positively contributing to any community, but rather show up to cause trouble for fun and profit. User API quotas are popular for a reason. =3
- EVa5I7bHFq9mnYK 2y agoExactly, I'm banned or captchaed from half of all web sites these days, because of the AI.
- Joel_Mckay 2y agoTry updating your web browser, as sites often flag outdated user agent strings hard-coded in many bots/spiders. Have a nice day, =3
- EVa5I7bHFq9mnYK 2y agoI am sure that bots/spiders use the very best and latest and least suspicious user agent strings.
- DamonHD 2y agoSome bad actors do indeed use ancient UA strings.
- immibis 2y agoSo do some users. Thanks for banning me.
- deleted 2y ago[deleted]
- Joel_Mckay 2y agoThe captcha trigger events on many sites will often keep nagging/blocking people till they update. Don't take this trend personally friend, as if we see a fake iPhone sporting bandwidth >1300Mbps+... than the host is getting permanently banned anyway. Have a wonderful day, =3