4 ms·
I feel the pain — it’s very difficult to detect many of the less ethical scrapers. They use residential IP pools, rotate IPs, and provide valid user agents.
by bodantogat 9mo ago
I feel the pain — it’s very difficult to detect many of the less ethical scrapers. They use residential IP pools, rotate IPs, and provide valid user agents.
- ranger_danger 9mo agoAnd many proxy/scraping providers now use real browsers that can automatically bypass cloudflare captchas as well, and a bot with a real browser similarly won't be clicking on any invisible links, so... I am skeptical just how long this will make an appreciable difference.
- trollbridge 9mo agoSo what you’re saying is that a CAPTCHA cannot actually T C and H A.
- chrismorgan 9mo agoTelling humans and computers apart was never the purpose of CAPTCHAs, only how they initially worked. The name has been a complete misnomer for at least a decade now. Its actual purpose is, and has always been, abuse prevention. Has it been successful? Some yes, some no, and a lot of collateral damage. Its mode of operation now looks a lot like inscrutable blacklisting for some plus inconvenience and bad rate limiting for the rest.
- j16sdiz 9mo ago> was never the purpose of CAPTCHAs, TCHA of CAPTCHA is literally "tell computer human apart"
- chrismorgan 9mo agoYou ignored the emphasis and the rest of the sentence. And the rest of the comment. (Also, the T was nominally Turing rather than telling.)
- olyjohn 9mo agoHow does a human abuse a website, and how does a CAPTCHA stop a human from abusing a website when it's designed to let a human in? If it doesn't stop humans from abusing the site, then it must stop... computers from abusing the site. And it stops computers by using the CAPTCHA to tell apart a human and a computer? Am I wrong here?
- chrismorgan 9mo agoI can tell you don’t live in a network Cloudflare hates.
- olyjohn 9mo agoI get hit with them all the time. That doesn't answer how does a human abuse a website? Hitting F5 on the keyboard 1000 times?
- chrismorgan 9mo agoAbuse isn’t just about inducing CPU or network traffic load (in fact I doubt that was even considered when CAPTCHA was first invented). It’s spam, fraud, that kind of thing. A lot of it is bot-perpetrated, but quite a bit is by humans too.
- everybodyknows 9mo ago> residential IP pools So, is this a new profit center for sleazeball household ISPs?
- vbernat 9mo agoNo, this is done by paying app developers to bundle some random SDK. Search for Bright Data.
- jeroenhd 9mo ago"Residential proxy" is just a word for a botnet. Apps and programs come with a trojan built in that offers your device as an exit node to "monetize" their apps. It "residential proxy" sounds a lot better when you're talking to VC investors, though.
- mlrtime 9mo ago"free" vpns do this.