3 ms·
I'm still struggling with how you would address the bots that don't care for speed at all. How much do we care about stopping crawlers that are slower than the
by bob1029 2y ago
I'm still struggling with how you would address the bots that don't care for speed at all.
How much do we care about stopping crawlers that are slower than the average human user? Is this even possible to do given perfect wire-level emulation of a typical UA?
Should I expect pages protected by this technology to periodically Turing test me?
- theamk 2y agothis is explicitly "misbehaving bots" - it's for performance, not against scraping in general. A bot which is slower than average user is likely well-behaving, and therefore is out of scope.
- Terr_ 2y agoI imagine part of the problem is "fast" bots can be disguised as many many different slow ones, and it's not always trivial to figure out that they're all tentacles of the same malign being.
- malfist 2y agoYou got to think though, that "slow" bot to you is a residential ip address hitting one or two endpoints on your website, but to the bot, it's probably doing this to thousands or millions of sites at the same time. Cloudflare is in a unique position to see enough of the Internet traffic to tell if that one ip address is browsing tens of domains at a time or thousands.