4 ms·
From the article: “…they do so using random User-Agents that overlap with end-users and come from tens of thousands of IP addresses - mostly residential, in un
by cgh 2y ago
From the article:
“…they do so using random User-Agents that overlap with end-users and come from tens of thousands of IP addresses - mostly residential, in unrelated subnets, each one making no more than one HTTP request over any time period we tried to measure - actively and maliciously adapting and blending in with end-user traffic and avoiding attempts to characterize their behavior or block their traffic.”
So it looks like much of the traffic, particularly from China, is indeed using consumer ips to disguise itself. That’s why they blocked based on browser type (MS Edge, in this case).
- dougb5 2y agoThis matches exactly with what I'm seeing on my own sites too and it's from all over the world, not just China. (I described my bot woes a few weeks ago at https://news.ycombinator.com/item?id=43208623 https://news.ycombinator.com/item?id=43208623. The "just block bots!" replies were well-intentioned but naive -- I've still found no signal that works reliably well to distinguish bots from real traffic.)
- kijin 2y agoI saw a fair amount of that kind of behavior, too, mostly around the summer of last year. At some point it dropped off sharply. Over the last few months, at least for the servers I keep an eye on, most of the trouble has been from Chinese cloud IPs. Either the LLM devs got more funding, or maybe the authorities took down the botnet they were using.