4 ms·

This kind of reasoning assumes the bot continues to be non-stealthy

swiftcoder 1y ago
I mean, forcing them to spend engineering effort the make their bot stealthy (or to be able to maintains 10's of thousands of open ports), is still driving up their costs, so I'd count it as a win. The OP doesn't say why the bot is hitting their endpoints, but I doubt the bot is a profit centre for the operator.
- lagosfractal42 1y ago
  You risk flagging real users as bots, which drives down your profits and reputation
  - swiftcoder 1y ago
    In this case I don't think they do - unless the legitimate users are also hitting your site at 700 RPS (in which case, the added load from the bot is going to be negligible)
    - hansvm 1y ago
      Once the bot is stealthy (the current sub-thread if I haven't misread) they absolutely do. A couple examples where I've been flagged as a bot for normal traffic: 1. Discord's telemetry was broken on my browser, and on failure they immediately retried. It didn't take many actions queued up on the site before my browser was initiating over 100RPS, on their behalf. 2. Target and eBay still flag my sessions as bot traffic (presumably because they don't recognize the user agent or because I use Linux or something). Target allows browsing their site for a few items before heavily rate-limiting me for a day or so, and eBay just resets my password a day or two after I log in, every single bloody time. The problem is that from time to time normal users will generate large traffic volumes, and if the bot owner uses many IPs then you're forced to use less reliable signals for that ban hammer (i.e., no single user will be near 700 RPS).
heavyset_go 1y ago
If going stealth means not blatantly DDoS'ing the OP then that's a better outcome than what's currently happening
somat 1y ago
xkcd 810 comes to mind. https://xkcd.com/810/ https://xkcd.com/810/ "what if we make the bots go stealthy and indistinguishable from actual human requests?" "Mission Accomplished"
- HPsquared 1y ago
  This has pretty much happened now in the internet at large, and it's kinda sad.
  - lotsofpulp 1y ago
    “Constructive” and “Helpful” are unfortunately not out weighed by garbage.
lucastech 1y ago
Yeah, there are some botnets I've been seeing that are much more stealthy, using 900-3000 IP's with rotating user agents to send enormous amounts of traffic. I've resorted to blocking entire AS routes to prevent it (fortunately I am mostly hosting US sites with US only residential audiences). I'm not sure who's behind it, but one of the later data centers is oxylabs, so they're probably involved somehow. https://wxp.io/blog/the-bots-that-keep-on-giving https://wxp.io/blog/the-bots-that-keep-on-giving

I mean, forcing them to spend engineering effort the make their bot stealthy (or to be able to maintains 10's of thousands of open ports), is still driving up their costs, so I'd count it as a win. The OP doesn't say why the bot is hitting their endpoints, but I doubt the bot is a profit centre for the operator.

lagosfractal42 1y ago
You risk flagging real users as bots, which drives down your profits and reputation
- swiftcoder 1y ago
  In this case I don't think they do - unless the legitimate users are also hitting your site at 700 RPS (in which case, the added load from the bot is going to be negligible)
  - hansvm 1y ago
    Once the bot is stealthy (the current sub-thread if I haven't misread) they absolutely do. A couple examples where I've been flagged as a bot for normal traffic: 1. Discord's telemetry was broken on my browser, and on failure they immediately retried. It didn't take many actions queued up on the site before my browser was initiating over 100RPS, on their behalf. 2. Target and eBay still flag my sessions as bot traffic (presumably because they don't recognize the user agent or because I use Linux or something). Target allows browsing their site for a few items before heavily rate-limiting me for a day or so, and eBay just resets my password a day or two after I log in, every single bloody time. The problem is that from time to time normal users will generate large traffic volumes, and if the bot owner uses many IPs then you're forced to use less reliable signals for that ban hammer (i.e., no single user will be near 700 RPS).

heavyset_go 1y ago

If going stealth means not blatantly DDoS'ing the OP then that's a better outcome than what's currently happening

somat 1y ago

xkcd 810 comes to mind. https://xkcd.com/810/ https://xkcd.com/810/ "what if we make the bots go stealthy and indistinguishable from actual human requests?" "Mission Accomplished"

HPsquared 1y ago
This has pretty much happened now in the internet at large, and it's kinda sad.
- lotsofpulp 1y ago
  “Constructive” and “Helpful” are unfortunately not out weighed by garbage.

lucastech 1y ago

Yeah, there are some botnets I've been seeing that are much more stealthy, using 900-3000 IP's with rotating user agents to send enormous amounts of traffic. I've resorted to blocking entire AS routes to prevent it (fortunately I am mostly hosting US sites with US only residential audiences). I'm not sure who's behind it, but one of the later data centers is oxylabs, so they're probably involved somehow. https://wxp.io/blog/the-bots-that-keep-on-giving https://wxp.io/blog/the-bots-that-keep-on-giving