5 ms·
> Rate limits are far simpler, anyway! Not when the bots are actively programmed to thwart them by using far-flung IP address carousels, request pacing, spoofe
by inejge 2y ago
> Rate limits are far simpler, anyway!
Not when the bots are actively programmed to thwart them by using far-flung IP address carousels, request pacing, spoofed user agents and similar techniques. It's open war these days.
- parineum 2y agoRequest pacing sounds intentionally unabusive.
- rollcat 2y agoIt's called DDoS. DDoS is abusive.
- j16sdiz 2y agoIt is not bringing down your server, but they are taking 80%+ of your bandwidth budget. Does this count as abuse?
- ithkuil 2y agoIsn't that what a rate limiter would address?
- mkl 2y agoNot when the traffic is coming from 10s of thousands of IP addresses, with very few requests from each one: https://drewdevault.com/2025/03/17/2025-03-17-Stop-externalizing-your-costs-on-me.html https://drewdevault.com/2025/03/17/2025-03-17-Stop-externali...
- KronisLV 2y agoThat very much reads like the rant of someone who is sick and tired of the state of things. I’m afraid that it doesn’t change anything in of itself and any sorts of solutions to only allow the users that you’re okay with are what’s direly needed all across the web. Though reading about the people trying to mine crypto on a CI solution, it feels that sometimes it won’t just be LLM scrapers that you need to protect against but any number of malicious people. At that point, you might as well run an invite only community.
- bayindirh 2y agoSource Hut implemented Anubis, and it works so well. I mostly never see the waiting screen. And after it whitelists me for a very long time, so I work without any limitations.
- KronisLV 2y agoThat’s great to hear and Anubis seems cool! I just worry about the idea of running public/free services on the web, due to the potential for misuse and bad actors, though making things paid also seems sensible, e.g. what was linked: https://man.sr.ht/ops/builds.sr.ht-migration.md https://man.sr.ht/ops/builds.sr.ht-migration.md
- ithkuil 2y agook, but my answer was about was how to react to request pacing. If the abuser is using request pacing to make less request then that's making the abuser less abusive. If you're still complaining that request pacing is not pacing the requests down enough because the pacing is designed to just not bring your server down and instead make you consume money, then you can counteract that just by tuning the rate limiting even further down. The 10s of thousands distinct IP address is another (and perfectly valid) issue, but it was not the point I answered to.
- immibis 2y agoAre you at a hoster with extortionately expensive bandwidth, such as AWS, GCP, or Azure?