3 ms·
I've seen _plenty_ of user agents that respond to 429 by immediately trying again. Like, literally immediately; full hammer. I had to eventually automatically b
by Sesse__ 2y ago
I've seen _plenty_ of user agents that respond to 429 by immediately trying again. Like, literally immediately; full hammer. I had to eventually automatically blackhole IP addresses that got 429 too often.
- jcgrillo 2y agoIt seems like it should be pretty cheap to detect violations of Retry-After on a 429 and just automatically blackhole that IP for idk 1hr. It could also be an interesting dataset for exposing the IPs those shady "anonymous scraping" comp intel companies use..
- OutOfHere 2y agoThat is just what a bot does by default. It will almost always give up after a few retries. The point of 429 is that you will not be using up your limited bandwidth sending the actual response, which will save you at least 99% of your bandwidth quota. It is not to find IPs to block, especially if the requestor gives up after a few requests. The IPs that you actually need to block are the ones that are actually DoSing you without stopping even after a few retries, and even then only temporarily.
- Sesse__ 2y ago> That is just what a bot does by default. It will almost always give up after a few retries. I've had them go on for hours. (This is not mainly bots, but various crap desktop applications.) > The IPs that you actually need to block are the ones that are actually DoSing you without stopping even after a few retries, and even then only temporarily. Yup. They only get blocked until next reboot :-)