4 ms·
I've implemented a lot of client handling code and always wondered what the optimal back-off strategy was when I hit a rate limit. It's interesting to read abou
by robertclaus 2y ago
I've implemented a lot of client handling code and always wondered what the optimal back-off strategy was when I hit a rate limit. It's interesting to read about the trade offs from the perspective of the service since that can inform how a client best reacts.
- Maxion 2y agoThere is also the opposite, as someone who once worked on a larger platform when we enabled rate limits, our first implementations caused issues with the service precisely because so many were hitting the API often enough, that when the rate limits were enabled they functionally expired at the same time for all the heavy users, meaning the service received a crapload of requests at the exact time, followed by a period of very low amount of requests, rince repeat.
- MereInterest 2y agoWhat was the solution to these? Naively, I would guess that either the expiration time for each limit would need some jitter to spread out the requests, or the rate limits would need to be tiered (e.g. max 10/second, max 60/minute, max 500/hour) so that the shorter timescales help to smooth out traffic over longer timescales. But I haven’t worked much with network rate limiting, so I’m curious what the actual solutions look like.