Rate limiting, and specifically leaky bucket algorithms that spaces out requests evenly, rather than servicing them as they happens to come in was shown to impr

Rate limiting, and specifically leaky bucket algorithms that spaces out requests evenly, rather than servicing them as they happens to come in was shown to improve the overall performance in a few systems I worked with. Using a leaky bucket algorithm, and a per-customer bucket, I think it's possible to build "fair" systems that also improves the performance. That is, you can run the system with a higher total transactions per seconds, just by queuing "simultaneous" requests a few milliseconds, as they will complete quicker. The reason is probably that it's reducing contention and levels out the resource usage. I thought it would be a feature in almost all web servers, since it's been known "since forever" in the telecom world, but I have not seen it. (Have not looked specifically either, so maybe there are good support for this everywhere and I missed it...)