5 ms·

For a different view on the topic watch "Stop Rate Limiting! Capacity Management Done Right" https://www.youtube.com/watch?v=m64SWl9bfvk https://www.youtube.com

For a different view on the topic watch "Stop Rate Limiting! Capacity Management Done Right" https://www.youtube.com/watch?v=m64SWl9bfvk https://www.youtube.com/watch?v=m64SWl9bfvk The basic premise is not do do req/s limiting but rather concurrency limiting which results in req/s limiting by itself. Concurrency limiting is rather simple and doesn't require a lot of code complexity.

daddykotex 9y ago
This video was very interesting, thanks for sharing!
snoman 9y ago
Wow! That was an incredible talk/demonstration. Thanks for that!
zie 9y ago
The problem here is, it depends on WHY you are rate limiting. If it's just to balance so you don't overload your backends, then absolutely this is a great way to do it. If however you are trying to limit client(s) because the service is an authentication gateway for instance, then you want to limit user/pass requests to X number then concurrency limiting isn't a good way to do that. So you may need both, depending on your use-cases, so it's not a one-size fits all solution.
- bogomipz 9y ago
  What is the use case for your second example? I am not following. Wouldn't that just be rate limiting by client IP though?
  - zlynx 9y ago
    I heard about a case where someone built a phone app backend without rate limiting. Someone found this hole and successfully attacked 70,000 accounts by running password dictionary cracks against the authentication API. Modern distributed systems are simply too fast and users are too dumb with picking passwords to allow unlimited password attempts.
  - zie 9y ago
    Well you can rate limit via a lot of things, definitely by IP is a good idea, but for ipv6, you typically want to limit by blocks(since every ipv6 user currently usually gets a full block of IP's), but you also probably want to also limit by username, i.e. if you keep trying to login as user 'root', you only get 3 attempts/minute or something. After X attempts via the same IP, you can block/ban that IP via a FW rule/etc for say 5 mins. or 30m, or whatever. It all depends on your security posture, you could go crazy and actually lock the account after 3 failures, but then you hand bad people a free DDOS... so you have to be careful about doing that.. But you could soft-lock and require a correct password and an email address verification after X failed attempts(i.e. correct login, plus they have to click a link in an email). Anyways, see what I'm saying here? Authentication access is something you really want to get right, and it's a complicated topic.
  - dboreham 9y ago
    >Wouldn't that just be rate limiting by client IP though? This won't defeat an adversary with a botnet.
    - heavenlyblue 9y ago
      But this is not something you deal with a rate-limiter. By your logic, then a sufficiently large botnet would dwarf any live traffic whatsoever, implying you’d probably want to filter them on network level instead.
thatusernametho 9y ago
He mentioned that nginx using lua managed the requests but I didn't see the code for that. Is that available anywhere?
pwdisswordfish2 9y ago
On the origin is he only measuring connections? HTTP/1.1 says a single connection can contain more than one request. The client I use has this feature. "Modern" browsers do not. Whether proxies pipeline requests to origin servers (irrespective of the client features) I do not know. But it seems like they could preserve some origin capacity that way. Perhaps they adhere to one request per connection. Ideally, as a user, I think it would be useful to get more diagnostics in response headers on the health of the proxies and origins. Something more informative than HTTP status codes. That way we can build "intelligent clients" that self-adjust to current conditions in response to diagnostics they get in the responses. (The presenter references the adaptive approach of the TCP congestion algorithm.) Some websites return diagnostics in webpage content. But of course one only gets them after a successful (cf. failed) transaction. For example, I believe wikipedia.org or archive.vector.co.uk return some diagnostics on some aspect of their setup (probably not intended to be utilised by clients).

daddykotex 9y ago

This video was very interesting, thanks for sharing!

snoman 9y ago

Wow! That was an incredible talk/demonstration. Thanks for that!

zie 9y ago

The problem here is, it depends on WHY you are rate limiting. If it's just to balance so you don't overload your backends, then absolutely this is a great way to do it. If however you are trying to limit client(s) because the service is an authentication gateway for instance, then you want to limit user/pass requests to X number then concurrency limiting isn't a good way to do that. So you may need both, depending on your use-cases, so it's not a one-size fits all solution.

bogomipz 9y ago
What is the use case for your second example? I am not following. Wouldn't that just be rate limiting by client IP though?
- zlynx 9y ago
  I heard about a case where someone built a phone app backend without rate limiting. Someone found this hole and successfully attacked 70,000 accounts by running password dictionary cracks against the authentication API. Modern distributed systems are simply too fast and users are too dumb with picking passwords to allow unlimited password attempts.
- zie 9y ago
  Well you can rate limit via a lot of things, definitely by IP is a good idea, but for ipv6, you typically want to limit by blocks(since every ipv6 user currently usually gets a full block of IP's), but you also probably want to also limit by username, i.e. if you keep trying to login as user 'root', you only get 3 attempts/minute or something. After X attempts via the same IP, you can block/ban that IP via a FW rule/etc for say 5 mins. or 30m, or whatever. It all depends on your security posture, you could go crazy and actually lock the account after 3 failures, but then you hand bad people a free DDOS... so you have to be careful about doing that.. But you could soft-lock and require a correct password and an email address verification after X failed attempts(i.e. correct login, plus they have to click a link in an email). Anyways, see what I'm saying here? Authentication access is something you really want to get right, and it's a complicated topic.
- dboreham 9y ago
  >Wouldn't that just be rate limiting by client IP though? This won't defeat an adversary with a botnet.
  - heavenlyblue 9y ago
    But this is not something you deal with a rate-limiter. By your logic, then a sufficiently large botnet would dwarf any live traffic whatsoever, implying you’d probably want to filter them on network level instead.

thatusernametho 9y ago

He mentioned that nginx using lua managed the requests but I didn't see the code for that. Is that available anywhere?

pwdisswordfish2 9y ago

On the origin is he only measuring connections? HTTP/1.1 says a single connection can contain more than one request. The client I use has this feature. "Modern" browsers do not. Whether proxies pipeline requests to origin servers (irrespective of the client features) I do not know. But it seems like they could preserve some origin capacity that way. Perhaps they adhere to one request per connection. Ideally, as a user, I think it would be useful to get more diagnostics in response headers on the health of the proxies and origins. Something more informative than HTTP status codes. That way we can build "intelligent clients" that self-adjust to current conditions in response to diagnostics they get in the responses. (The presenter references the adaptive approach of the TCP congestion algorithm.) Some websites return diagnostics in webpage content. But of course one only gets them after a successful (cf. failed) transaction. For example, I believe wikipedia.org or archive.vector.co.uk return some diagnostics on some aspect of their setup (probably not intended to be utilised by clients).