7 ms·

Gubernator: Cloud-Native Distributed Rate Limiting for Microservices

61 points by medmunds 7y ago

jrockway 7y ago
The most interesting part of this is the k8s integration to find your peers and connect to them. I've wanted this for a variety of projects; I wonder if this is worth extracting into a separate library?
- newaccoutnas 7y ago
  Isn't that just Service Discovery? https://kubernetes.io/docs/concepts/services-networking/service/#discovering-services https://kubernetes.io/docs/concepts/services-networking/serv...
  - jrockway 7y ago
    Getting the list of endpoints is easy. Doing something with that list of endpoints is the hard part.
- alpb 7y ago
  You can do service discovery with a k/v store like etcd, or consul, which is how Kubernetes does service discovery. In fact, Consul very much positions itself as a service discovery tool. I'm not sure how it would be possible to do it in a "library" (without external centralized storage).
Scaevolus 7y ago
Previous pun on Kubernetes->Gubernator : https://github.com/kubernetes/test-infra/tree/master/gubernator https://github.com/kubernetes/test-infra/tree/master/guberna... Etymology of Latin "guberno": From a non-Indo-European Mediterranean language; akin to Ancient Greek κυβερνάω (kubernáō).
- spxtr 7y ago
  Does this name have any significance other than what you said here and what they say in this article?
  - peanutgal2600 7y ago
    In 2003 Republicans threw a bunch of dark money to recall a democratically elected governor, Gray Davis, and replace him with a son of a nazi, Arnold Schwarzaneger, a bad actor, as governor. Really. Republicans, especially out of state Mormons opposed to gay marriage), spent a few hundred million dollars when they saw the chance to invalidate our hippy vote and .. replace our chosen governor with .. An aryan son of a nazi. I'm not creative enough to make this up. Turns out it only takes about $200m to invalidate an election and replace a real lawmaker with Kindergarten Cop. It was the second biggest theft in our political history, second only to that crying alcoholic fratboy Brett Kavanaugh, who was installed by the Republicans refusing to do their job for a year and a half and process Obama's final Supreme Court nominee, Merrick Garland. Governor + Terminator == Gubernator.
    - rat9988 7y ago
      I'm not sure being an aryan or son of a nazi is a crime of any sort.
      - peanutgal2600 7y ago
        It shows intent, and also tries to give more context behind the simultaneously humorous and negative connotation behind the name. The name itself is a reference to an understanding that in the USA money, violence, and corruption required to be in politics. The son of a nazi reference really needs to be there, as it highlights the Nazi element of the Republican party. They didn't just steal the governorship from a bunch of hippies, they replaced a democratically elected governor with "the aryan ideal", accent and racist lineage et al, to prove a point: Californians are second class Americans, white christians only. The Gubernator does not take votes. The Gubernator terminates democracy.
jedberg 7y ago
This looks very cool, but I don't understand why they didn't build their entire caching infrastructure this way. Or to put it another way, why not build all caching like this, and then use your standard cache for rate limits?
haolez 7y ago
Is there something similar, but for outbound requests? The motivation is to be “polite” when accessing third party APIs regarding the rate limit.
- dijit 7y ago
  A leaky bucket?
- Denzel 7y ago
  They call this out as a use-case that's supported by Gubernator: Egress limiting – Blasting external SMTP servers with millions of messages is No Bueno.
- chosenken 7y ago
  I think this could be used for both incoming and outgoing requests. From what I gathered reading the docs and API, this doesn't actually forward your request to the API, it just keeps track of rate's and limits, leaving the actual limiting up to you. Its a microservice that keeps track of limits for you, and is highly distributed. Basically, and I can be totally wrong, what it looks like is you send a rate limit request stating I'm doing 1 hit, I can do 100 hits a minute, am I over limit? Gubernator responds with either a status 0 (no, under limit), how many requests you have left, and the time the rate limit will reset. Or it will return 1 (yes, over limit), the time the limit resets, and you then have to hold your request until the reset time. So you could use this as a self imposed rate limit for external services. I think the confusion is from the announcement, it sounded like this was a proxy type service that would hold your request until the limit was met. From what I read, it looks like you ask for the rate limit status, then its up to you to actually limit your requests.
- chatmasta 7y ago
  You can use haproxy+squid for this purpose. Basically implement the rate limiting on haproxy and send requests to squid “backend” that is a forward proxy to the internet.
  - haolez 7y ago
Exuma 7y ago
Is this something Envoy does? I just learned about it recently and it seemed very cool.
bbmario 7y ago
How does it know that a specific client is over the quota if it is stateless?
- cbetti 7y ago
  It maintains zero configuration state. It is not completely stateless. Consumption of available requests is maintained statefully in RAM. If your Gubernator goes down, the quotas are effectively topped back up when Gubernator returns from the dead, upon next request from that client.
  - bbmario 7y ago
    Oh, guess the documentation confused me. When it said stateless, I automatically assumed no storage whatsoever, not even RAM.

61 points by medmunds 7y ago

The most interesting part of this is the k8s integration to find your peers and connect to them. I've wanted this for a variety of projects; I wonder if this is worth extracting into a separate library?

newaccoutnas 7y ago
Isn't that just Service Discovery? https://kubernetes.io/docs/concepts/services-networking/service/#discovering-services https://kubernetes.io/docs/concepts/services-networking/serv...
- jrockway 7y ago
  Getting the list of endpoints is easy. Doing something with that list of endpoints is the hard part.
alpb 7y ago
You can do service discovery with a k/v store like etcd, or consul, which is how Kubernetes does service discovery. In fact, Consul very much positions itself as a service discovery tool. I'm not sure how it would be possible to do it in a "library" (without external centralized storage).

Scaevolus 7y ago

Previous pun on Kubernetes->Gubernator : https://github.com/kubernetes/test-infra/tree/master/gubernator https://github.com/kubernetes/test-infra/tree/master/guberna... Etymology of Latin "guberno": From a non-Indo-European Mediterranean language; akin to Ancient Greek κυβερνάω (kubernáō).

spxtr 7y ago
Does this name have any significance other than what you said here and what they say in this article?
- peanutgal2600 7y ago
  In 2003 Republicans threw a bunch of dark money to recall a democratically elected governor, Gray Davis, and replace him with a son of a nazi, Arnold Schwarzaneger, a bad actor, as governor. Really. Republicans, especially out of state Mormons opposed to gay marriage), spent a few hundred million dollars when they saw the chance to invalidate our hippy vote and .. replace our chosen governor with .. An aryan son of a nazi. I'm not creative enough to make this up. Turns out it only takes about $200m to invalidate an election and replace a real lawmaker with Kindergarten Cop. It was the second biggest theft in our political history, second only to that crying alcoholic fratboy Brett Kavanaugh, who was installed by the Republicans refusing to do their job for a year and a half and process Obama's final Supreme Court nominee, Merrick Garland. Governor + Terminator == Gubernator.
  - rat9988 7y ago
    I'm not sure being an aryan or son of a nazi is a crime of any sort.
    - peanutgal2600 7y ago
      It shows intent, and also tries to give more context behind the simultaneously humorous and negative connotation behind the name. The name itself is a reference to an understanding that in the USA money, violence, and corruption required to be in politics. The son of a nazi reference really needs to be there, as it highlights the Nazi element of the Republican party. They didn't just steal the governorship from a bunch of hippies, they replaced a democratically elected governor with "the aryan ideal", accent and racist lineage et al, to prove a point: Californians are second class Americans, white christians only. The Gubernator does not take votes. The Gubernator terminates democracy.

jedberg 7y ago

This looks very cool, but I don't understand why they didn't build their entire caching infrastructure this way. Or to put it another way, why not build all caching like this, and then use your standard cache for rate limits?

haolez 7y ago

Is there something similar, but for outbound requests? The motivation is to be “polite” when accessing third party APIs regarding the rate limit.

dijit 7y ago
A leaky bucket?
Denzel 7y ago
They call this out as a use-case that's supported by Gubernator: Egress limiting – Blasting external SMTP servers with millions of messages is No Bueno.
chosenken 7y ago
I think this could be used for both incoming and outgoing requests. From what I gathered reading the docs and API, this doesn't actually forward your request to the API, it just keeps track of rate's and limits, leaving the actual limiting up to you. Its a microservice that keeps track of limits for you, and is highly distributed. Basically, and I can be totally wrong, what it looks like is you send a rate limit request stating I'm doing 1 hit, I can do 100 hits a minute, am I over limit? Gubernator responds with either a status 0 (no, under limit), how many requests you have left, and the time the rate limit will reset. Or it will return 1 (yes, over limit), the time the limit resets, and you then have to hold your request until the reset time. So you could use this as a self imposed rate limit for external services. I think the confusion is from the announcement, it sounded like this was a proxy type service that would hold your request until the limit was met. From what I read, it looks like you ask for the rate limit status, then its up to you to actually limit your requests.
chatmasta 7y ago
You can use haproxy+squid for this purpose. Basically implement the rate limiting on haproxy and send requests to squid “backend” that is a forward proxy to the internet.
- haolez 7y ago

Exuma 7y ago

Is this something Envoy does? I just learned about it recently and it seemed very cool.

bbmario 7y ago

How does it know that a specific client is over the quota if it is stateless?

cbetti 7y ago
It maintains zero configuration state. It is not completely stateless. Consumption of available requests is maintained statefully in RAM. If your Gubernator goes down, the quotas are effectively topped back up when Gubernator returns from the dead, upon next request from that client.
- bbmario 7y ago
  Oh, guess the documentation confused me. When it said stateless, I automatically assumed no storage whatsoever, not even RAM.