5 ms·

I a bit torn here. I love Stripe, but I feel like this is awful advice for API-first companies. If you are building a developer product where your API is your

by rdegges 10y ago

I a bit torn here. I love Stripe, but I feel like this is awful advice for API-first companies. If you are building a developer product where your API is your business, using rate-limiters is akin to preventing customers from giving you money. If your product is an API, you should encourage usage. The more usage you get, the more success both you and your customer will have. It's a win-win. Because of this, I believe implementing rate-limiting strategies result in not only poor-DX for the product, but also a loss of trust (if these limits are in place to prevent your backend from failing, what other things do I need to worry about while using your product?), AND most importantly, they result in loss of business for both the API company and the customer. IMO, if you're an API company, and you can't handle bursts of traffic from your customers, you should work on improving your backend and stop wasting time messing around with implementing patterns like this. It's a lose-lose situation for you and your customers. I was really inspired by this recently after spending some time @ Twilio. Jeff (and the rest of the Twilio team) are hardcore about their API first product / thinking. They have a motto which is something along the lines of this: if you and your customer are both more successful and both make more money when the API usage goes up, do whatever you can to get out of the way and let the API be used as much as possible. I thought that was an awesome approach to take.

teej 10y ago
Rate limiters are fantastic for companies and developers starting to integrate with a new API. There's great comfort in knowing - no matter how badly I screw up I won't end up with a $10k+ bill at the end of the month. The way most smart companies do it is totally sufficient. Low but reasonable entry-level rate limit with a simple support contact to get it lifted. Maybe you're not aware but that's the exact model that Twilio uses[0]. [0] - https://support.twilio.com/hc/en-us/articles/223183648-Sending-and-Receiving-Limitations-on-Calls-and-SMS-Messages https://support.twilio.com/hc/en-us/articles/223183648-Sendi...
- rdegges 10y ago
  I was not aware of this. Thank you. I'm not against allowing the customer to set rate limits, etc. What annoys me in general are services where I want to use them and sometimes do burst traffic for a variety of reasons: but can't for an artificial reason :(
  - pvg 10y ago
    The reason is not artificial, though, that's the whole point.
draw_down 10y ago
I don't think the idea is to reduce API usage overall. And, you don't make more money because someone's buggy script is hammering you with 1000 GET requests a second (for example).
- rdegges 10y ago
  It depends. If your product is your API, and you're getting paid from usage -- then usually yes -- the more API usage you get, the more $ you make, and the more success the customer has (hopefully!)
  - zeisss 10y ago
    but they are not getting paid for an api call, they are getting paid for successful transactions going through their api. And if I can hammer the API with useless unworthy listing calls while big-customer cannot initialize transactions they are losing money.
saosebastiao 10y ago
> IMO, if you're an API company, and you can't handle bursts of traffic from your customers, you should work on improving your backend and stop wasting time messing around with implementing patterns like this. It's a lose-lose situation for you and your customers. I'm surprised this isn't more obvious, especially to a company like Stripe that is built on one of the slowest programming languages/platforms in existence. It isn't hard to come up with a list of languages that would easily provide an order of magnitude more capability per server (yes, maybe compute is not their bottleneck, but I've never seen an RoR setup that was bottlenecked by anything other than RoR). And that not only makes it cheaper for the baseline, but it also makes it cheaper to overprovision for safely handling spikes, and both cheaper and faster to scale when needed. And like you said, not responding to demand is just preventing your customers from giving you money...but it is worse. If you are a growing company and one of your badly needed services is your bottleneck, that service is gonna be the first one to go. API limiting should be considered an attrition risk.
- SEJeff 10y ago
  Why not go out and build a better stripe with a better language and show them how it is really done? Clearly you know more about building web scale payment apis than they do.
  - deleted 10y ago
    [deleted]
  - user5994461 10y ago
    Done! That's called Adyen :D https://www.adyen.com/ https://www.adyen.com/ It's an European company that's 10 times the size and has lower fees. Stripe having a harder time on that side of the Atlantic ;)
jonaf 10y ago
Sorry, but this is naive and bad advice. Rate limiting is a critically missing component in many service APIs, for more reasons than I can even comprehensively enumerate off the top of my head. Scaling an API of any real value is NOT trivial, and struggling to scale an API to meet user demand does NOT necessarily mean that the backend was poorly designed. This is a naive generalization that is hazardous to the industry. Please don't spread it. Here are some reasons why a lack of rate limiting / user auth is practically negligence. There are more, to be sure. I have experience operating a customer facing API for Bazaarvoice, so I think I know what I'm talking about. (We do thousands of requests per second and power reviews for the likes of Walmart, Best Buy, and 4,000 other retailers and brands worldwide.) * Multi-tenancy * * client A over extends and causes client B to be unable to use the API * * client A needs scale independent of clients B-F * monitoring * * suddenly a client is making fewer than usual calls, why? * * suddenly a client is making more than usual calls, why? * billing * * want more requests / second? Upgrade your contract * * it's easier to measure how much I should charge customers per request or type of request, when I can see the rates of those requests and what it costs me * security * * DDOS attack? Start by setting the limit to nothing, or rejecting the requests * * leaking API auth info is less dangerous, if it happens I think some other sibling comments mentioned other great reasons. The takeaway is that a valuable API will most likely be difficult, expensive or both difficult and expensive to scale, and rate limiting is extremely important.
- kpil 10y ago
  Rate limiting, and specifically leaky bucket algorithms that spaces out requests evenly, rather than servicing them as they happens to come in was shown to improve the overall performance in a few systems I worked with. Using a leaky bucket algorithm, and a per-customer bucket, I think it's possible to build "fair" systems that also improves the performance. That is, you can run the system with a higher total transactions per seconds, just by queuing "simultaneous" requests a few milliseconds, as they will complete quicker. The reason is probably that it's reducing contention and levels out the resource usage. I thought it would be a feature in almost all web servers, since it's been known "since forever" in the telecom world, but I have not seen it. (Have not looked specifically either, so maybe there are good support for this everywhere and I missed it...)
plandis 10y ago
I think you might be missing the assumptions here. No service I've ever seen is 100% available. If you assume that things can (and will!) fail then you need to design for failure. Load shedding helps mitigate failures of insufficient capacity. When you need to prioritize some things over others load shedding helps here.
dlubarov 10y ago
Rate limiting is a way of failing gracefully. Ideally your request handlers never fail, but if they have to, having a rate limiter filter out some requests (based on rules you control) is usually better than the default failure behavior of your server framework.
detaro 10y ago
> IMO, if you're an API company, and you can't handle bursts of traffic from your customers, you should work on improving your backend and stop wasting time messing around with implementing patterns like this. It's a lose-lose situation for you and your customers. Counter-argument: That's not a reason not have these things, that's why you should think about implementing patterns like this before a unforeseen situation happens where they are useful. So they can limit the pain while you fix whatever causes the current load problem. I agree when it comes to strict limits like "a single customer can only do X requests per hour", but that's not all rate limiting is good for.
quacker 10y ago
IMO, if you're an API company, and you can't handle bursts of traffic from your customers, you should work on improving your backend and stop wasting time messing around with implementing patterns like this. It's a lose-lose situation for you and your customers. Improving performance can be significantly more expensive and time-consuming than implementing simple rate limits. A realistic scenario is something along the lines of: - 99% of time your request rate is N requests/min or lower - 1% of time your request rate exceeds N requests/min, which could cause service degradation You can deploy infrastructure to handle the 99% case, slap rate limits in front of the service, and sleep well at night. It's often not worth it to pay for additional infrastructure, spend time optimizing, etc for that 1% case. As an API user, the way to think about rate limits is as a form of protection from other (misbehaving) users. Everyone is going to be upset when one user's script has a tight-loop firing at 100000 requests/sec and then the API becomes lethargic, throws errors, or goes down all together.
- user5994461 10y ago
  Request rate are accounted in requests/second. If you are counting requests/minute, you're seriously averaging your peak load and you're about to epicly fail in production.
  - quacker 10y ago
    You're being downvoted but there is some truth to this. For example, I inherited a relatively poorly performing service with a per-user limit set to 60 req/min. As the service operator, I have the choice of setting either a rate limit of 60 req/min or 1 req/sec. The former (60/min) leaves you open to per-user spikes of up to 60 req/sec. This is a real risk: 10 users together _could_ produce a spike of 600 req/sec. Still, we went with the per-minute rate limit. Why? Those multi-user bursts are fairly improbable (most of our users were well-behaving) and queueing let us manage these isolated spikes gracefully. The per-minute rate limit is a bit more flexible from a user perspective (reward the good users) and it still combats the problem of sustained heavy load on the service, which is the true danger (stop/limit the "bad" users).
ericjang 10y ago
I disagree with the parent comment, but why is everyone downvoting this? It was a politely presented opinion, and the following discussion is entirely civil.

by rdegges 10y ago

teej 10y ago

Rate limiters are fantastic for companies and developers starting to integrate with a new API. There's great comfort in knowing - no matter how badly I screw up I won't end up with a $10k+ bill at the end of the month. The way most smart companies do it is totally sufficient. Low but reasonable entry-level rate limit with a simple support contact to get it lifted. Maybe you're not aware but that's the exact model that Twilio uses[0]. [0] - https://support.twilio.com/hc/en-us/articles/223183648-Sending-and-Receiving-Limitations-on-Calls-and-SMS-Messages https://support.twilio.com/hc/en-us/articles/223183648-Sendi...

rdegges 10y ago
I was not aware of this. Thank you. I'm not against allowing the customer to set rate limits, etc. What annoys me in general are services where I want to use them and sometimes do burst traffic for a variety of reasons: but can't for an artificial reason :(
- pvg 10y ago
  The reason is not artificial, though, that's the whole point.

draw_down 10y ago

I don't think the idea is to reduce API usage overall. And, you don't make more money because someone's buggy script is hammering you with 1000 GET requests a second (for example).

rdegges 10y ago
It depends. If your product is your API, and you're getting paid from usage -- then usually yes -- the more API usage you get, the more $ you make, and the more success the customer has (hopefully!)
- zeisss 10y ago
  but they are not getting paid for an api call, they are getting paid for successful transactions going through their api. And if I can hammer the API with useless unworthy listing calls while big-customer cannot initialize transactions they are losing money.

saosebastiao 10y ago

> IMO, if you're an API company, and you can't handle bursts of traffic from your customers, you should work on improving your backend and stop wasting time messing around with implementing patterns like this. It's a lose-lose situation for you and your customers. I'm surprised this isn't more obvious, especially to a company like Stripe that is built on one of the slowest programming languages/platforms in existence. It isn't hard to come up with a list of languages that would easily provide an order of magnitude more capability per server (yes, maybe compute is not their bottleneck, but I've never seen an RoR setup that was bottlenecked by anything other than RoR). And that not only makes it cheaper for the baseline, but it also makes it cheaper to overprovision for safely handling spikes, and both cheaper and faster to scale when needed. And like you said, not responding to demand is just preventing your customers from giving you money...but it is worse. If you are a growing company and one of your badly needed services is your bottleneck, that service is gonna be the first one to go. API limiting should be considered an attrition risk.

SEJeff 10y ago
Why not go out and build a better stripe with a better language and show them how it is really done? Clearly you know more about building web scale payment apis than they do.
- deleted 10y ago
  [deleted]
- user5994461 10y ago
  Done! That's called Adyen :D https://www.adyen.com/ https://www.adyen.com/ It's an European company that's 10 times the size and has lower fees. Stripe having a harder time on that side of the Atlantic ;)

jonaf 10y ago

Sorry, but this is naive and bad advice. Rate limiting is a critically missing component in many service APIs, for more reasons than I can even comprehensively enumerate off the top of my head. Scaling an API of any real value is NOT trivial, and struggling to scale an API to meet user demand does NOT necessarily mean that the backend was poorly designed. This is a naive generalization that is hazardous to the industry. Please don't spread it. Here are some reasons why a lack of rate limiting / user auth is practically negligence. There are more, to be sure. I have experience operating a customer facing API for Bazaarvoice, so I think I know what I'm talking about. (We do thousands of requests per second and power reviews for the likes of Walmart, Best Buy, and 4,000 other retailers and brands worldwide.) * Multi-tenancy * * client A over extends and causes client B to be unable to use the API * * client A needs scale independent of clients B-F * monitoring * * suddenly a client is making fewer than usual calls, why? * * suddenly a client is making more than usual calls, why? * billing * * want more requests / second? Upgrade your contract * * it's easier to measure how much I should charge customers per request or type of request, when I can see the rates of those requests and what it costs me * security * * DDOS attack? Start by setting the limit to nothing, or rejecting the requests * * leaking API auth info is less dangerous, if it happens I think some other sibling comments mentioned other great reasons. The takeaway is that a valuable API will most likely be difficult, expensive or both difficult and expensive to scale, and rate limiting is extremely important.

kpil 10y ago
Rate limiting, and specifically leaky bucket algorithms that spaces out requests evenly, rather than servicing them as they happens to come in was shown to improve the overall performance in a few systems I worked with. Using a leaky bucket algorithm, and a per-customer bucket, I think it's possible to build "fair" systems that also improves the performance. That is, you can run the system with a higher total transactions per seconds, just by queuing "simultaneous" requests a few milliseconds, as they will complete quicker. The reason is probably that it's reducing contention and levels out the resource usage. I thought it would be a feature in almost all web servers, since it's been known "since forever" in the telecom world, but I have not seen it. (Have not looked specifically either, so maybe there are good support for this everywhere and I missed it...)

plandis 10y ago

I think you might be missing the assumptions here. No service I've ever seen is 100% available. If you assume that things can (and will!) fail then you need to design for failure. Load shedding helps mitigate failures of insufficient capacity. When you need to prioritize some things over others load shedding helps here.

dlubarov 10y ago

Rate limiting is a way of failing gracefully. Ideally your request handlers never fail, but if they have to, having a rate limiter filter out some requests (based on rules you control) is usually better than the default failure behavior of your server framework.

detaro 10y ago

quacker 10y ago

IMO, if you're an API company, and you can't handle bursts of traffic from your customers, you should work on improving your backend and stop wasting time messing around with implementing patterns like this. It's a lose-lose situation for you and your customers. Improving performance can be significantly more expensive and time-consuming than implementing simple rate limits. A realistic scenario is something along the lines of: - 99% of time your request rate is N requests/min or lower - 1% of time your request rate exceeds N requests/min, which could cause service degradation You can deploy infrastructure to handle the 99% case, slap rate limits in front of the service, and sleep well at night. It's often not worth it to pay for additional infrastructure, spend time optimizing, etc for that 1% case. As an API user, the way to think about rate limits is as a form of protection from other (misbehaving) users. Everyone is going to be upset when one user's script has a tight-loop firing at 100000 requests/sec and then the API becomes lethargic, throws errors, or goes down all together.

user5994461 10y ago
Request rate are accounted in requests/second. If you are counting requests/minute, you're seriously averaging your peak load and you're about to epicly fail in production.
- quacker 10y ago
  You're being downvoted but there is some truth to this. For example, I inherited a relatively poorly performing service with a per-user limit set to 60 req/min. As the service operator, I have the choice of setting either a rate limit of 60 req/min or 1 req/sec. The former (60/min) leaves you open to per-user spikes of up to 60 req/sec. This is a real risk: 10 users together _could_ produce a spike of 600 req/sec. Still, we went with the per-minute rate limit. Why? Those multi-user bursts are fairly improbable (most of our users were well-behaving) and queueing let us manage these isolated spikes gracefully. The per-minute rate limit is a bit more flexible from a user perspective (reward the good users) and it still combats the problem of sustained heavy load on the service, which is the true danger (stop/limit the "bad" users).

ericjang 10y ago

I disagree with the parent comment, but why is everyone downvoting this? It was a politely presented opinion, and the following discussion is entirely civil.