6 ms·

This bit jumped out at me: "200 API servers ... to serve 3000 requests per second". That's only 15 RPS per server. Is that normal for Rails?

by rdw 11y ago

spimmy 11y ago
we had to way overprovision to handle even momentary spikes in availability from any backing store. we aimed to run at around 20% unicorn worker utilization under normal conditions.
- railsfailsagaid 11y ago
  Lol - rails performance fails again. Still, all the magic and security holes were worth it, right? And no with Google's go - the language so hipster it don't need no stinking generics, or proper error handling, life will be awesome?
- twelvenmonkeys 11y ago
  But doesn't this sound less about over provisioning and more about optimizing the code you already had?
  - spimmy 11y ago
    optimizing won't help you here, unfortunately. the process-per-request model is fundamentally flawed past a certain scaling point.
andrewvc 11y ago
Rails can serve data far in excess of 15RPS. I've built apps that serve plenty of responses in <15ms. That's one app server on one CPU core. Of course, if you do a lot in a given request it's pretty easy to get that number down as low as you want.
- mereghost 11y ago
  Those numbers kinda surprised me too. We serve millions of requests per day and have some slow responses (75+ ms) but on any given day our servers handle 175 requests per second without breaking a sweat. =/
  - spimmy 11y ago
    Keep in mind that we're a platform, not a website, and we have very little control over things like schemas, slow queries, inefficient looping requests that our developers send us. We can't optimize 500 thousand apps' queries for them. So for example if an app does something bad like performing a full table scan on every request against a 300 million collection, so every request to that backend is timing out at 30 seconds, and there are thousands of them per second, well -- pretty soon your fixed pool will be full of requests timing out to that backend.
    - rdw 11y ago
      This is totally a great explanation, thanks! This sort of thing is why I hate working on platforms (but also why it's so in-demand).
  - spimmy 11y ago
    (we can, and do, do a lot of things to limit the impact any one app can have ... but there are limits. Async is much better for our API model.)
mpdehaan2 11y ago
This should pretty much be able scale with RAM and processor using a preforking web server, you are effectively running many copies of your application and the web server is routing them. Many organizations scale out too soon or for the wrong reasons, and some just have ineffecient database queries and other things that result in bad request times that they could also optimize -- which helps the user regardless of scale with faster page loads.
vlucas 11y ago
Jumped out at me too. I did contract work for 2 years helping the API team that powers the bible app (bible.com), and they peak at over 5,000+ RPS on Sunday morning and never dropped below 2,000 RPS 24/7. Their stack was an old Kohana 2 PHP app run on about 18 physical servers at Softlayer. It boggles my mind that you would need 200 servers for this.

This bit jumped out at me: "200 API servers ... to serve 3000 requests per second". That's only 15 RPS per server. Is that normal for Rails?

by rdw 11y ago

This bit jumped out at me: "200 API servers ... to serve 3000 requests per second". That's only 15 RPS per server. Is that normal for Rails?

spimmy 11y ago

we had to way overprovision to handle even momentary spikes in availability from any backing store. we aimed to run at around 20% unicorn worker utilization under normal conditions.

railsfailsagaid 11y ago
Lol - rails performance fails again. Still, all the magic and security holes were worth it, right? And no with Google's go - the language so hipster it don't need no stinking generics, or proper error handling, life will be awesome?
twelvenmonkeys 11y ago
But doesn't this sound less about over provisioning and more about optimizing the code you already had?
- spimmy 11y ago
  optimizing won't help you here, unfortunately. the process-per-request model is fundamentally flawed past a certain scaling point.

andrewvc 11y ago

Rails can serve data far in excess of 15RPS. I've built apps that serve plenty of responses in <15ms. That's one app server on one CPU core. Of course, if you do a lot in a given request it's pretty easy to get that number down as low as you want.

mereghost 11y ago
Those numbers kinda surprised me too. We serve millions of requests per day and have some slow responses (75+ ms) but on any given day our servers handle 175 requests per second without breaking a sweat. =/
- spimmy 11y ago
  Keep in mind that we're a platform, not a website, and we have very little control over things like schemas, slow queries, inefficient looping requests that our developers send us. We can't optimize 500 thousand apps' queries for them. So for example if an app does something bad like performing a full table scan on every request against a 300 million collection, so every request to that backend is timing out at 30 seconds, and there are thousands of them per second, well -- pretty soon your fixed pool will be full of requests timing out to that backend.
  - rdw 11y ago
    This is totally a great explanation, thanks! This sort of thing is why I hate working on platforms (but also why it's so in-demand).
- spimmy 11y ago
  (we can, and do, do a lot of things to limit the impact any one app can have ... but there are limits. Async is much better for our API model.)

mpdehaan2 11y ago

This should pretty much be able scale with RAM and processor using a preforking web server, you are effectively running many copies of your application and the web server is routing them. Many organizations scale out too soon or for the wrong reasons, and some just have ineffecient database queries and other things that result in bad request times that they could also optimize -- which helps the user regardless of scale with faster page loads.

vlucas 11y ago

Jumped out at me too. I did contract work for 2 years helping the API team that powers the bible app (bible.com), and they peak at over 5,000+ RPS on Sunday morning and never dropped below 2,000 RPS 24/7. Their stack was an old Kohana 2 PHP app run on about 18 physical servers at Softlayer. It boggles my mind that you would need 200 servers for this.