4 ms·

This has been talked to death, but for the general case, for applications that can handle concurrent requests (like Rails with a rack server just a tiny bit sma

by zrail 13y ago

This has been talked to death, but for the general case, for applications that can handle concurrent requests (like Rails with a rack server just a tiny bit smarter than Webrick), random routing works just as well as intelligent routing with far less coupling in the routing later, making it more robust and easier to manage and scale.

sync 13y ago
Rails servers like Unicorn can still only support ~4 concurrent requests on regular dynos (roughly 1 per virtualized core), lessening the problem but not solving it if you have many requests per second.
- jules 13y ago
  Actually that makes a tremendous difference. It is a bit difficult to explain but even 2 concurrent requests per dyno is a lot better than 1. As you go up from 2 to 3 to 4 the difference quickly becomes minimal. I'll try to explain why, but warning: hand waving ahead! The problem with only 1 concurrent request is that if that request takes long to process then it can block other requests that are queued up in that dyno. Lets say that 1% of the requests take long. If a dyno can process only 1 request concurrently then at each request there is a 0.01 chance that that will block other requests. If however it can process 4 concurrent requests, then the dyno will only be blocked if it gets 4 long requests at the same time. So there is only a 0.01^4 = 0.00000001 chance that a dyno will be blocked.
  - adrianpike 13y ago
    Not quite. If you keep routing to a blocked backend, any requests that hit that backend will have unacceptable latencies. Since the routing is essentially random, you'll still be fielding about 1/4 of your requests to that backend, so one long request winds up making 1/4 of all your requests have unacceptable latencies.
    - jules 13y ago
      I don't think that's how it works. Within a single dyno with concurrency of 4 a single request can not block block any requests. It can only block 1 thread, but then the other requests will be processed on the other threads. Routing is only random to dynos, within dynos it is not random.
  - oillio 13y ago
    At the end of the day, there must be some applications that cannot be fixed by increasing concurrency to the limit in a standard dyno. They would not bother implementing XL dynos if this was not the case.
    - jules 13y ago
      It's most beneficial for application that cannot fit more than 1 on a single dyno.
- thinkbohemian 13y ago
  I run http://codetriage.com/ http://codetriage.com/ on Rails 4, Ruby 2.1.0 and have 5 unicorn workers per (regular 1x) dyno. With a little GC tweaking I could likely get 6, but i'm being cautious. With puma hybid server I can get even more concurrency, but don't need it for now.
oillio 13y ago
This is assuming the application developers understand the problem and write the application accordingly. There are workarounds, which require some legwork and forsight. Reducing the pitfalls inherit in deploying a web app is generally what Heroku is selling. It is why they are making the big bucks. It would have been nice if Heroku would have been able to find a solution and implement smarter routing so that this just became a non-issue.

by zrail 13y ago

sync 13y ago

Rails servers like Unicorn can still only support ~4 concurrent requests on regular dynos (roughly 1 per virtualized core), lessening the problem but not solving it if you have many requests per second.

jules 13y ago
Actually that makes a tremendous difference. It is a bit difficult to explain but even 2 concurrent requests per dyno is a lot better than 1. As you go up from 2 to 3 to 4 the difference quickly becomes minimal. I'll try to explain why, but warning: hand waving ahead! The problem with only 1 concurrent request is that if that request takes long to process then it can block other requests that are queued up in that dyno. Lets say that 1% of the requests take long. If a dyno can process only 1 request concurrently then at each request there is a 0.01 chance that that will block other requests. If however it can process 4 concurrent requests, then the dyno will only be blocked if it gets 4 long requests at the same time. So there is only a 0.01^4 = 0.00000001 chance that a dyno will be blocked.
- adrianpike 13y ago
  Not quite. If you keep routing to a blocked backend, any requests that hit that backend will have unacceptable latencies. Since the routing is essentially random, you'll still be fielding about 1/4 of your requests to that backend, so one long request winds up making 1/4 of all your requests have unacceptable latencies.
  - jules 13y ago
    I don't think that's how it works. Within a single dyno with concurrency of 4 a single request can not block block any requests. It can only block 1 thread, but then the other requests will be processed on the other threads. Routing is only random to dynos, within dynos it is not random.
- oillio 13y ago
  At the end of the day, there must be some applications that cannot be fixed by increasing concurrency to the limit in a standard dyno. They would not bother implementing XL dynos if this was not the case.
  - jules 13y ago
    It's most beneficial for application that cannot fit more than 1 on a single dyno.
thinkbohemian 13y ago
I run http://codetriage.com/ http://codetriage.com/ on Rails 4, Ruby 2.1.0 and have 5 unicorn workers per (regular 1x) dyno. With a little GC tweaking I could likely get 6, but i'm being cautious. With puma hybid server I can get even more concurrency, but don't need it for now.

oillio 13y ago

This is assuming the application developers understand the problem and write the application accordingly. There are workarounds, which require some legwork and forsight. Reducing the pitfalls inherit in deploying a web app is generally what Heroku is selling. It is why they are making the big bucks. It would have been nice if Heroku would have been able to find a solution and implement smarter routing so that this just became a non-issue.