4 ms·
I'm inclined to wait until Heroku weighs in to render judgement. Specifically, because their argument depends on this premise: > But elsewhere in their current
by nthj 14y ago
I'm inclined to wait until Heroku weighs in to render judgement. Specifically, because their argument depends on this premise:
> But elsewhere in their current docs, they make the same old statement loud and clear:
> The heroku.com stack only supports single threaded requests. Even if your applicaExplaintion were to fork and support handling multiple requests at once, the routing mesh will never serve more than a single request to a dyno at a time.
They pull this from Heroku's documentation on the Bamboo stack [1], but then extrapolate and say it also applies to Heroku's Cedar stack.
However, I don't believe this to be true. Recently, I wrote a brief tutorial on implementing Google Apps' openID into your Rails app.
The underlying problem with doing so on a free (single-dyno) Heroku app is that while your app makes an authentication request to Google, Google turns around and makes a "oh hey" request to your app. With a single-concurrency system, Google your app times out waiting for Google to get back to you and Google won't get back to you until your app gets back to you so hey deadlock.
However, there is a work-around on the Cedar stack: configure the unicorn server to supply 4 or so worker processes for your web server, and the Heroku routing mesh appropriately routes multiple concurrent requests to Unicorn/my app. This immediately fixed my deadlock problem. I have code and more details in a blog post I wrote recently. [2]
This seems to be confirmed by Heroku's documentation on dynos [3]:
> Multi-threaded or event-driven environments like Java, Unicorn, and Node.js can handle many concurrent requests. Load testing these applications is the only realistic way to determine request throughput.
I might be missing something really obvious here, but to summarize: their premise is that Heroku only supports single-threaded requests, which is true on the legacy Bamboo stack but I don't believe to be true on Cedar, which they consider their "canonical" stack and where I have been hosting Rails apps for quite a while.
[1] https://devcenter.heroku.com/articles/http-routing-bamboo https://devcenter.heroku.com/articles/http-routing-bamboo
[2] http://www.thirdprestige.com/posts/your-website-and-email-accounts-should-be-friends-part-ii http://www.thirdprestige.com/posts/your-website-and-email-ac...
[3] https://devcenter.heroku.com/articles/dynos#dynos-and-requests https://devcenter.heroku.com/articles/dynos#dynos-and-reques...
[edit: formatting]
- jarcoal 14y agoThis is how I run my apps as well, and they seem to handle more than one request concurrently per dyno, but I'm not smart enough to dispute this post, so I'm just sitting back and watching.
- seivan 14y agoHow I have been doing it for the last year. Puma define 4:8 threads or Unicorn 3 workers.
- kennystone 14y agoIf you have 2 unicorn servers and you happen to get 3 slow requests routed to it, you are still screwed, right? Seems to me like it will still queue on that dyno.
- michaelrkn 14y agoThat's exactly what happened to us - switching to unicorn bought us a little time and a bit of performance, but we hit the exact same problems again after a couple more weeks of growth.
- ibdknox 14y agoYeah, the only real question is whether or not it's true that they no longer do intelligent routing. If that is the case, then regardless of anything else the problem exists once you pass a certain scale/request cost. It won't matter if that one dyno can handle hundreds of requests at once, it will still queue stupidly.
- barmstrong 14y agoThis is true - unicorn masks the symptoms for a period of time but does not solve the underlying problem in the way a global request queue would. Also, if the unicorn process is doing something cpu intensive (vs waiting on a 3rd party service or io etc) then it won't serve 3 requests simultaneously as fast as single processes would.
- rjacoby5 14y agoOne of the hidden costs of Unicorn is spin-up time. Unicorn takes a long time to start, then fork. We would get a ton of request timeouts during this period. Switching back to Thin, we never got timeouts during deploys - even under very heavy load.
- richcollins 14y agohow does it decide to queue at the dyno level anyway? Does it check for connection refusal at the TCP level?
- dedsm 14y agoNevertheless, random routing is a bad idea even if a dyno can handle multiple requests simultaneously
- wwarnerandrew 14y agoYes, it's true that the Cedar stack supports forking web servers like unicorn, and that an individual dyno can run multiple workers and therefore serve multiple requests at the same time. However, dumb routing is still very problematic – even if your dyno can work on two requests simultaneously it's still bad for it to get sent a third request when there are other open dynos. Also, for apps with a large-ish memory footprint, you can't run very many workers. A heroku dyno has 512mb memory, so if your app has a 250mb footprint, then you can basically only have two workers. Another essential point to note is that the routing between cedar and bamboo is essentially unchanged. They simply changed the type of apps you can run.
- richcollins 14y agoRight if the apps internal queue is full and it stops accepting connections I'm assuming it will still queue at the dyno level anyway.
- spoiler 14y agoSlightly off topic, but what are everyone's. experience and thoughts about Puma[1]? I am using it on a small production environment with Heroku and I like it, but when we officially launch the app, should we switch to Unicorn? [1] http://puma.io/ http://puma.io/
- tcc619 14y agoI have been using it on a few small projects. Haven't ran into an issue yet and the setup has been really easy.
- ylansegal 14y agoI did some performance analysis on puma vs unicorn vs thin a while ago: http://ylan.segal-family.com/blog/2012/08/20/better-performance-on-heroku-thins-vs-unicorn-vs-puma/ http://ylan.segal-family.com/blog/2012/08/20/better-performa... Although as noted in the comments, I neglected to run threadsafe! and should have probably tried rubinius or jruby. I have been meaning to redo. Take with a grain of salt
- NatW 14y agoYou seem have written one of the very few articles I've seen benchmarking this. I'd love to know more about how Puma compares to Unicorn (especially unicorn configurations mentioned by some in this conversation) and Thin for serving rails on Heroku. Many of the Unicorners pushing their solution don't appear to be aware of Puma and its potential benefits. I'm curious if Puma with MRI has benefits, too. Thanks!
- miloshadzic 14y agoAFAIK Puma should have a lower memory footprint on MRI than Thin but I haven't done benchmarks myself.
- steveklabnik 14y agoAlso note that Rails 4 will have threadsafe! on by default, too, so if you didn't have it on here, that'll make it different for those apps.
- scottshea 14y agoTo amplify, I do a ton of queue adjustment work with Unicorn at the backlog level. It is so frequent that we set up Unicorn startup to read from an ENV variable on Heroku that we set as needed. With two Unicorn workers we found that 25 was the best backlog threshold to accept (it refuses additional requests). When we were able to go to 5 Unicorn workers on Heroku we had to start to adjust that.
- tibbon 14y agoYou don't happen to have any documentation for how to do that do you? Very curious. Never seen anything about setting up Unicorn like this prior (I'm just using 3 or 4 Unicorns/dyno currently)
- scottshea 14y agoWe did this in unicorn.rb `:backlog => Integer(ENV['UNICORN_BACKLOG'] || 200)` and then set the UNICORN_BACKLOG variable by the Heroku command line `heroku config:set UNICORN_BACKLOG=25 -a <app_name>`. We have been as high as 1024 and as low as 10. We settled in at 25 for us.
- jwrubel 14y agoHere's a gist of our unicorn.rb config (https://gist.github.com/apangeajwrubel/4953849 https://gist.github.com/apangeajwrubel/4953849) Using env variables lets us adjust parameters without a code push (still requires a restart). We saw dramatic reduction in 'long tail' requests when we dropped the backlog to 25. We're experimenting now (thanks to @scottshea) with even lower values. At some point the routing mesh will give up retrying and throw an H21 (https://devcenter.heroku.com/articles/error-codes#h21-backend-connection-refused https://devcenter.heroku.com/articles/error-codes#h21-backen...). One data point that would be useful from heroku is how many retries we get.
- pointful 14y agoTo expand on this: You have to remove the port declaration from the line for Unicorn in your Procfile, and then add a line like this to your unicorn.rb file to define the listener port along with adjusting the backlog size: listen ENV['PORT'], :backlog => Integer(ENV['UNICORN_BACKLOG'] || 100)