3 ms·

So exactly which webserver is Heroku recommending atm? Your article says Unicorn and the Heroku devcenter docs says Puma

jordanthoms 11y ago
Puma's currently the recommended one, but I think that's actually a bad recommendation. Puma is great if you have a Ruby implementation without a GIL (e.g. JRuby, which it was built for), but it's not so good on MRI, because you have to use clustered mode, which runs multiple processes. What they don't tell you is that when running clustered mode, the requests are randomly distributed to each process, and then the requests queued at that level. Each process can run multiple threads each processing a request simultaneously, but because of the GIL anything that's computationally heavy will cause the other requests in that process to run slowly. That means that it's possible for one process to become backed up with requests, while the other processes on the dyno are sitting idle. And it's actually not just possible, but likely when under reasonable load. So there is actually no deployment configuration of Puma which makes sense under MRI, at least without a reverse proxy on the server handling queuing: - Single process (no clusters), 1 thread: Bad because you use only one core, but consistent performance assuming you're not on a multitenant 1X or 2X dyno. - Clustered, 1 thread per process: Bad because requests are queued at the process level, so a slow request will cause any others sent to that process to have to wait. If you have a request which takes 25s to execute, the one sitting behind it in the queue is highly likely to timeout, and this can cascade to the next request, etc. - Clustered, multiple threads per process: Better, but still bad, because while multiple requests will be running at the same time in the process, they are competing for the GIL and can slow each other down so much you'll still get timeouts. The best solution we found to all this was to run a buildpack with a nginx reverse-proxy to a Unicorn server. It's nice because each request is run in it's own process, and the requests are queued at the dyno-level by nginx until any of the processes is free. Combine that with Performance dynos, so you can run a small number of dynos with lots of workers each, where each dyno has consistent performance, and the queuing issues mostly go away. As you may be able to tell, I've spent far more time debugging this behavior than I would have liked to!
- schneems 11y ago
  > Puma is great if you have a Ruby implementation without a GIL Puma is also great if you have blocking IO like database queries or file uploads which most ruby/rails apps do. The GIL is released when Ruby does IO so it can context switch and run other threads. > there is actually no deployment configuration of Puma which makes sense under MRI I disagree, puma makes lots of sense on MRI but it is app specific. By this same argument Sidekiq would be worthless on MRI but it's the dominant work queueing library even though it is solely threaded. In my experience Puma with MRI using "hybrid" with clusters and threads is faster than Unicorn (or rather you can get more throughput and less queueing). However it's app specific. If your app is mostly doing number crunching in Ruby code and not touching IO, then the GIL never gets released and you don't gain much by using threads. Again most Ruby apps use a lot of IO. > anything that's computationally heavy will cause the other requests in that process to run slowly. Exactly. One of the main reasons I pulled recommendation for Unicorn was it's lack of slow client handling, it doesn't help to have a fast webserver, if all your workers are...waiting...on...bytes...and...can't...do...anything...else. Nginx handles slow clients for you so nginx + unicorn is good, it's a harder deployment configuration and it's easier to tell our customers to configure one thing instead of 2, especially if that 1 thing yields better results for the majority of use cases. I'm glad you got things working with nginx + unicorn. I'm a big fan of loading up PX dynos with lots of workers :) I hope some of the above helps clear up a few points for future readers. I always encourage people to try different options and take our recommendations with a grain of salt. It's also why we've kept our Unicorn article so that anyone not happy with puma performance can try another webserver. Thanks for sharing!
  - jordanthoms 11y ago
    Thanks - I wrote this mostly as a braindump of some of the things I discovered in investigating this stuff - We had a period of several months where we were constantly getting timeouts. There's not really a huge amount of really detailed information out there on how the different servers work and the tradeoffs between them. RE threads on MRI, I think it comes down to what your latency requirements are and the type of work as you said. I'm happy with sidekiq on MRI because it doesn't matter hugely if the process gets bogged down for 60 seconds while a ruby-heavy job runs and the other jobs run slowly - jobs are only pulled from redis when there is a spare thread, and sidekiq jobs won't timeout like web requests do. IO-heavy work will get a boost with threads on Puma, but if you get a single Ruby-heavy request running for 20 seconds serializing something, you are going to get timeouts happen. I guess it does depend on the type of app you are running - Ours is actually quite Ruby-crunching heavy, because it has to serialize some big json responses, which is still incredibly slow in Rails. Apps which don't have occasional Ruby-intensive requests wouldn't have as much trouble with all this :)

jordanthoms 11y ago

Puma's currently the recommended one, but I think that's actually a bad recommendation. Puma is great if you have a Ruby implementation without a GIL (e.g. JRuby, which it was built for), but it's not so good on MRI, because you have to use clustered mode, which runs multiple processes. What they don't tell you is that when running clustered mode, the requests are randomly distributed to each process, and then the requests queued at that level. Each process can run multiple threads each processing a request simultaneously, but because of the GIL anything that's computationally heavy will cause the other requests in that process to run slowly. That means that it's possible for one process to become backed up with requests, while the other processes on the dyno are sitting idle. And it's actually not just possible, but likely when under reasonable load. So there is actually no deployment configuration of Puma which makes sense under MRI, at least without a reverse proxy on the server handling queuing: - Single process (no clusters), 1 thread: Bad because you use only one core, but consistent performance assuming you're not on a multitenant 1X or 2X dyno. - Clustered, 1 thread per process: Bad because requests are queued at the process level, so a slow request will cause any others sent to that process to have to wait. If you have a request which takes 25s to execute, the one sitting behind it in the queue is highly likely to timeout, and this can cascade to the next request, etc. - Clustered, multiple threads per process: Better, but still bad, because while multiple requests will be running at the same time in the process, they are competing for the GIL and can slow each other down so much you'll still get timeouts. The best solution we found to all this was to run a buildpack with a nginx reverse-proxy to a Unicorn server. It's nice because each request is run in it's own process, and the requests are queued at the dyno-level by nginx until any of the processes is free. Combine that with Performance dynos, so you can run a small number of dynos with lots of workers each, where each dyno has consistent performance, and the queuing issues mostly go away. As you may be able to tell, I've spent far more time debugging this behavior than I would have liked to!

schneems 11y ago
> Puma is great if you have a Ruby implementation without a GIL Puma is also great if you have blocking IO like database queries or file uploads which most ruby/rails apps do. The GIL is released when Ruby does IO so it can context switch and run other threads. > there is actually no deployment configuration of Puma which makes sense under MRI I disagree, puma makes lots of sense on MRI but it is app specific. By this same argument Sidekiq would be worthless on MRI but it's the dominant work queueing library even though it is solely threaded. In my experience Puma with MRI using "hybrid" with clusters and threads is faster than Unicorn (or rather you can get more throughput and less queueing). However it's app specific. If your app is mostly doing number crunching in Ruby code and not touching IO, then the GIL never gets released and you don't gain much by using threads. Again most Ruby apps use a lot of IO. > anything that's computationally heavy will cause the other requests in that process to run slowly. Exactly. One of the main reasons I pulled recommendation for Unicorn was it's lack of slow client handling, it doesn't help to have a fast webserver, if all your workers are...waiting...on...bytes...and...can't...do...anything...else. Nginx handles slow clients for you so nginx + unicorn is good, it's a harder deployment configuration and it's easier to tell our customers to configure one thing instead of 2, especially if that 1 thing yields better results for the majority of use cases. I'm glad you got things working with nginx + unicorn. I'm a big fan of loading up PX dynos with lots of workers :) I hope some of the above helps clear up a few points for future readers. I always encourage people to try different options and take our recommendations with a grain of salt. It's also why we've kept our Unicorn article so that anyone not happy with puma performance can try another webserver. Thanks for sharing!
- jordanthoms 11y ago
  Thanks - I wrote this mostly as a braindump of some of the things I discovered in investigating this stuff - We had a period of several months where we were constantly getting timeouts. There's not really a huge amount of really detailed information out there on how the different servers work and the tradeoffs between them. RE threads on MRI, I think it comes down to what your latency requirements are and the type of work as you said. I'm happy with sidekiq on MRI because it doesn't matter hugely if the process gets bogged down for 60 seconds while a ruby-heavy job runs and the other jobs run slowly - jobs are only pulled from redis when there is a spare thread, and sidekiq jobs won't timeout like web requests do. IO-heavy work will get a boost with threads on Puma, but if you get a single Ruby-heavy request running for 20 seconds serializing something, you are going to get timeouts happen. I guess it does depend on the type of app you are running - Ours is actually quite Ruby-crunching heavy, because it has to serialize some big json responses, which is still incredibly slow in Rails. Apps which don't have occasional Ruby-intensive requests wouldn't have as much trouble with all this :)