Just adding a top-level post to point out something buried in one of the threads here that is an important point on what is happening here: The "queue at the d

Just adding a top-level post to point out something buried in one of the threads here that is an important point on what is happening here: The "queue at the dyno level" is coming from the Rails stack -- it's not something that Heroku is doing to/for the dynos. Thin and Unicorn (and others, I imagine) will queue requests as socket connections on their listener. Both default to 1024 backlog requests. If you lower that number, Heroku will (according to the implications in the documentation on H21 errors) try multiple other dynos first before giving up. See https://devcenter.heroku.com/articles/error-codes#h21-backend-connection-refused https://devcenter.heroku.com/articles/error-codes#h21-backen... For a single-threaded process to be willing to backlog a thousand requests is problematic when combined with random load balancing. Dropping this number down significantly will lead to more sane load-balancing behavior by the overall stack, as long as there are other dynos available to take up the slack. Also, the time the request spends on the dyno, including the time in the dyno's own backlog, is available in the heroku router log. It's the "service" time that you'll see as something like "... wait=0ms connect=1ms service=383ms ...". Definitely wish New Relic was graphing that somewhere...