4 ms·
Actually that makes a tremendous difference. It is a bit difficult to explain but even 2 concurrent requests per dyno is a lot better than 1. As you go up from
by jules 13y ago
Actually that makes a tremendous difference. It is a bit difficult to explain but even 2 concurrent requests per dyno is a lot better than 1. As you go up from 2 to 3 to 4 the difference quickly becomes minimal. I'll try to explain why, but warning: hand waving ahead!
The problem with only 1 concurrent request is that if that request takes long to process then it can block other requests that are queued up in that dyno. Lets say that 1% of the requests take long. If a dyno can process only 1 request concurrently then at each request there is a 0.01 chance that that will block other requests. If however it can process 4 concurrent requests, then the dyno will only be blocked if it gets 4 long requests at the same time. So there is only a 0.01^4 = 0.00000001 chance that a dyno will be blocked.
- adrianpike 13y agoNot quite. If you keep routing to a blocked backend, any requests that hit that backend will have unacceptable latencies. Since the routing is essentially random, you'll still be fielding about 1/4 of your requests to that backend, so one long request winds up making 1/4 of all your requests have unacceptable latencies.
- jules 13y agoI don't think that's how it works. Within a single dyno with concurrency of 4 a single request can not block block any requests. It can only block 1 thread, but then the other requests will be processed on the other threads. Routing is only random to dynos, within dynos it is not random.
- oillio 13y agoAt the end of the day, there must be some applications that cannot be fixed by increasing concurrency to the limit in a standard dyno. They would not bother implementing XL dynos if this was not the case.
- jules 13y agoIt's most beneficial for application that cannot fit more than 1 on a single dyno.