This is a useful addition. Yes, the above reasoning was assuming that we (a) had located a bottleneck, (b) were planning to vertically scale the capacity of t

This is a useful addition. Yes, the above reasoning was assuming that we (a) had located a bottleneck, (b) were planning to vertically scale the capacity of that bottleneck, and (c) in doing so won't run into a different bottleneck! It is still useful for many cases of horizontal scaling because from sufficiently far away, c identical servers looks a lot like a single server at c times the capacity of a single one. Many applications I encounter in practise does not require one to be very far away to make that simplifying assumption. > How close to half depends on the ratio of queue time to processing time. I don't think this is generally true, but I think I see what you're going for: you're using queue length as a proxy for how far away we have to be to pretend horizontal scaling is a useful proxy for vertical scaling, right?