That's latency, not bandwidth. This is to help scale out concurrent request counts per box, rather than time to serve a single request (though obviously with re

That's latency, not bandwidth. This is to help scale out concurrent request counts per box, rather than time to serve a single request (though obviously with request queuing, failure to serve requests fast enough makes concurrency a latency bottleneck). And sure, most people aren't instagram. This is a python-at-big-scale problem, when the cost benefits of fitting more requests onto fewer servers actually matter. One of the biggest reasons this actually makes a difference is that, in SoA, internal API scaling against your monolith can become expensive.