That's a really good question... I think the answer is something like this (just spitballing here, not trying to make a strong claim): At some point in a syste

That's a really good question... I think the answer is something like this (just spitballing here, not trying to make a strong claim): At some point in a system, you may need a response to one request in order to generate the next request. At this point latency affects throughput. Also, apparently, their "lattice composition" technique let's them push concurrency to a lower level, which would allow them to avoid the overhead of doing concurrency at a higher level. What I mean is, if you have to process items one-at-a-time in a replica, then to process more items at a time, you need more replicas. But each replica has overhead -- the details depend on what technology we're talking about, but there's overhead to each replica... e.g., to utilize 4 cores on your server you might host four replicas on that server... but now you have to maintain four memory spaces as well, which takes cycles, even though all you wanted was to make use of all your cores to process items. And it might not be four cores, but 18 or 36 or perhaps 1000s. At some point, the cost of the overhead is greater than amount of benefit.