You don't always necessarily need to choose one or the other. Frequently there exists perfectly good compromise that has almost the benefit of best latency and

You don't always necessarily need to choose one or the other. Frequently there exists perfectly good compromise that has almost the benefit of best latency and almost the best throughput. For example, you have a batch process to process 1 million elements to return a response. You can either process all 1 million elements together and then return response (best throughput at the cost of waiting until everything is processed) or stream results of each one item as they are being processed (best latency but you pay additional cost per item). But there is possibility you can process them in batches of 100? 1000? This means all the additional costs of processing items individually are amortized to something like 1/100th or 1/1000th of the original cost but you still are getting first piece of response very soon (1/1000th or 1/10000th of what you would have to wait if you processed everything in one go). I am currently building large scale trade processing systems using those rules to stream what would traditionally be a batch process, and so far this works beautifully and applies neatly to wide range of problems.