3 ms·
> Utilization is choosing throughout and cost over latency. For a given rate of incoming work, increasing throughput decreases utilization. So I guess it's not
by anonymoushn 4y ago
> Utilization is choosing throughout and cost over latency.
For a given rate of incoming work, increasing throughput decreases utilization. So I guess it's not choosing throughput too much.
> People care about utilization when, eg, processing petabytes of data — something that takes hours to days, where you’re optimizing batch throughout and cost. By making sure you efficiently utilize your hardware.
If you have fixed resources and lots of work, it would be good to have nonempty queues to prevent them from idling. That is maybe the big unwritten part of the comment upthread. It doesn't seem worth saying "no, the article is wrong in all cases because of this particular case!" though, so I'm not sure.
- remram 4y agoIf you define throughput as "processed items over time per processor", the tradeoff becomes clear. Given the rate of incoming requests, you can get more processors, which will increase cost and reduce latency and throughput-per-processor, or you can get fewer processors, which will decrease cost and increase latency and throughput-per-processor. If you have fixed resources and fixed work, wanting your queues to be empty is a wish, not a strategy.