The 100% CPU utilization is an odd one because, although theoretically it's more "efficient" to use a system at full capacity, performance tends to suffer and p

The 100% CPU utilization is an odd one because, although theoretically it's more "efficient" to use a system at full capacity, performance tends to suffer and practically make it unusable (others' pointed here to queueing theory). We are building a high load, low latency inference system, and we noticed that latency is correlated to CPU usage. Latency grows slowly linearly with CPU usage, up to a point where it starts to grow exponentially. We had a bug where a host was pegged at 100% CPU, and our tail latency suffered greatly, since any request that hit that host was basically toast. Funnily enough OP could have used examples outside computing where efficiency directly trades off with resiliency. For example, in farming large monocultures are the most efficient, but a single disease can wipe entire plantations. In manufacturing, keeping stock is seen as an inefficiency, but problems in the supply chain can get you in hot water (as COVID made painfully obvious).