Dirty little secret: Much the computing on these machines isn't supercomputing and doesn't need fast interconnects. In fact, many (most?) workloads would be be

Dirty little secret: Much the computing on these machines isn't supercomputing and doesn't need fast interconnects. In fact, many (most?) workloads would be best done in the cloud. Except we already bought these machines. So there's tons of frameworks around meant to use these machines like they are AWS pizza boxes. If it's inefficient due to an architectural mismatch, that's fine. You're still using the machine that was paid for, and the only numbers that matter is utilization, and those metrics aren't even defined in terms of CPU utilization. If you reserve 10 nodes, and only run on one node, it's still reported that a 10 node job ran for 6 hours, not that a 10 node job job ran at 10% efficiency for 6 hours. As far as why the accounting is this way, it's assumed that users will attempt to maximize their usage since allocations are defined. But even then, you can blow through your allocation and generally ask for more. But that's not necessarily what happens. In summary: Because we bought these big computers, we have to use them, even if the workload isn't a good match.