The average GPU utilization in AI clusters is nowhere near 100%. More like 15-20% outside of hyperscalers being generous and maybe mid 30% for the hyperscalers

The average GPU utilization in AI clusters is nowhere near 100%. More like 15-20% outside of hyperscalers being generous and maybe mid 30% for the hyperscalers for periods of heavy activity. There is way too much gated on data movement at this point. Sure, CXL or other interconnect schemes may help but there are whole-system challenges for scheduling and dependency management and staging that drive these numbers.