Surely the optimum case is to have all your machines pegged at 100% CPU True, but given the typical difference between average load and peak load, you wouldn't

Surely the optimum case is to have all your machines pegged at 100% CPU True, but given the typical difference between average load and peak load, you wouldn't want to have your systems at 100% most of the time anyway. Of course, what you really ought to do is scale down during non-peak periods and scale up when traffic increases so that your utilization is always high, but before EC2/etc. that wasn't feasible for most people to do.