Are you talking about the measured load (as part of the load average) or a general state of the machine as 'heavily loaded'? The short answer is: it should be

Are you talking about the measured load (as part of the load average) or a general state of the machine as 'heavily loaded'? The short answer is: it should be smaller than the number of cpu cores you have, if the sort of tasks you're running are actually using the cpu (vs just dispatching I/O, for example). A 24-core machine with load averages in their teens is still pretty responsive. Then again, considering that the right definition should specifically say that it's the number of processes in the 'Runnable' state. It's a measurement of the CPU as a performance bottleneck. Other bottlenecks (e.g. your disk(s)) aren't covered.