3 ms·
Geekbench doesn’t use the GPU; Geekbench also doesn’t use AVX, so half of the floating-point performance of recent cores is left on the table. Actually, lookin
by stephencanon 13y ago
Geekbench doesn’t use the GPU; Geekbench also doesn’t use AVX, so half of the floating-point performance of recent cores is left on the table.
Actually, looking at disassembly of Geekbench, they barely even use SSE vectors, so it’s really nearly 3/4 of the floating-point performance that goes unmeasured by the benchmark.
One could perhaps argue in the defense of the benchmark that many consumer applications don’t take advantage of these ISA extensions, but you definitely cannot argue that pro apps don’t use them; this alone makes Geekbench (and in fairness, most benchmarks, which tend to be pretty naive) extremely misleading for evaluation of “pro” hardware. Ultimately there’s only one benchmark that’s useful for such evaluation: actually exercising the workload that you intend to run on the machine.
Geekbench also doesn’t use available libraries that offer high-performance implementations of the computations in question (of which any serious application developer would avail themselves). Looking at numbers for released hardware, the “LU decomposition” results in Geekbench make clear that they are either using far too small of a problem size or using a naive C implementation of the operation and measuring the performance of whatever code their compiler generated. If they wanted to show the performance of the system, they would use one of the available library implementations.
Such benchmarks tell you something, but it isn’t really how good hardware is; it’s closer to a measure of how well a compiler can salvage naive C code when it isn’t allowed to take advantage of the available hardware features. I’m not sure what that’s representative of, but I know that it’s not representative of professional workloads.
- diroussel 13y agoI guess a LINPAK Benchmark would be more suitable? http://www.top500.org/project/linpack/ http://www.top500.org/project/linpack/
- stephencanon 13y agoLINPACK has its own issues as a benchmark; it only exercises extremely predictable floating-point multiply add loops. If a well-tuned implementation of the operation is used (as you see in the scientific computing community), this is representative of the peak arithmetic throughput that the hardware is capable of (so it is a good proxy for raw compute power), but it is still not representative of the way most real workloads (even professional or scientific computing) drive hardware. Even the scientific computing community is slowly starting to move away from LINPACK (see, for instance, http://www.sandia.gov/~maherou/docs/HPCG-Benchmark.pdf http://www.sandia.gov/~maherou/docs/HPCG-Benchmark.pdf). LINPACK as you typically see reported for consumer and mobile devices is far worse; the implementations used there are often naive scalar implementations that do not do any form of cache tiling. This means that for small problem sizes they are actually measuring how clever the compiler is, and for large problem sizes they are actually measuring memory throughput (both of which are interesting, but they are not what a “CPU benchmark” should be reporting). The set of operations covered by Geekbench is actually fairly interesting (with some exceptions). If they simply used the best available libraries instead of their own naive implementations it would almost be a respectable benchmark.
- frozenport 13y agoAlso LINPACK doesn't do IO, which a frequently sets the max speed.
- sjtgraham 13y agoApple also has it's own heavily optimised version of LINPACK available in Accelerate.framework