6 ms·

CPU-Z's Inadequate Benchmark

69 points by picture 3y ago

tedunangst 3y ago
I can't recall ever seeing the CPU-Z benchmark used anywhere. Didn't even know it existed until just now.
- deleted 3y ago
  [deleted]
NelsonMinar 3y ago
Next you're gonna tell me that Bogomips aren't really MIPS.
w0z_ 3y ago
"They should run typical desktop applications" This by itself is a terrible benchmark. What "typical desktop application" 1) does not slaughter your RAM usage, 2) does not require a high-end spec computer to run without hiccuping constantly, 3) does not open 10 processes for 1 application. I have Adobe Creative cloud, I still use Photoshop CS6 because of how bad new Adobe products use resources. Same with Google Chrome - I only QA with it because it's a TERRIBLE daily browser resource-wise. Excel? F*ck I hate opening large data-sets with that. Now if you steer away from business applications and look at gaming (what most CPU-Z users benchmark for) - look at the game engines or the games. 99% of games are absolute trash at utilizing resources correctly. Valve and some Blizzard titles were the only companies that really cared and put forth effort behind optimizing their games so lower end PC's can play them, hence their explosive popularity world-wide.
- Jonnax 3y ago
  "They should run typical desktop applications and monitor performance counters to see what typical branch and memory footprints look like. These types of data should be used to inform benchmark design for future CPU-Z versions." You selectively quoted them to rant about something they weren't talking about. The entire article was about how the design of their benchmark doesn't reflect how applications people use work. ~100% of games don't fit into the L1 cache of a CPU.
  - beebeepka 3y ago
    ~100% of games don't fit into the L1 cache of a CPU. It's the weekend, so I am just gonna say that a Ryzen 7800x3d has 512KB of L1 cache. Plenty of games would fit. Not even talking about those tiny games that generate everything.
    - mort96 3y ago
      L1 cache isn’t shared between cores. One core in the 7800X3D only has 64k L1 cache.
      - adrian_b 3y ago
        Of the 64 kB L1 cache, only 32 kB is available for data.
    - adrian_b 3y ago
      Despite the fact that AMD writes "512 kB" in its spec sheet, this number does not matter, because it multiplies the cache size with the number of cores. The Ryzen 7000 CPUs have 32 kB of L1 instruction cache, 32 kB of L1 data cache and 1 MB of L2 cache per core, and 7800X3D also has 96 MB of L3 cache shared by the 8 cores. These numbers determine at which cache level the data used by an execution thread can fit. No modern game uses data that fits in the 32 kB of L1 data cache, and the game code also does not fit in the 32 kB instruction cache.
- snvzz 3y ago
  Games and desktop applications are what people actually use. Thus there's strong practical sense in benchmarking them.
  - croes 3y ago
    Games and applications get updates that change the performance, not to mention other effects an performance by drivers etc.
AlotOfReading 3y ago
CPU-Z's benchmark is designed to give results that reflect the "average IPC" across many generations of processor without privileging specific architectural and microarchitectural choices. In fact, the current benchmark was written because the old one accidentally privileged Ryzen due to microarchitectural quirks in the ALU [0]. On-CPU procedural generation didn't end up being the major thing people thought it would be, so it's not all that representative of modern applications, but it's fine if your goal is to make a reasonably fair social media leaderboard of "CPU performance numbers". It's hard to call it inadequate when that seems to be the intended purpose. [0] https://www.cpuid.com/news/51-cpu-z-1-79-new-benchmark-new-scores.html https://www.cpuid.com/news/51-cpu-z-1-79-new-benchmark-new-s...
- facialwipe 3y ago
  The thing about CPU-Z’s benchmark is it highlights that all 100% CPU stress tests (and workloads) are not created equal. Your overclock may be able to survive CPU-Z and Cinebench, but will fall to its knees with Prime95. All are taxing the CPU to 100% in different ways.
- snvzz 3y ago
  >CPU-Z's benchmark is designed to give results that reflect the "average IPC" across many generations of processor without privileging specific architectural and microarchitectural choices. If it was the case, it should have limited itself to the top eight instructions that matter[0], rather than the relatively esoteric SSE. CPU-Z is just a terrible benchmark. With this article, now we can point somewhere every time someone treats CPU-Z's result as if it was meaningful. Thus I am thankful for it. 0. https://www.anandtech.com/show/16762/an-anandtech-interview-with-jim-keller-laziest-person-at-tesla https://www.anandtech.com/show/16762/an-anandtech-interview-...
  - berkut 3y ago
    SSE2 is supported by all x64 processors (SSE and SSE2 are literally part of the AMD64/x86-64 spec), so in terms of support, I don't see how that could be "relatively esoteric"...
pomian 3y ago
And yet, CPU-Z, is still a fantastic tool to see what ram your system has, what motherboard, what processor, especially in am old system you may want to upgrade. Just to know what family of processor you have, is extremely useful. It's a great tool at what it does well.
- Laforet 3y ago
  I would take whatever information they show with a grain of salt as it seems to simply relay superficial information the system BIOS/UEFI reports without verification. For example, a stick or channel of system RAM may fail training at boot and end up being silently disabled. CPU-Z would blindly report all memory channels being present despite Windows task manager showing correctly that some of the memory are unavailable.
- Neil44 3y ago
  I love that it's a small portable exe. I don't want to install some app to view some info one time.
- bj-rn 3y ago
  Alternatively there is HWinFO (which also has a portable version). https://www.hwinfo.com https://www.hwinfo.com
- pixelpoet 3y ago
  > Just to know what family of processor you have, is extremely useful. Just start task manager, ctrl+shift+esc... (BTW to me owning a computer and not knowing what's inside is like not knowing what brand of car you own [never owned a car personally but I imagine this analogy works for most].)
  - Moru 3y ago
    After owning computers for 40 years and helping friends and family with computers just as long, this gets harder and harder to keep track of. Especially now when there is no real need for a new computer unless something breaks. Then I need to know what I have and what I can replace it with. So now I run the tools and save the info on my harddrive to be able to look up before I go home to someone to fix a computer.
snvzz 3y ago
I favor emBench[0]. Compile for the target platform, run, then note result. 0. https://www.embench.org/ https://www.embench.org/
- duskwuff 3y ago
  emBench is designed for embedded platforms, not desktop systems; one of the design criteria is that the entire test suite should compile to under 64 kB of code, and should use under 64 kB of RAM. This has the effect of making the benchmark absolutely trivial for desktop-class CPUs -- most tests will run entirely in cache, placing no load at all on the memory controller, and features like branch prediction will be unfairly effective.
  - snvzz 3y ago
    >most tests will run entirely in cache, placing no load at all on the memory controller, Exactly. Like CPU-Z. Yet unlike CPU-Z, it is not an extremely biased sse microbenchmark, but does instead provide decent coverage of a CPU's functionality. The benchmarks within emBench include common code like gzip or jpeg crunch/decrunch routines, which are frequent stress points in real world applications.

69 points by picture 3y ago

tedunangst 3y ago

I can't recall ever seeing the CPU-Z benchmark used anywhere. Didn't even know it existed until just now.

deleted 3y ago
[deleted]

NelsonMinar 3y ago

Next you're gonna tell me that Bogomips aren't really MIPS.

w0z_ 3y ago

"They should run typical desktop applications" This by itself is a terrible benchmark. What "typical desktop application" 1) does not slaughter your RAM usage, 2) does not require a high-end spec computer to run without hiccuping constantly, 3) does not open 10 processes for 1 application. I have Adobe Creative cloud, I still use Photoshop CS6 because of how bad new Adobe products use resources. Same with Google Chrome - I only QA with it because it's a TERRIBLE daily browser resource-wise. Excel? F*ck I hate opening large data-sets with that. Now if you steer away from business applications and look at gaming (what most CPU-Z users benchmark for) - look at the game engines or the games. 99% of games are absolute trash at utilizing resources correctly. Valve and some Blizzard titles were the only companies that really cared and put forth effort behind optimizing their games so lower end PC's can play them, hence their explosive popularity world-wide.

Jonnax 3y ago
"They should run typical desktop applications and monitor performance counters to see what typical branch and memory footprints look like. These types of data should be used to inform benchmark design for future CPU-Z versions." You selectively quoted them to rant about something they weren't talking about. The entire article was about how the design of their benchmark doesn't reflect how applications people use work. ~100% of games don't fit into the L1 cache of a CPU.
- beebeepka 3y ago
  ~100% of games don't fit into the L1 cache of a CPU. It's the weekend, so I am just gonna say that a Ryzen 7800x3d has 512KB of L1 cache. Plenty of games would fit. Not even talking about those tiny games that generate everything.
  - mort96 3y ago
    L1 cache isn’t shared between cores. One core in the 7800X3D only has 64k L1 cache.
    - adrian_b 3y ago
      Of the 64 kB L1 cache, only 32 kB is available for data.
  - adrian_b 3y ago
    Despite the fact that AMD writes "512 kB" in its spec sheet, this number does not matter, because it multiplies the cache size with the number of cores. The Ryzen 7000 CPUs have 32 kB of L1 instruction cache, 32 kB of L1 data cache and 1 MB of L2 cache per core, and 7800X3D also has 96 MB of L3 cache shared by the 8 cores. These numbers determine at which cache level the data used by an execution thread can fit. No modern game uses data that fits in the 32 kB of L1 data cache, and the game code also does not fit in the 32 kB instruction cache.
snvzz 3y ago
Games and desktop applications are what people actually use. Thus there's strong practical sense in benchmarking them.
- croes 3y ago
  Games and applications get updates that change the performance, not to mention other effects an performance by drivers etc.

AlotOfReading 3y ago

CPU-Z's benchmark is designed to give results that reflect the "average IPC" across many generations of processor without privileging specific architectural and microarchitectural choices. In fact, the current benchmark was written because the old one accidentally privileged Ryzen due to microarchitectural quirks in the ALU [0]. On-CPU procedural generation didn't end up being the major thing people thought it would be, so it's not all that representative of modern applications, but it's fine if your goal is to make a reasonably fair social media leaderboard of "CPU performance numbers". It's hard to call it inadequate when that seems to be the intended purpose. [0] https://www.cpuid.com/news/51-cpu-z-1-79-new-benchmark-new-scores.html https://www.cpuid.com/news/51-cpu-z-1-79-new-benchmark-new-s...

facialwipe 3y ago
The thing about CPU-Z’s benchmark is it highlights that all 100% CPU stress tests (and workloads) are not created equal. Your overclock may be able to survive CPU-Z and Cinebench, but will fall to its knees with Prime95. All are taxing the CPU to 100% in different ways.
snvzz 3y ago
>CPU-Z's benchmark is designed to give results that reflect the "average IPC" across many generations of processor without privileging specific architectural and microarchitectural choices. If it was the case, it should have limited itself to the top eight instructions that matter[0], rather than the relatively esoteric SSE. CPU-Z is just a terrible benchmark. With this article, now we can point somewhere every time someone treats CPU-Z's result as if it was meaningful. Thus I am thankful for it. 0. https://www.anandtech.com/show/16762/an-anandtech-interview-with-jim-keller-laziest-person-at-tesla https://www.anandtech.com/show/16762/an-anandtech-interview-...
- berkut 3y ago
  SSE2 is supported by all x64 processors (SSE and SSE2 are literally part of the AMD64/x86-64 spec), so in terms of support, I don't see how that could be "relatively esoteric"...

pomian 3y ago

And yet, CPU-Z, is still a fantastic tool to see what ram your system has, what motherboard, what processor, especially in am old system you may want to upgrade. Just to know what family of processor you have, is extremely useful. It's a great tool at what it does well.

Laforet 3y ago
I would take whatever information they show with a grain of salt as it seems to simply relay superficial information the system BIOS/UEFI reports without verification. For example, a stick or channel of system RAM may fail training at boot and end up being silently disabled. CPU-Z would blindly report all memory channels being present despite Windows task manager showing correctly that some of the memory are unavailable.
Neil44 3y ago
I love that it's a small portable exe. I don't want to install some app to view some info one time.
bj-rn 3y ago
Alternatively there is HWinFO (which also has a portable version). https://www.hwinfo.com https://www.hwinfo.com
pixelpoet 3y ago
> Just to know what family of processor you have, is extremely useful. Just start task manager, ctrl+shift+esc... (BTW to me owning a computer and not knowing what's inside is like not knowing what brand of car you own [never owned a car personally but I imagine this analogy works for most].)
- Moru 3y ago
  After owning computers for 40 years and helping friends and family with computers just as long, this gets harder and harder to keep track of. Especially now when there is no real need for a new computer unless something breaks. Then I need to know what I have and what I can replace it with. So now I run the tools and save the info on my harddrive to be able to look up before I go home to someone to fix a computer.

snvzz 3y ago

I favor emBench[0]. Compile for the target platform, run, then note result. 0. https://www.embench.org/ https://www.embench.org/

duskwuff 3y ago
emBench is designed for embedded platforms, not desktop systems; one of the design criteria is that the entire test suite should compile to under 64 kB of code, and should use under 64 kB of RAM. This has the effect of making the benchmark absolutely trivial for desktop-class CPUs -- most tests will run entirely in cache, placing no load at all on the memory controller, and features like branch prediction will be unfairly effective.
- snvzz 3y ago
  >most tests will run entirely in cache, placing no load at all on the memory controller, Exactly. Like CPU-Z. Yet unlike CPU-Z, it is not an extremely biased sse microbenchmark, but does instead provide decent coverage of a CPU's functionality. The benchmarks within emBench include common code like gzip or jpeg crunch/decrunch routines, which are frequent stress points in real world applications.