4 ms·

They're testing CPU performance, not memory capacity. They need that much for the channels.

by _joel 3y ago

ahahahahah 3y ago
It just seems so incredibly sloppy to not have that be consistent. Do they expect all of their readers to then audit the code of the benchmarking that they are doing to ensure that it cannot possibly be bound by this?
- xcv123 3y ago
  They used Cinebench. You can download this and verify that it uses less than 4GB of RAM. It is a CPU bound benchmark. Adding 384GB of RAM makes no difference whatsoever.
- deleted 3y ago
  [deleted]
- asfdgianionio 3y ago
  There is no need to start out every article with remedial lessons. It is reasonable for an enthusiast magazine to assume familiarity with how computers work. The average ExtremeTech reader should already know what memory is and how increasing the amount of memory changes (or doesn't change) performance. It is not possible to give all the systems the same memory configuration. They don't support the same memory and don't have the same number of channels. And I expect giving all the systems 512 GiB of memory would only widen the lead because larger DIMMs are usually slower. >Do they expect all of their readers to then audit the code of the benchmarking that they are doing to ensure that it cannot possibly be bound by this? Cinebench is hardly obscure. The average ExtremeTech reader will already know what it is and what it measures.
  - ksec 3y ago
    I feel sad just reading this reply has to be spelled out so clearly on HN. The standard practice that has been the case since before the Internet.
- dathinab 3y ago
  it's not sloppy, it's required! you want to max out memory channels on all devices to test max performance but you also want to use RAM DIMs which are equally good, if possible the same as AMD has more memory channels this means it will get more "equally good"(1) RAM DIMs and in turn have more memory and as long as you make sure that no of your benchmarks perform pressure wrt. RAM capacity (e.g. max RAM usage 64GiB) you now have a fair benchmark wrt. max performance naturally if you don't plan to max out your RAM channels on AMD you do need a different benchmark for non-max performance (1): "equally good" is also a bit tricky as different vendors have slightly subtle differences in access patterns and DDR5 ram training also can make the same RAM run with different settings on different systems which can make the same ram dim not equally good, but there probably is some RAM which doesn't have such issues
mumblemumble 3y ago
If the benchmark has any cache misses at all then you also want to make sure everything about the memory configuration is the same, too, for a fair CPU benchmark, right? Number of channels, latency, etc. No, the actual amount of RAM isn't directly relevant, but it seems like a possible brown m&m to me. (https://www.snopes.com/fact-check/brown-out/ https://www.snopes.com/fact-check/brown-out/)
- dathinab 3y ago
  > for a fair CPU benchmark, right? Number of channels, That's exactly where the differences comes from you want to max out the available memory channels on each device not have the same number. Same for latency etc. I mean your also not fixing the clock speed to 2GHz no boost or when comparing cars limit things just to the first 3 gears or similar. Because if you would add such artificial limits you won't compare fairly how fast the _max performance_ can be, it would be a very different comparison and wrt. evaluating max performance very unfair. Now you still need to e.g. have a fair choice of RAM, so preferably if possible use the same RAM sticks (and also use RAM which works equally well with both Intel and AMD). But what that means is that you _have to_ give the AMD system more RAM because it has more channels. Now you also _have to_ make sure that you benchmarks don't have any RAM pressure wrt. capacity. Also I'm not sure why you mention cache misses? Because number of cache misses is quite independent of RAM amount? Do you mean disk cache or swap misses instead, but I don't think the benchmark does disk I/O and shouldn't have swap either? But even if the benchmark only uses let's say 32GiB of memory for getting a proper GPU
- xcv123 3y ago
  > If the benchmark has any cache misses at all then you also want to make sure everything about the memory configuration is the same, too, for a fair CPU benchmark, right? Number of channels, latency, etc. No. Adding that memory bottleneck would make it an unfair and worthless comparison. If you are testing the maximum CPU performance then you need to eliminate those bottlenecks, not create them.

They're testing CPU performance, not memory capacity. They need that much for the channels.

by _joel 3y ago

They're testing CPU performance, not memory capacity. They need that much for the channels.

It just seems so incredibly sloppy to not have that be consistent. Do they expect all of their readers to then audit the code of the benchmarking that they are doing to ensure that it cannot possibly be bound by this?

xcv123 3y ago
They used Cinebench. You can download this and verify that it uses less than 4GB of RAM. It is a CPU bound benchmark. Adding 384GB of RAM makes no difference whatsoever.
deleted 3y ago
[deleted]
asfdgianionio 3y ago
There is no need to start out every article with remedial lessons. It is reasonable for an enthusiast magazine to assume familiarity with how computers work. The average ExtremeTech reader should already know what memory is and how increasing the amount of memory changes (or doesn't change) performance. It is not possible to give all the systems the same memory configuration. They don't support the same memory and don't have the same number of channels. And I expect giving all the systems 512 GiB of memory would only widen the lead because larger DIMMs are usually slower. >Do they expect all of their readers to then audit the code of the benchmarking that they are doing to ensure that it cannot possibly be bound by this? Cinebench is hardly obscure. The average ExtremeTech reader will already know what it is and what it measures.
- ksec 3y ago
  I feel sad just reading this reply has to be spelled out so clearly on HN. The standard practice that has been the case since before the Internet.
dathinab 3y ago
it's not sloppy, it's required! you want to max out memory channels on all devices to test max performance but you also want to use RAM DIMs which are equally good, if possible the same as AMD has more memory channels this means it will get more "equally good"(1) RAM DIMs and in turn have more memory and as long as you make sure that no of your benchmarks perform pressure wrt. RAM capacity (e.g. max RAM usage 64GiB) you now have a fair benchmark wrt. max performance naturally if you don't plan to max out your RAM channels on AMD you do need a different benchmark for non-max performance (1): "equally good" is also a bit tricky as different vendors have slightly subtle differences in access patterns and DDR5 ram training also can make the same RAM run with different settings on different systems which can make the same ram dim not equally good, but there probably is some RAM which doesn't have such issues

mumblemumble 3y ago

If the benchmark has any cache misses at all then you also want to make sure everything about the memory configuration is the same, too, for a fair CPU benchmark, right? Number of channels, latency, etc. No, the actual amount of RAM isn't directly relevant, but it seems like a possible brown m&m to me. (https://www.snopes.com/fact-check/brown-out/ https://www.snopes.com/fact-check/brown-out/)

dathinab 3y ago
> for a fair CPU benchmark, right? Number of channels, That's exactly where the differences comes from you want to max out the available memory channels on each device not have the same number. Same for latency etc. I mean your also not fixing the clock speed to 2GHz no boost or when comparing cars limit things just to the first 3 gears or similar. Because if you would add such artificial limits you won't compare fairly how fast the _max performance_ can be, it would be a very different comparison and wrt. evaluating max performance very unfair. Now you still need to e.g. have a fair choice of RAM, so preferably if possible use the same RAM sticks (and also use RAM which works equally well with both Intel and AMD). But what that means is that you _have to_ give the AMD system more RAM because it has more channels. Now you also _have to_ make sure that you benchmarks don't have any RAM pressure wrt. capacity. Also I'm not sure why you mention cache misses? Because number of cache misses is quite independent of RAM amount? Do you mean disk cache or swap misses instead, but I don't think the benchmark does disk I/O and shouldn't have swap either? But even if the benchmark only uses let's say 32GiB of memory for getting a proper GPU
xcv123 3y ago
> If the benchmark has any cache misses at all then you also want to make sure everything about the memory configuration is the same, too, for a fair CPU benchmark, right? Number of channels, latency, etc. No. Adding that memory bottleneck would make it an unfair and worthless comparison. If you are testing the maximum CPU performance then you need to eliminate those bottlenecks, not create them.