3 ms·

"What we will note is that Ampere de-rated both the AMD EPYC 7742 and Xeon Platinum 8280 results by 16.5% and 24% respectively. This was done to adjust for usin

"What we will note is that Ampere de-rated both the AMD EPYC 7742 and Xeon Platinum 8280 results by 16.5% and 24% respectively. This was done to adjust for using GCC versus AOCC2.0 and ICC 19.0.1.144. Ampere disclosed this, and it is a big impact. Arm servers tend to use GCC as the compiler while there are more optimized compilers out there for AMD and Intel." If I read this right, they reduce their competitors' benchmarks because they have better compilers? Can anyone justify this?

wmf 7y ago
If you assume that most real customers use a regular compiler like GCC or Clang then benchmarks using tuned compilers like AOCC (never heard of it before today) or ICC are unrepresentative. However, the proper way to make such a comparison would be to run benchmarks using GCC on all the chips, not to apply magic derating factors. Shame on Ampere for such voodoo benchmarketing. Oh, and don't miss the 3.0 vs. 3.3 GHz.
Patrick-STH 7y ago
GCC is the open-source compiler that is used all over. Not every AMD EPYC system people are using AOCC2.0 on. Likewise, people do not only compile code on ICC that is used on Intel Xeons. Arm has focused efforts on getting optimizations in GCC because it is so popular. Generally, that is why we prefer to publish "compiler optimized" as best-case performance as well as "GCC" as more of the least common denominator. Both sets of data points are important. Official SPECint published numbers will not use GCC because the organizations that submit them always want to see the best performance. Ampere used a scaling factor off of published numbers. If you want to see the impact, we have some numbers from my ThunderX2 review: https://www.servethehome.com/cavium-thunderx2-review-benchmarks-real-arm-server-option/5/ https://www.servethehome.com/cavium-thunderx2-review-benchma... You can see the impact clearly there even though that was from a few years ago. Cray has a better performing compiler for ThunderX2 but we did not get to use it due to licensing restrictions. I hope that helps. The bigger need is for more data since this is one view of performance. There are other needs as well such as FP performance.
- ksec 7y ago
  Nice to know both Patrick and Ian ( Anandtech ) are on HN :) When could we expect a review on Altra?
  - Patrick-STH 7y ago
    Ha! Sometimes it surprises people that we know each other and hang out a bit when we are in the same town. On an Altra review. Great question. I have been bringing it up for some time and live 15 minutes from their headquarters. The invitation is open on our end.
p1necone 7y ago
Very little software is actually compiled with AOCC and ICC. Really Intel and AMD are being dishonest by publishing benchmarks that don't match reality. Of course it's different if you're compiling everything yourself, then those benchmarks might be relevant.
- adev_ 7y ago
  > It's different if you're compiling everything yourself, then those benchmarks might be relevant. And even if you do it's irrelevant. Most common large/important frameworks won't compile with proprietary compilers. Doing Bench's with ICC, XLC or other is hypocritical and often does not reflect anything useful. Only the HPC world can afford to recompile everything with proprietary compilers and justify the man power to do so. And even so, they already have passed most compute intensive kernels on GPGPU with cuda a long time ago.
  - pjmlp 7y ago
    IBM, Intel and PGI reign on HPC.
  - pierrebai 7y ago
    No, the correct thing to do is to publish multiple columns showing performance under the different compilers. Large co do use the specific compiler that will give them better performance. I know many big software compiled with ICC. Then, there is the lack of tlak about MSVC. That's one standard compiler used extensively. In benchmarking you have two choices: publish the real numbers or not. Which option you choose marks you as honest or not. You can argue about why the numbers for your product are lower in the discussion section of your report. Not in an asterisk.
- endorphone 7y ago
  "Really Intel and AMD are being dishonest by publishing benchmarks that don't match reality" People can discard with the various SPEC* benchmarks as unrealistic -- they generally are -- but this is a bridge way too far. The whole point of the SPEC* benchmarks is that you bring out the best of the best and do everything you can to achieve the pinnacle of performance. And those SPEC* benchmarks are most interesting, and most applicable, to HPC users, who actually do use hardware-specific compilers. Of course another person mentioned that "everything is on GPUs now" (it isn't): If it's on a GPU, then you don't need a 60 core CPU. If it's on a GPU, then SPEC* benchmarks aren't relevant.
sliken 7y ago
Assuming you are going to buy a server to run gcc compiled code, that seems fair. After all just about every linux distro uses gcc of clang/llvm to compile their binaries. Using Intel's compiler or AOCC to compile binaries for normal server use is pretty rare.

wmf 7y ago

If you assume that most real customers use a regular compiler like GCC or Clang then benchmarks using tuned compilers like AOCC (never heard of it before today) or ICC are unrepresentative. However, the proper way to make such a comparison would be to run benchmarks using GCC on all the chips, not to apply magic derating factors. Shame on Ampere for such voodoo benchmarketing. Oh, and don't miss the 3.0 vs. 3.3 GHz.

Patrick-STH 7y ago

GCC is the open-source compiler that is used all over. Not every AMD EPYC system people are using AOCC2.0 on. Likewise, people do not only compile code on ICC that is used on Intel Xeons. Arm has focused efforts on getting optimizations in GCC because it is so popular. Generally, that is why we prefer to publish "compiler optimized" as best-case performance as well as "GCC" as more of the least common denominator. Both sets of data points are important. Official SPECint published numbers will not use GCC because the organizations that submit them always want to see the best performance. Ampere used a scaling factor off of published numbers. If you want to see the impact, we have some numbers from my ThunderX2 review: https://www.servethehome.com/cavium-thunderx2-review-benchmarks-real-arm-server-option/5/ https://www.servethehome.com/cavium-thunderx2-review-benchma... You can see the impact clearly there even though that was from a few years ago. Cray has a better performing compiler for ThunderX2 but we did not get to use it due to licensing restrictions. I hope that helps. The bigger need is for more data since this is one view of performance. There are other needs as well such as FP performance.

ksec 7y ago
Nice to know both Patrick and Ian ( Anandtech ) are on HN :) When could we expect a review on Altra?
- Patrick-STH 7y ago
  Ha! Sometimes it surprises people that we know each other and hang out a bit when we are in the same town. On an Altra review. Great question. I have been bringing it up for some time and live 15 minutes from their headquarters. The invitation is open on our end.

p1necone 7y ago

Very little software is actually compiled with AOCC and ICC. Really Intel and AMD are being dishonest by publishing benchmarks that don't match reality. Of course it's different if you're compiling everything yourself, then those benchmarks might be relevant.

adev_ 7y ago
> It's different if you're compiling everything yourself, then those benchmarks might be relevant. And even if you do it's irrelevant. Most common large/important frameworks won't compile with proprietary compilers. Doing Bench's with ICC, XLC or other is hypocritical and often does not reflect anything useful. Only the HPC world can afford to recompile everything with proprietary compilers and justify the man power to do so. And even so, they already have passed most compute intensive kernels on GPGPU with cuda a long time ago.
- pjmlp 7y ago
  IBM, Intel and PGI reign on HPC.
- pierrebai 7y ago
  No, the correct thing to do is to publish multiple columns showing performance under the different compilers. Large co do use the specific compiler that will give them better performance. I know many big software compiled with ICC. Then, there is the lack of tlak about MSVC. That's one standard compiler used extensively. In benchmarking you have two choices: publish the real numbers or not. Which option you choose marks you as honest or not. You can argue about why the numbers for your product are lower in the discussion section of your report. Not in an asterisk.
endorphone 7y ago
"Really Intel and AMD are being dishonest by publishing benchmarks that don't match reality" People can discard with the various SPEC* benchmarks as unrealistic -- they generally are -- but this is a bridge way too far. The whole point of the SPEC* benchmarks is that you bring out the best of the best and do everything you can to achieve the pinnacle of performance. And those SPEC* benchmarks are most interesting, and most applicable, to HPC users, who actually do use hardware-specific compilers. Of course another person mentioned that "everything is on GPUs now" (it isn't): If it's on a GPU, then you don't need a 60 core CPU. If it's on a GPU, then SPEC* benchmarks aren't relevant.

sliken 7y ago

Assuming you are going to buy a server to run gcc compiled code, that seems fair. After all just about every linux distro uses gcc of clang/llvm to compile their binaries. Using Intel's compiler or AOCC to compile binaries for normal server use is pretty rare.