5 ms·

In the HPC segment, there might also be compiler issues.

ams6110 8y ago
That's true, universities for example will often use Intel compilers on academic licenses, other sites may use other performance-oriented commercial compilers such as PGI.
greglindahl 8y ago
HPC compilers have supported AMD pretty well for the past 15 years, from back when Opteron was the best x64 for a couple of generations.
bayindirh 8y ago
Move aside, I'm an HPC administrator! :) In the HPC world, things are not as clear cut benchmarks, or the vendors' own marketing materials/numbers. First of all, the application you're running may be developed for a specific compiler, and the code sometimes depends on optimization behavior of a compiler. So, changing compilers changes a lot of things. This is why we have both Intel's tools, and GCC toolchain fully supported. For example, LAPACK and its siblings take compiler behavior and CPU specifications into consideration while compiling in an optimized way to maximize its performance IIRC. Also, there's no guarantee that Intel's compilers are fastest on Intel hardware. In the days of Opteron 6100s, using Intel compilers, we were able to beat Intel processors of the same era. You heard it right: Compile using intel compiler with specific flags, run on AMD CPUs, get higher performance, profit! Intel's AVX512 is well used and abused in HPC world, however AMD's HPC performance is not as bad as jandrewrogers implied in his comment [0]. AMD is originally an FPU company, and while their scalar instructions may lack on paper, they run really fast. In HPC world, the CPU/board architecture becomes irrelevant after some point. SpecCPU benchmarks are the ultimate benchmarks, because their behavior is compiler agnostic and push every aspect of the CPU very very hard. If you can get the same SpecFP with an Intel part, you can get the more or the less same performance on real workloads. If you have any other questions, you can AMA. I'll try my best to answer. Funny addenda: We have some applications used widely by users, and when fully optimized, some older Intel CPUs outpace the newer ones by a significant margin. This is some heavy handed, exotic optimization. [0]: https://news.ycombinator.com/item?id=18628592 https://news.ycombinator.com/item?id=18628592
- jcranmer 8y ago
  > Intel's AVX512 is well used and abused in HPC world, however AMD's HPC performance is not as bad as jandrewrogers implied in his comment [0]. I know earlier AMD processors didn't actually have a 256-bit support, so AVX instructions were actually implemented by soaking up two 128-bit lanes (it helps that AVX doesn't have many instructions that actually permit you to move data between the two 128-bit slices of a 256-bit vector). For their AVX-512 performance to not be absolutely horrible, I take it they've actually built real AVX-512 units at some point?
  - bayindirh 8y ago
    From what I've read now [0], it looks like AMD still uses 2 x 128bit AVX units to execute AVX2 instructions. Also, AMD is always coming a generation behind Intel in terms of FP instructions sets, so Zen doesn't support AVX512. According to WikiChip [4], Zen 2 actually has 256 bit FPU paths. I was unable to find a credible benchmark for Zen 2, so I can't talk about its performance. However, when analyzed from the perspective I've given below, it's not hard to assume that Zen 2 is a heavy hitter in terms of floating point performance. However, the interesting part is, when you look to SpecCPU 2017 FP Rate [1], AMD Epyc 7601 [2] system has a similar per core performance with a much bigger Intel Xeon Platinum 8180 [3] system. Why interesting? * AMD's per core base (lowest) rate is 4.1875. * Intel's per core base (lowest) rate is 4.3482. * AMD is running GCC compiled code. * Intel is running Intel compiled code. * Intel has higher clock speed. Intel has some CPUs (like Gold 5118, Gold 6148) which have per core base rate of ~5.125. These are the CPUs are considered as HPC processors, and used by a lot of people. As I said before, it looks like Zen 2 is going to be a better HPC processor than Zen. Zen looks like a very good Enterprise processor now. So with my hat, I can conclude that not having 512 bit hardware is not a crippling omission. Addenda: I forgot to say that Intel has something called "AVX frequency". Since AVX, AVX2 and AVX512 has tremendous power requirements when compared to other operations, Intel lowers CPU to an undisclosed frequency. When I last checked, AVX frequencies of Intel CPUs that we use weren't in the technical guides and were not public in any way. So, the peak SpecFP Rate is not very different from the base ones. Also, since the CPUs thermal budget is very constrained during AVXx operations, other ports' speed is also reduced. So at the end of the day, AVX512 is not a free turbo boost in HPC environments and heavy/continuous loads. [0]: https://en.wikichip.org/wiki/amd/microarchitectures/zen#Floating_Point https://en.wikichip.org/wiki/amd/microarchitectures/zen#Floa... [1]: http://spec.org/cpu2017/results/rfp2017.html http://spec.org/cpu2017/results/rfp2017.html [2]: http://spec.org/cpu2017/results/res2018q4/cpu2017-20180917-08862.html http://spec.org/cpu2017/results/res2018q4/cpu2017-20180917-0... [3]: http://spec.org/cpu2017/results/res2017q4/cpu2017-20171017-00119.html http://spec.org/cpu2017/results/res2017q4/cpu2017-20171017-0... [4]: https://en.wikichip.org/wiki/amd/microarchitectures/zen_2#Key_changes_from_Zen.2B https://en.wikichip.org/wiki/amd/microarchitectures/zen_2#Ke...

That's true, universities for example will often use Intel compilers on academic licenses, other sites may use other performance-oriented commercial compilers such as PGI.

greglindahl 8y ago

HPC compilers have supported AMD pretty well for the past 15 years, from back when Opteron was the best x64 for a couple of generations.

bayindirh 8y ago

Move aside, I'm an HPC administrator! :) In the HPC world, things are not as clear cut benchmarks, or the vendors' own marketing materials/numbers. First of all, the application you're running may be developed for a specific compiler, and the code sometimes depends on optimization behavior of a compiler. So, changing compilers changes a lot of things. This is why we have both Intel's tools, and GCC toolchain fully supported. For example, LAPACK and its siblings take compiler behavior and CPU specifications into consideration while compiling in an optimized way to maximize its performance IIRC. Also, there's no guarantee that Intel's compilers are fastest on Intel hardware. In the days of Opteron 6100s, using Intel compilers, we were able to beat Intel processors of the same era. You heard it right: Compile using intel compiler with specific flags, run on AMD CPUs, get higher performance, profit! Intel's AVX512 is well used and abused in HPC world, however AMD's HPC performance is not as bad as jandrewrogers implied in his comment [0]. AMD is originally an FPU company, and while their scalar instructions may lack on paper, they run really fast. In HPC world, the CPU/board architecture becomes irrelevant after some point. SpecCPU benchmarks are the ultimate benchmarks, because their behavior is compiler agnostic and push every aspect of the CPU very very hard. If you can get the same SpecFP with an Intel part, you can get the more or the less same performance on real workloads. If you have any other questions, you can AMA. I'll try my best to answer. Funny addenda: We have some applications used widely by users, and when fully optimized, some older Intel CPUs outpace the newer ones by a significant margin. This is some heavy handed, exotic optimization. [0]: https://news.ycombinator.com/item?id=18628592 https://news.ycombinator.com/item?id=18628592

jcranmer 8y ago
> Intel's AVX512 is well used and abused in HPC world, however AMD's HPC performance is not as bad as jandrewrogers implied in his comment [0]. I know earlier AMD processors didn't actually have a 256-bit support, so AVX instructions were actually implemented by soaking up two 128-bit lanes (it helps that AVX doesn't have many instructions that actually permit you to move data between the two 128-bit slices of a 256-bit vector). For their AVX-512 performance to not be absolutely horrible, I take it they've actually built real AVX-512 units at some point?
- bayindirh 8y ago
  From what I've read now [0], it looks like AMD still uses 2 x 128bit AVX units to execute AVX2 instructions. Also, AMD is always coming a generation behind Intel in terms of FP instructions sets, so Zen doesn't support AVX512. According to WikiChip [4], Zen 2 actually has 256 bit FPU paths. I was unable to find a credible benchmark for Zen 2, so I can't talk about its performance. However, when analyzed from the perspective I've given below, it's not hard to assume that Zen 2 is a heavy hitter in terms of floating point performance. However, the interesting part is, when you look to SpecCPU 2017 FP Rate [1], AMD Epyc 7601 [2] system has a similar per core performance with a much bigger Intel Xeon Platinum 8180 [3] system. Why interesting? * AMD's per core base (lowest) rate is 4.1875. * Intel's per core base (lowest) rate is 4.3482. * AMD is running GCC compiled code. * Intel is running Intel compiled code. * Intel has higher clock speed. Intel has some CPUs (like Gold 5118, Gold 6148) which have per core base rate of ~5.125. These are the CPUs are considered as HPC processors, and used by a lot of people. As I said before, it looks like Zen 2 is going to be a better HPC processor than Zen. Zen looks like a very good Enterprise processor now. So with my hat, I can conclude that not having 512 bit hardware is not a crippling omission. Addenda: I forgot to say that Intel has something called "AVX frequency". Since AVX, AVX2 and AVX512 has tremendous power requirements when compared to other operations, Intel lowers CPU to an undisclosed frequency. When I last checked, AVX frequencies of Intel CPUs that we use weren't in the technical guides and were not public in any way. So, the peak SpecFP Rate is not very different from the base ones. Also, since the CPUs thermal budget is very constrained during AVXx operations, other ports' speed is also reduced. So at the end of the day, AVX512 is not a free turbo boost in HPC environments and heavy/continuous loads. [0]: https://en.wikichip.org/wiki/amd/microarchitectures/zen#Floating_Point https://en.wikichip.org/wiki/amd/microarchitectures/zen#Floa... [1]: http://spec.org/cpu2017/results/rfp2017.html http://spec.org/cpu2017/results/rfp2017.html [2]: http://spec.org/cpu2017/results/res2018q4/cpu2017-20180917-08862.html http://spec.org/cpu2017/results/res2018q4/cpu2017-20180917-0... [3]: http://spec.org/cpu2017/results/res2017q4/cpu2017-20171017-00119.html http://spec.org/cpu2017/results/res2017q4/cpu2017-20171017-0... [4]: https://en.wikichip.org/wiki/amd/microarchitectures/zen_2#Key_changes_from_Zen.2B https://en.wikichip.org/wiki/amd/microarchitectures/zen_2#Ke...