5 ms·

Little bit disappointed by the POWER8 data. I guess it would start paying bigger dividends in scale up systems, but the power and thermals are a little shockin

by kev009 12y ago

Little bit disappointed by the POWER8 data. I guess it would start paying bigger dividends in scale up systems, but the power and thermals are a little shocking - I think there may need to be some tuning and AFAICT Linux may not be running on bare metal. Same for ARM, but I guess the sweet spot there is scale out. Intel has been putting out amazing chips since Haswell. We need the others to do well to keep them honest and innovative.

m_mueller 12y ago
The one big advantage of power architecture is memory bandwidth. Power8 has 230GB/s sustained on a 12 core chip while Xeon Haswell-EP has 102GB/s on 12 core. This means you need more parallelism (=harder to program) and more sockets to get the same speed on Xeons vs. Power8 for memory-bandwidth-limited problems (which >50% of HPC applications are). Power8 has I believe ~50% higher TDP than Xeons, so even in performance per watt for memory bandwidth limited problems, you are still ahead, especially considering all the components you need to power besides the socket. TLDR; These things are for a specific segment of HPC, albeit the one that doesn't care too much about Linpack performance. [1] http://en.wikipedia.org/wiki/POWER8 http://en.wikipedia.org/wiki/POWER8 [2] http://www.pugetsystems.com/blog/2014/09/08/Memory-Performance-for-Intel-Xeon-Haswell-EP-DDR4-596/ http://www.pugetsystems.com/blog/2014/09/08/Memory-Performan...
- kev009 12y ago
  No doubt HPC is a big influence on design since they can sink big gov contracts, but IBM is claiming it's a serious competitor to scale out x86. They've done this in the past.. PowerPC (6xx), PReP, CHRP, PowerLinux, now OpenPOWER. It looks too similar this time where they cannot get things into gear lowering margins to gain volume. The Tyan box is basically a reference platform.. over priced and poorly designed (2U, 3.5" disks, several thousand dollars?!). Crossing my fingers something good will happen, but placing little stock in it.
- StillBored 12y ago
  You should be careful on the memory bandwidth. The more recent xeons (2011v3, ddr4 2133) are rated 68 GB/s per socket. That is 136GB/s in a dual socket. But, what I really came here to say, is that POWER8 has nearly twice the single thread memory bandwidth available. From some of my tests a single thread can access ~40GB/sec of bandwidth vs the intel at ~20GB/sec. But this doesn't translate into clear cut wins even on codes which at first pass appear to be bandwidth limited. Mostly because the processor cores themselves are about 1/2 the speed in tight loopy code (think nbench) running out of cache. So, in our in house benchmarks the POWER ran anywhere from 75% as fast to roughly as fast. Only on a few tests did it actually manage to best the E5 xeon we were testing it against, and then never by more than 150%. Plus, the dynamic thread controls are cool, but beyond 4 threads/core I could never actually improve performance, and beyond 2x it was major diminishing returns.
  - userbinator 12y ago
    the processor cores themselves are about 1/2 the speed in tight loopy code (think nbench) running out of cache. I wonder where the bottleneck is there, since on paper the POWER8 specs beat Haswell: it's a 10-issue core, 224-entry reorder buffer vs. a 4-issue core with 192-entry reorder buffer for Haswell. But if it takes several POWER instructions to do the work of one x86 instruction, or icache bandwidth is insufficient, the advantage goes the other way.
  - m_mueller 12y ago
    Thanks for your input. I have yet to directly compare the two systems for the code I'm working on (weather prediction), something I'm looking forward to. Since the dynamical processes are heavily memory bandwidth bound (stencil codes) I assumed that Power8 should beat Xeon handily, but I'll have to see I guess.
lvalsan 12y ago
Indeed, in the past, on POWER architectures the OS was running inside PowerVM. That's not the case any longer with OpenPOWER and Linux is running on the bare metal.

by kev009 12y ago

m_mueller 12y ago

The one big advantage of power architecture is memory bandwidth. Power8 has 230GB/s sustained on a 12 core chip while Xeon Haswell-EP has 102GB/s on 12 core. This means you need more parallelism (=harder to program) and more sockets to get the same speed on Xeons vs. Power8 for memory-bandwidth-limited problems (which >50% of HPC applications are). Power8 has I believe ~50% higher TDP than Xeons, so even in performance per watt for memory bandwidth limited problems, you are still ahead, especially considering all the components you need to power besides the socket. TLDR; These things are for a specific segment of HPC, albeit the one that doesn't care too much about Linpack performance. [1] http://en.wikipedia.org/wiki/POWER8 http://en.wikipedia.org/wiki/POWER8 [2] http://www.pugetsystems.com/blog/2014/09/08/Memory-Performance-for-Intel-Xeon-Haswell-EP-DDR4-596/ http://www.pugetsystems.com/blog/2014/09/08/Memory-Performan...

kev009 12y ago
No doubt HPC is a big influence on design since they can sink big gov contracts, but IBM is claiming it's a serious competitor to scale out x86. They've done this in the past.. PowerPC (6xx), PReP, CHRP, PowerLinux, now OpenPOWER. It looks too similar this time where they cannot get things into gear lowering margins to gain volume. The Tyan box is basically a reference platform.. over priced and poorly designed (2U, 3.5" disks, several thousand dollars?!). Crossing my fingers something good will happen, but placing little stock in it.
StillBored 12y ago
You should be careful on the memory bandwidth. The more recent xeons (2011v3, ddr4 2133) are rated 68 GB/s per socket. That is 136GB/s in a dual socket. But, what I really came here to say, is that POWER8 has nearly twice the single thread memory bandwidth available. From some of my tests a single thread can access ~40GB/sec of bandwidth vs the intel at ~20GB/sec. But this doesn't translate into clear cut wins even on codes which at first pass appear to be bandwidth limited. Mostly because the processor cores themselves are about 1/2 the speed in tight loopy code (think nbench) running out of cache. So, in our in house benchmarks the POWER ran anywhere from 75% as fast to roughly as fast. Only on a few tests did it actually manage to best the E5 xeon we were testing it against, and then never by more than 150%. Plus, the dynamic thread controls are cool, but beyond 4 threads/core I could never actually improve performance, and beyond 2x it was major diminishing returns.
- userbinator 12y ago
  the processor cores themselves are about 1/2 the speed in tight loopy code (think nbench) running out of cache. I wonder where the bottleneck is there, since on paper the POWER8 specs beat Haswell: it's a 10-issue core, 224-entry reorder buffer vs. a 4-issue core with 192-entry reorder buffer for Haswell. But if it takes several POWER instructions to do the work of one x86 instruction, or icache bandwidth is insufficient, the advantage goes the other way.
- m_mueller 12y ago
  Thanks for your input. I have yet to directly compare the two systems for the code I'm working on (weather prediction), something I'm looking forward to. Since the dynamical processes are heavily memory bandwidth bound (stencil codes) I assumed that Power8 should beat Xeon handily, but I'll have to see I guess.

lvalsan 12y ago

Indeed, in the past, on POWER architectures the OS was running inside PowerVM. That's not the case any longer with OpenPOWER and Linux is running on the bare metal.