12 ms·

How does POWER8 compare to x86, e.g. Haswell? Just skimming some of the architecture details... * 4x hardware threads per core (8-way SMT vs. 2) * 1/4th FP th

How does POWER8 compare to x86, e.g. Haswell? Just skimming some of the architecture details... * 4x hardware threads per core (8-way SMT vs. 2) * 1/4th FP throughput per core (8 SP flops/cycle vs. 32) * 3x bandwidth to RAM (230 GB/s vs. 68) [edit: updated for Haswell-EP] https://en.wikipedia.org/wiki/POWER8#Specifications https://en.wikipedia.org/wiki/POWER8#Specifications http://www.redbooks.ibm.com/abstracts/tips1153.html http://www.redbooks.ibm.com/abstracts/tips1153.html What is it good for?

jacques_chester 12y ago
> What is it good for? Running very demanding single-system software. It's the sort of box you'd install DB2 or Oracle on, especially if you've already tied your business to IBM solutions in the past.
- PhuFighter 12y ago
  It depends on what you mean by "demanding". I think that single threaded applications on x86 (with nothing else running) will smoke the Power8 system. But if you have lots of threads and high levels of concurrency going on, i think that the results will be very much workload dependent.
Marat_Dukhan 12y ago
+ Hardware Transactional Memory that isn't buggy
benjarrell 12y ago
Based on experience with POWER7, emptying your wallet.
- PhuFighter 12y ago
  it's surprising how much cheaper these Power8 boxes are compared to the Power7 boxes. You can get a tyan 2U, 1 socket system for $2800 FOB HK: http://www.tyan.com/campaign/openpower/ http://www.tyan.com/campaign/openpower/
- deleted 12y ago
  [deleted]
jedbrown 12y ago
> 9x bandwidth to RAM (230 GB/s vs. 25.6) Haswell-EP supports 4 DDR4-2133 channels, or 68 GB/s (theoretical). TDP on POWER8 is almost double and it's much more expensive, so we should think about it as having perhaps double the DRAM bandwidth (still excellent). Note that POWER8 gets this bandwidth through many more channels, which favors the use of many memory streams. This has been the case historically, with POWER favoring data structures that result in many streams, while the same transformation has been catastrophic for memory performance on Blue Gene's PPC, with its limited prefetch engine and small number of outstanding memory requests.
- rdtsc 12y ago
  That is interesting. RAM bandwidth could significantly impact a lot of applications. I guess the only ways it to measure and compare. The problem I see with power is it has a different endiannes and even though compilers know how to handle it, a lot of libraries and code might just assume some specific ordering (little endian) and thus fail un-expectedly on POWER.
  - wmf 12y ago
    That's why Power is now little-endian.
    - gsnedders 12y ago
      As far as I'm aware, IBM intend on maintaining both big and little endian Linux ports (and toolchains).
    - PhuFighter 12y ago
      Power has been little endian for a while. No one bothered to tell the software guys to write a little endian OS and tools for it....
      - wmf 12y ago
        My point is that little-endian Ubuntu is now available.
apaprocki 12y ago
Worth mentioning that POWER is the only chip I'm aware of that has hardware DFP instructions.
- desdiv 12y ago
  In case anyone else is confused: DFP = Decimal Floating Point, i.e. native base 10 arithmetics on the FPU. Googling "DFP instructions" led me to nothing but DoubleClick links.
- polack 12y ago
  Sounds interesting! What's needed from the user in order to use this feature? Will it be possible to use from high level languages like java or do you have to use assembly? Can think of quite a lot of applications that would benefit A LOT from this.
  - desdiv 12y ago
    High level languages like Python, Ruby, and Go could take advantage of this without having to make any code changes, provided that the appropriate libraries are updated to use this new hardware. Java, however, is a special case. Since Java bytecode does not have a BigDecimal type, there's no way to update the JVM to take advantages of this hardware unfortunately.
    - krylon 12y ago
      I think I remember reading that zSeries has Decimal Floating Point, too, and that IBM's JVM does take advantage of the instructions on z/OS. Of course, even if that is true, it is a different architecture. But still, if they made the change on one architecture, it would be silly not to make it on the other one as well. Unless that was intentional to keep some sort of advantage for zSeries.
    - wmf 12y ago
      I would assume that the IBM JVM uses DFP instructions to implement the BigDecimal class.
nkurz 12y ago
IBM has solid documentation of more of the Power8 details here: http://www.setphaserstostun.org/power8/POWER8_UM_external_22APR2014_pub.pdf http://www.setphaserstostun.org/power8/POWER8_UM_external_22... Going only by specs, it seems that in many other cases each Power8 core has about 2-4x the resources of a Haswell core: 64KB vs 32KB L1D 512KB vs 64KB L2 8MB vs 2.5MB L3 16 vs 10(?) outstanding L1 requests 8 vs 4 instructions issued per cycle 2 vs 1 stores per cycle (or 4 vs 2 loads if no stores) 3 vs 4/5 cycle L1 latency for 64-bit data 5 vs 6/7 cycle L1 latency for vector data 2048 vs 512 TLB entries (integrated with huge pages) Specs that aren't higher than Haswell are usually the same --- skimming, I haven't found any that are lower. This makes me think that a 4xSMT or 8xSMT Power8 core will probably be about equivalent to a 2xSMT (hyperthreaded) Haswell core. Dedicated core operation (non-hyperthreaded) will vary more based on workload, but likely be 1x to 2x in favor of Power8. But these are guesses based on written specs -- I'm eager to see more real benchmarks.
- justincormack 12y ago
  Your huge page comment also reminded me that Power has 64k page support, rather than the 4k or 2MB that amd64 has, and this is normally the pag size (eg in RHEL) which reduces TLB pressure a lot.
  - apaprocki 12y ago
    In AIX the text/data page size can be controlled independently per-process with environment variables. What drives me up a wall is that the per-process setting does not actually flow through to their POSIX layer. Have any code that calls sysconf(_SC_PAGESIZE)? The page size is 8k! No, wait...
apaprocki 12y ago
Curious aside -- is there any blog / article out there where someone has tried to max out network bandwidth on an x86 box? I checked a POWER8 we have today and it has 14 PCIe3 slots (mix of 8x/16x), so theoretically one could load up 14 40GbE NICs on it. I'd be interested to see some kind of I/O shootout to see where both systems hit a wall. e.g. Snabb Switch 20x10GbE on x86 https://lukego.github.io/blog/2013/06/23/echoing-packets-with-snabb-switch/ https://lukego.github.io/blog/2013/06/23/echoing-packets-wit...
- wmf 12y ago
  I have some x86 servers with seven slots and 14 10G ports, but we only managed to get 8 ports working in netmap. Intel has done some crazy stuff with DPDK and I've seen a demo of over 300 Gbps on a 4S machine. I've also seen some stuff on Power8 networking but it's not public.

jacques_chester 12y ago

> What is it good for? Running very demanding single-system software. It's the sort of box you'd install DB2 or Oracle on, especially if you've already tied your business to IBM solutions in the past.

PhuFighter 12y ago
It depends on what you mean by "demanding". I think that single threaded applications on x86 (with nothing else running) will smoke the Power8 system. But if you have lots of threads and high levels of concurrency going on, i think that the results will be very much workload dependent.

Marat_Dukhan 12y ago

+ Hardware Transactional Memory that isn't buggy

benjarrell 12y ago

Based on experience with POWER7, emptying your wallet.

PhuFighter 12y ago
it's surprising how much cheaper these Power8 boxes are compared to the Power7 boxes. You can get a tyan 2U, 1 socket system for $2800 FOB HK: http://www.tyan.com/campaign/openpower/ http://www.tyan.com/campaign/openpower/
deleted 12y ago
[deleted]

jedbrown 12y ago

> 9x bandwidth to RAM (230 GB/s vs. 25.6) Haswell-EP supports 4 DDR4-2133 channels, or 68 GB/s (theoretical). TDP on POWER8 is almost double and it's much more expensive, so we should think about it as having perhaps double the DRAM bandwidth (still excellent). Note that POWER8 gets this bandwidth through many more channels, which favors the use of many memory streams. This has been the case historically, with POWER favoring data structures that result in many streams, while the same transformation has been catastrophic for memory performance on Blue Gene's PPC, with its limited prefetch engine and small number of outstanding memory requests.

rdtsc 12y ago
That is interesting. RAM bandwidth could significantly impact a lot of applications. I guess the only ways it to measure and compare. The problem I see with power is it has a different endiannes and even though compilers know how to handle it, a lot of libraries and code might just assume some specific ordering (little endian) and thus fail un-expectedly on POWER.
- wmf 12y ago
  That's why Power is now little-endian.
  - gsnedders 12y ago
    As far as I'm aware, IBM intend on maintaining both big and little endian Linux ports (and toolchains).
  - PhuFighter 12y ago
    Power has been little endian for a while. No one bothered to tell the software guys to write a little endian OS and tools for it....
    - wmf 12y ago
      My point is that little-endian Ubuntu is now available.

apaprocki 12y ago

Worth mentioning that POWER is the only chip I'm aware of that has hardware DFP instructions.

desdiv 12y ago
In case anyone else is confused: DFP = Decimal Floating Point, i.e. native base 10 arithmetics on the FPU. Googling "DFP instructions" led me to nothing but DoubleClick links.
polack 12y ago
Sounds interesting! What's needed from the user in order to use this feature? Will it be possible to use from high level languages like java or do you have to use assembly? Can think of quite a lot of applications that would benefit A LOT from this.
- desdiv 12y ago
  High level languages like Python, Ruby, and Go could take advantage of this without having to make any code changes, provided that the appropriate libraries are updated to use this new hardware. Java, however, is a special case. Since Java bytecode does not have a BigDecimal type, there's no way to update the JVM to take advantages of this hardware unfortunately.
  - krylon 12y ago
    I think I remember reading that zSeries has Decimal Floating Point, too, and that IBM's JVM does take advantage of the instructions on z/OS. Of course, even if that is true, it is a different architecture. But still, if they made the change on one architecture, it would be silly not to make it on the other one as well. Unless that was intentional to keep some sort of advantage for zSeries.
  - wmf 12y ago
    I would assume that the IBM JVM uses DFP instructions to implement the BigDecimal class.

nkurz 12y ago

IBM has solid documentation of more of the Power8 details here: http://www.setphaserstostun.org/power8/POWER8_UM_external_22APR2014_pub.pdf http://www.setphaserstostun.org/power8/POWER8_UM_external_22... Going only by specs, it seems that in many other cases each Power8 core has about 2-4x the resources of a Haswell core: 64KB vs 32KB L1D 512KB vs 64KB L2 8MB vs 2.5MB L3 16 vs 10(?) outstanding L1 requests 8 vs 4 instructions issued per cycle 2 vs 1 stores per cycle (or 4 vs 2 loads if no stores) 3 vs 4/5 cycle L1 latency for 64-bit data 5 vs 6/7 cycle L1 latency for vector data 2048 vs 512 TLB entries (integrated with huge pages) Specs that aren't higher than Haswell are usually the same --- skimming, I haven't found any that are lower. This makes me think that a 4xSMT or 8xSMT Power8 core will probably be about equivalent to a 2xSMT (hyperthreaded) Haswell core. Dedicated core operation (non-hyperthreaded) will vary more based on workload, but likely be 1x to 2x in favor of Power8. But these are guesses based on written specs -- I'm eager to see more real benchmarks.

justincormack 12y ago
Your huge page comment also reminded me that Power has 64k page support, rather than the 4k or 2MB that amd64 has, and this is normally the pag size (eg in RHEL) which reduces TLB pressure a lot.
- apaprocki 12y ago
  In AIX the text/data page size can be controlled independently per-process with environment variables. What drives me up a wall is that the per-process setting does not actually flow through to their POSIX layer. Have any code that calls sysconf(_SC_PAGESIZE)? The page size is 8k! No, wait...

apaprocki 12y ago

Curious aside -- is there any blog / article out there where someone has tried to max out network bandwidth on an x86 box? I checked a POWER8 we have today and it has 14 PCIe3 slots (mix of 8x/16x), so theoretically one could load up 14 40GbE NICs on it. I'd be interested to see some kind of I/O shootout to see where both systems hit a wall. e.g. Snabb Switch 20x10GbE on x86 https://lukego.github.io/blog/2013/06/23/echoing-packets-with-snabb-switch/ https://lukego.github.io/blog/2013/06/23/echoing-packets-wit...

wmf 12y ago
I have some x86 servers with seven slots and 14 10G ports, but we only managed to get 8 ports working in netmap. Intel has done some crazy stuff with DPDK and I've seen a demo of over 300 Gbps on a 4S machine. I've also seen some stuff on Power8 networking but it's not public.