7 ms·
There's some great info in these slides: https://regmedia.co.uk/2020/08/17/ibm_power10_summary.pdf https://regmedia.co.uk/2020/08/17/ibm_power10_summary.pdf -
by jiggawatts 6y ago
There's some great info in these slides: https://regmedia.co.uk/2020/08/17/ibm_power10_summary.pdf https://regmedia.co.uk/2020/08/17/ibm_power10_summary.pdf
- They leapfrogged everyone else with PCIe v5 and DDR5
- 1 TB/s memory bandwidth, which is comparable to high-end NVIDIA GPUs, but for CPUs
- Socket-to-socket interconnect is 1 TB/s also.
- 120 GB/s/core L3 cache read rate sustained.
- Floating point rate comparable to GPUs
- 8-way SMT makes this into a hybrid between a CPU and a GPU in terms of the latency hiding and memory management, but programmable exactly like a full CPU, without the limitations of a GPU.
- Memory disaggregation similar to how most modern enterprise architectures separate disk from compute. You can have memory-less compute nodes talking to a central memory node!
- 16-socket glueless servers
- Has instructions for accelerating gzip.
- baybal2 6y agoIt leaves me salivating
- gnufx 6y ago> - Floating point rate comparable to GPUs Where do you get the FP performance, exactly, and for what value of "FP"? It's unclear to me in the slides from El Reg, which appear to be about the MMA specifically, and it's not clear what the SIMD units actually are. (I don't know if that's specified by the ISA.) One thing is that it presumably has a better chance of keeping the SIMD fed than some.
- predicating 6y agoWill they still be ahead when this actually releases in 2H21?
- burtonator 6y ago> - Has instructions for accelerating gzip. Has there been any research for finding out how much these custom instructions have accelerated modern computational power in practice? There are a lot of these instructions in modern CPUs for AES, gzip, SSL, etc but you have to have a library coded to support them. I'm curious what the performance improvements look like in practice.
- jiggawatts 6y agoGlad you asked! I recently did a bunch of tests to see what the "ultimate bottlenecks" are for basic web applications. Think latency to the database and AES256 throughput. Some rough numbers: - Local latency to SQL Server from ASP.NET is about 150 μs, or about 6000 synchronous queries per second, max. - Even with SR-IOV and Mellanox adapters, that rises to 250 μs if a physical network hop is involved. - Typical networks have a latency floor around 500-600 μs, and it's not uncommon to see 1.3 ms VM-to-VM. Now we're down to 800 queries per second! - Similarly, older CPUs struggle to exceed 250 MB/s/core (2 Gbps) for AES256, which is the fundamental limit to HTTPS throughput for a single client. - Newer CPUs, e.g.: AMD EPYC or any recent Intel Xeon can do about 1 GB/s/core, but I haven't seen any CPUs that significantly exceed that. That's not even 10 Gbps. If you have a high-spec cloud VM with 40 or 50 Gbps NICs, there is no way a single HTTPS stream can saturate that link. You have to parallelise somehow to get the full throughout (or drop encryption.) - HTTPS accelerators such as F5 BIG IP or Citrix ADC (NetScaler) are actually HTTPS decelerators for individual users, because even hardware models with SSL offload cards can't keep up with the 1 GB/s from a modern CPU. Their SSL cards are designed for improving the aggregate bandwidth of hundreds of simultaneous streams, and don't do well at all for a single stream, or even a couple of concurrent streams. This matters when "end-to-end encryption" is mandated, because back end connections are often pooled. So you end up with N users being muxed onto just one back-end connection which then becomes the bottleneck.
- mjb 6y ago> Newer CPUs, e.g.: AMD EPYC or any recent Intel Xeon can do about 1 GB/s/core, but I haven't seen any CPUs that significantly exceed that. Modern processors can do ~2.1GB/s per core for AES-256-GCM (about 2x what they can do for AES-CBC).
- dragontamer 6y agoIt should be noted that AES-CBC doesn't have any instruction-level parallelism available. Both EPYC and Xeons have 2-AES units per core now. But CBC can only effectively use one of them at a time. (Block(n+1) cannot be computed until block(n) is done computing. Because Block(n) is used as input into Block(n+1) in CBC mode). AES-256-GCM can compute block(n) and block(n+1) simultaneously. So you need to use such a parallel algorithm if you actually want to use the 2x AES pipelines on EPYC or Xeon.
- caddie 6y agoDoes this mean GPUs might become less relevant in the future if CPUs can perform similar tasks at same speed?
- visarga 6y agoThey might be focusing on efficient inference, that's the main use case in ML.
- dragontamer 6y agoCPU-SIMD is less about competing against GPUs and more about latency. GPUs will always have more GFlops and memory bandwidth at a lower cost. They're specifically built GFlop and memory-bandwidth machines. Case in point: the NVidia 2070 Super is 8 TFlops of compute at $400, a tiny fraction of what this POWER10 will cost. If POWER10 costs anything like POWER9, we're looking at well over $2000 for the bigger chips and $1000 for reasonable multisocket motherboards. And holy moly: 602mm^2 at 7nm is going to be EXPENSIVE. EDIT: I'm only calculating ~2 TFlops from the hypothetical 60x SMT4 Power10 at 4GHz. That's no where close to GPU-level Flops. However, the CPU-GPU link is slow in comparison to CPU-L1 cache (or even CPU-DDR4 / DDR5). A CPU can "win the race" by using SIMD onboard, completing your task before it even spent the ~5-microseconds needed to communicate to the GPU. ---------- With that being said: POWER10 also implements PCIe 5.0, which means it will be one of the fastest processors for communicating with future GPUs.
- jedbrown 6y agoIt's more appropriate to compare pricing of Tesla with a datacenter-grade CPU like POWER10 (or Epyc/Xeon/etc.). A64FX (in Fugaku, the current #1 machine on all popular supercomputing benchmarks) has shown that CPUs can compete with top-shelf GPUs on bandwidth and floating point energy efficiency.
- dragontamer 6y agoFugaku has 158,976 nodes x2 chips each, or 317,952 A64FX chips. Summit has 4,608 nodes x 6 GPUs each, or 27,648 V100 GPUs. It also was built back in 2018. --------- While Fugaku is certainly an interesting design, it seems inevitable that a modern GPU (say A100 Amperes) would crush it in FLOPs. Really, Fugaku's most interesting point is its high rate of HPCG, showing that its interconnect is hugely efficient. Per-node, Fugaku is weaker. They built an amazing interconnect to compensate for that weakness. Fugaku also is an HBM-based computer, meaning you cannot easily add or remove RAM (like a CPU / GPU team can configure to more, or less RAM by adding sticks). These are the little differences that make a difference in practicality. But yes, A64FX is certainly an accomplishment, but I wouldn't go so far as to say its proven that CPUs can keep up with GPUs in terms of raw FLOPs.
- dragontamer 6y agoI appreciate the slides, but you've got some critical errors. > - Floating point rate comparable to GPUs No where close. POWER10 caps out at 60x SMT4, and only has 2x 128-bit SIMD units per SMT4. That's 480 FLOPs per clock cycle. At 4GHz, that's only 1.9 TFlops single-precision compute. An NVidia 2070 Super ($400 consumer GPU) hits 8.2 TFlops with 448 GB/s bandwidth. > - 8-way SMT makes this into a hybrid between a CPU and a GPU in terms of the latency hiding and memory management, but programmable exactly like a full CPU, without the limitations of a GPU. Note that 8-way SMT is the "big core", and I'm almost convinced that 8-SMT is more about licensing than actually scaling. By supporting 8 threads per core (and doubling the size of the core), you get a 30-core system that supports 240-threads. That's a licensing hack: since most enterprise software is paid per-core. I'd expect that 4-SMT will be more popular with consumers (ie: Talos II sized systems). Similar to how Power9 was
- twoodfin 6y agoNote that 8-way SMT is the "big core", and I'm almost convinced that 8-SMT is more about licensing than actually scaling. By supporting 8 threads per core (and doubling the size of the core), you get a 30-core system that supports 240-threads. That's a licensing hack: since most enterprise software is paid per-core. I don't think that's entirely fair: Massively multi-user transactional applications (read: databases) are right in the wheelhouse of POWER, and they're exactly the kind of applications that benefit most from SMT. Lots of opportunities for latency hiding as you're chasing pointers from indexes to data blocks.
- dragontamer 6y agoWhile that's a fair point, databases are also among the most costly software that is paid per-core. So the SMT8 system will have half the per-core costs as an SMT4 system (because an SMT8 system will have half the "cores" of any SMT4 system. Even though an SMT8 core is just two SMT4 cores grafted together).
- staticassertion 6y ago> That's a licensing hack: since most enterprise software is paid per-core. Shit. That's actually awesome and could easily pay for itself.
- jhallenworld 6y agoHow is the single thread performance? Intel Xeon was always faster (thanks to competition from AMD Opteron...), But maybe that's changed with Intel's recent problems...
- dragontamer 6y agoWhen Phoronix tested POWER9 SMT4 a while back, single-thread performance seems disappointing at first glance. But it seems to be made up with their implementation of SMT4. The 2nd thread on a core didn't have much slowdown at all, while thread3 and thread4 per core barely affected performance. It seems like POWER9 at least, benefits from running significantly more threads per core (at least compared to Xeon or AMD). EDIT: It should be noted that IBM's 128-bit vector units are downright terrible compared to Intel's 512-bit or AMD's 256-bit vector units. SIMD compute is the weakest point of the Power9, and probably the Power10. They'll be the worst at multimedia performance (or other code using SIMD units: Raytracing, graphics, etc. etc.). Power9's best use case was highly-threaded 64-bit code without SIMD. Power10 looks like the SIMD units are improving, but they're still grossly undersized compared to AMD or Intel SIMD units.
- omikun 6y agoSounds like IBM is not wasting area and power on out of order scheduling to find independent instructions within one thread. If you're running a lot of threads anyway, you get more independent instructions to work with for free!
- dragontamer 6y agoWhen in SMT4 mode, various hardware resources are "partitioned off" in Power9. The first, and third, threads use the "Left Superslice", while the second and fourth threads use the "Right Superslice". All four threads share a decoder (Bulldozer style). 1/4th of the branch predictor (EAT) is given to each of the 4x threads per core. Register rename buffer is shared 2-threads at a time. (Two threads use the "left superslice", two other threads use the "right superslice"). An SMT1 mode, the single thread can use all 4 resources simultaneously. A lot of the out-of-order stuff looks like it'd work as expected in 1-thread to 4-thread modes. At least, looking through the Power9 user guide / in theory. -------- Honestly, I think the weirdest thing about POWER9 is the 2-cycle minimum latency (even on simple instructions like ADD and XOR). With that kind of latency, I bet that a number of inner-loops and code needs 2-threads loaded on the core, just to stay fully fed. That'd be my theory for why 2-threads seem to be needed before POWER9 cores feel like they're being utilized well. Obviously, POWER10 probably will change some of these details. But I'd expect POWER10 to largely be the same as POWER9 (aside from being bigger, faster, more efficient).
- stjohnswarts 6y agoGood to see they're keeping pace. For some reason I keep cheering for the old beast but I don't see how they can keep up with the big 5
- ponker 6y agoWho are the Big 5? Googling turns up no definitive list.
- robarr 6y ago“GAFAM A more inclusive grouping referred to as GAFAM or "Big Five", defines Google, Amazon, Facebook, Apple, and Microsoft as the tech giants.[18][19][20][21] Besides Saudi Aramco, the GAFAM companies are the five most valuable public corporations in the world as measured by market capitalization.[3] Nikos Smyrnaios justified the GAFAM grouping as an oligopoly that appears to take control of the Internet by concentrating market power, financial power and using patent rights and copyright within a context of capitalism.[22]“ https://en.m.wikipedia.org/wiki/Big_Tech https://en.m.wikipedia.org/wiki/Big_Tech
- ponker 6y agoThose wouldn't be IBM's top competitors in semiconductors would they? I don't think Facebook or Amazon or Microsoft is spending more than 1% of their R&D budget on semiconductors.
- ksec 6y agoI am excited. But also sad that most of us wont ever get to play with it. Unlike Intel and AMD's Xeon and EPYC, getting hold of POWER doesn't seems like an easy task unless the cooperation you work for have very specific needs. Edit: Turns out there is a major section on it below.