5 ms·

This is an interesting take. If you look at the current top 10 supercomputers, half of them have GPUs. As as outsider, I've assumed this recent move of adding m

by pavon 6y ago

This is an interesting take. If you look at the current top 10 supercomputers, half of them have GPUs. As as outsider, I've assumed this recent move of adding matrix support and wider SIMD to HPC CPUs as a means to help with the bandwidth problem, not ignore it, by decreasing the need to shuttle data between the CPU and GPU. I wonder what I'm missing.

rbanffy 6y ago
I was very impressed by the performance numbers quoted in the paper for the POWER10.
- Katydid 6y ago
  Same here but that architecture is getting far less attention than Epyc right now. Seems a bit of a shame.
  - dragontamer 6y ago
    EPYC is actually out, so people can actually play with it. POWER9 made a big splash with Summit, and POWER9 is no bandwidth slouch either. POWER9 also had Talos II so that "normies" could actually get a machine to play with under $10k. Raptor Computing (makers of Talos) haven't said anything about POWER10 support yet. So its hard to get excited about a product that seems impossible to use.
    - rbanffy 6y ago
      With AMD pressuring Intel, I fear Raptor will be unable to make a price/performance-competitive POWER10 box the same way they did with POWER9. IBM needs to understand that if there's no easy onboarding to a platform, it becomes a legacy one, as new things are built elsewhere.
      - tachyonbeam 6y ago
        > IBM needs to understand that if there's no easy onboarding to a platform, it becomes a legacy one, as new things are built elsewhere. You're absolutely right. 10K is a huge amount of money to spend on a computer, even a workstation. Most startups can't afford that, so they will never run POWER hardware, and the startups of today are the big corporations of tomorrow. POWER will likely disappear just like "minicomputers" did. This makes me think of Apple making their ecosystem increasingly more locked in and restrictive. The result is that Apple devices become increasingly less fun for developers to play with. The end result is that more developers will migrate towards Linux. These people will then be less likely to develop software for Apple.
        rbanffy 6y ago
        > POWER will likely disappear just like "minicomputers" did. They'll stay with the IBM i, but there isn't much AIX can do that Linux on Xeon can't do as well as. AIX is not like their mainframe business (and even there, they have surrendered to Linux). > These people will then be less likely to develop software for Apple. They still have a pretty good UX for developers and entry-level machines. I'll probably replace my aging Mac Mini with an ARM-based one next year or so. What will really upset me is if MacPorts is no longer supported on ARM. I hope Apple is sponsoring someone ensuring key parts of the 3rd party ecosystem are there or Macs will become second-class developer machines.
dragontamer 6y ago
GPUs simply have more bandwidth than CPUs. GPUs have huge latencies on VRAM, but are fed by GDDR6 or HBM2 at 1TBps at relatively cheap prices. There's no comparison with bandwidth. NVidia NVLink is 300GBps GPU-to-GPU communications. In contrast, AMD EPYC is 50GBps CCX-to-CCX communications. Once again, there's no comparison with bandwidth, the GPUs simply win. GPU-to-GPU is wider than CPU-to-CPU. VRAM-to-GPU is wider than RAM-to-CPU. ------- If you have a problem that cares more about bandwidth than latency, then GPUs are simply the best architecture. Period. The pure compute performance is paired with absurdly huge bandwidth numbers. Most consumer problems are latency-bound. HPC however, is specifically programmed to be bandwidth-bound instead. It takes effort and optimization to get there however.
- tachyonbeam 6y ago
  > Most consumer problems are latency-bound. Is that even true? I'd argue a lot of what happens on consumer devices can (or could) be done in parallel. There's some need for a fast response, but things like, responding to a button click and updating some internal state, take a trivial amount of computation to implement. Updating your UI in response to that button click, that's again something that can be parallelized.
  - dragontamer 6y ago
    Think of the following line of code: blah = blah->next A simple linked-list traversal: 50-nanoseconds on a typical consumer CPU. This could represent a DOM-tree traversal. Or it could be a Javascript object being allocated. Or it could be the text getting added to the end of your rope-data structure. Or it could be a binary-search (finding a new address to check next). Or a malloc() (which traverses a linked-list to find memory). This dereference happens very often. 50-nanoseconds to hit main-RAM, 10-nanoseconds if L3 cache, 4-nanoseconds if in L2 cache, 1-nanosecond if in L1 cache. This sort of operation is very fast on CPUs, especially if you're cached. Heck, your CPU won't even wait on the result: its going to branch-predict the answer and start executing the next bit of code before it even knows its correct. ---------- That same memory-dereference is 300-nanoseconds from GPU VRAM, and maybe 100-nanoseconds if you're in GPU L1 cache. If you got a tree, its probably going to be far faster to traverse on a CPU. Where the GPU wins is Raytracing: you can have millions of individual tasks traversing the same tree, in parallel, at larger overall bandwidths than a CPU ever could. IE: GPUs can perform 1000 binary searches faster than a CPU can perform 1000 binary searches. But how many applications need 1000 binary searches? Usually, you need 1 binary search, and once you get the result, you need to do something else. (And that result can be branch predicted for even further accelerations to speed).
stonogo 6y ago
It's almost a tautology -- I know several recent supercomputers that have GPUs not because of mission drive, but to 'inflate' the LINPACK results to satisfy the sponsor's demands for performance numbers. It's hoped the users will adopt the GPUs, but that doesn't always happen; see the discontinued Xeon PHI accelerators for an example of that. When that happens, you've got a lot of idle GPUs...
- llukas 6y ago
  GPU != Xeon Phi
  - stonogo 6y ago
    A distinction without a difference, in this case.
    - llukas 6y ago
      One had half-assed support and one didn't. Also one wasn't undermined by Xeon CPU division.
jasonwatkinspdx 6y ago
Some context on the machine most of these authors worked on would probably help. Fugaku has two really important features: Stacks of HBM on die, and the Scalable Vector Extensions. HBM is used by gpus as well. This gives A64FX comparable raw bandwidth even if the programming model is different. The SVE design is more general purpose than other short vector extensions. It abstracts over lane width / number of lanes at the binary level, and has some bypassing features similar to the old Cray machines that again have the effect of delivering massive bandwidth. The result of this is that A64FX is competitive with GPUs on most HPC workloads, but with a single chip, single programming architecture, and lower overall power consumption. So I'd interpret the author's comments in that context. It's not so much about adding SIMD as a bad idea, but the specifics of how that needs to be implemented, and whether a dedicated coprocessor with a wholly different programming model makes sense vs well designed vector extensions backed by a high bandwidth memory subsystem.

by pavon 6y ago

rbanffy 6y ago

I was very impressed by the performance numbers quoted in the paper for the POWER10.

Katydid 6y ago
Same here but that architecture is getting far less attention than Epyc right now. Seems a bit of a shame.
- dragontamer 6y ago
  EPYC is actually out, so people can actually play with it. POWER9 made a big splash with Summit, and POWER9 is no bandwidth slouch either. POWER9 also had Talos II so that "normies" could actually get a machine to play with under $10k. Raptor Computing (makers of Talos) haven't said anything about POWER10 support yet. So its hard to get excited about a product that seems impossible to use.
  - rbanffy 6y ago
    With AMD pressuring Intel, I fear Raptor will be unable to make a price/performance-competitive POWER10 box the same way they did with POWER9. IBM needs to understand that if there's no easy onboarding to a platform, it becomes a legacy one, as new things are built elsewhere.
    - tachyonbeam 6y ago
      > IBM needs to understand that if there's no easy onboarding to a platform, it becomes a legacy one, as new things are built elsewhere. You're absolutely right. 10K is a huge amount of money to spend on a computer, even a workstation. Most startups can't afford that, so they will never run POWER hardware, and the startups of today are the big corporations of tomorrow. POWER will likely disappear just like "minicomputers" did. This makes me think of Apple making their ecosystem increasingly more locked in and restrictive. The result is that Apple devices become increasingly less fun for developers to play with. The end result is that more developers will migrate towards Linux. These people will then be less likely to develop software for Apple.
      - rbanffy 6y ago
        > POWER will likely disappear just like "minicomputers" did. They'll stay with the IBM i, but there isn't much AIX can do that Linux on Xeon can't do as well as. AIX is not like their mainframe business (and even there, they have surrendered to Linux). > These people will then be less likely to develop software for Apple. They still have a pretty good UX for developers and entry-level machines. I'll probably replace my aging Mac Mini with an ARM-based one next year or so. What will really upset me is if MacPorts is no longer supported on ARM. I hope Apple is sponsoring someone ensuring key parts of the 3rd party ecosystem are there or Macs will become second-class developer machines.

dragontamer 6y ago

GPUs simply have more bandwidth than CPUs. GPUs have huge latencies on VRAM, but are fed by GDDR6 or HBM2 at 1TBps at relatively cheap prices. There's no comparison with bandwidth. NVidia NVLink is 300GBps GPU-to-GPU communications. In contrast, AMD EPYC is 50GBps CCX-to-CCX communications. Once again, there's no comparison with bandwidth, the GPUs simply win. GPU-to-GPU is wider than CPU-to-CPU. VRAM-to-GPU is wider than RAM-to-CPU. ------- If you have a problem that cares more about bandwidth than latency, then GPUs are simply the best architecture. Period. The pure compute performance is paired with absurdly huge bandwidth numbers. Most consumer problems are latency-bound. HPC however, is specifically programmed to be bandwidth-bound instead. It takes effort and optimization to get there however.

tachyonbeam 6y ago
> Most consumer problems are latency-bound. Is that even true? I'd argue a lot of what happens on consumer devices can (or could) be done in parallel. There's some need for a fast response, but things like, responding to a button click and updating some internal state, take a trivial amount of computation to implement. Updating your UI in response to that button click, that's again something that can be parallelized.
- dragontamer 6y ago
  Think of the following line of code: blah = blah->next A simple linked-list traversal: 50-nanoseconds on a typical consumer CPU. This could represent a DOM-tree traversal. Or it could be a Javascript object being allocated. Or it could be the text getting added to the end of your rope-data structure. Or it could be a binary-search (finding a new address to check next). Or a malloc() (which traverses a linked-list to find memory). This dereference happens very often. 50-nanoseconds to hit main-RAM, 10-nanoseconds if L3 cache, 4-nanoseconds if in L2 cache, 1-nanosecond if in L1 cache. This sort of operation is very fast on CPUs, especially if you're cached. Heck, your CPU won't even wait on the result: its going to branch-predict the answer and start executing the next bit of code before it even knows its correct. ---------- That same memory-dereference is 300-nanoseconds from GPU VRAM, and maybe 100-nanoseconds if you're in GPU L1 cache. If you got a tree, its probably going to be far faster to traverse on a CPU. Where the GPU wins is Raytracing: you can have millions of individual tasks traversing the same tree, in parallel, at larger overall bandwidths than a CPU ever could. IE: GPUs can perform 1000 binary searches faster than a CPU can perform 1000 binary searches. But how many applications need 1000 binary searches? Usually, you need 1 binary search, and once you get the result, you need to do something else. (And that result can be branch predicted for even further accelerations to speed).

stonogo 6y ago

It's almost a tautology -- I know several recent supercomputers that have GPUs not because of mission drive, but to 'inflate' the LINPACK results to satisfy the sponsor's demands for performance numbers. It's hoped the users will adopt the GPUs, but that doesn't always happen; see the discontinued Xeon PHI accelerators for an example of that. When that happens, you've got a lot of idle GPUs...

llukas 6y ago
GPU != Xeon Phi
- stonogo 6y ago
  A distinction without a difference, in this case.
  - llukas 6y ago
    One had half-assed support and one didn't. Also one wasn't undermined by Xeon CPU division.

jasonwatkinspdx 6y ago

Some context on the machine most of these authors worked on would probably help. Fugaku has two really important features: Stacks of HBM on die, and the Scalable Vector Extensions. HBM is used by gpus as well. This gives A64FX comparable raw bandwidth even if the programming model is different. The SVE design is more general purpose than other short vector extensions. It abstracts over lane width / number of lanes at the binary level, and has some bypassing features similar to the old Cray machines that again have the effect of delivering massive bandwidth. The result of this is that A64FX is competitive with GPUs on most HPC workloads, but with a single chip, single programming architecture, and lower overall power consumption. So I'd interpret the author's comments in that context. It's not so much about adding SIMD as a bad idea, but the specifics of how that needs to be implemented, and whether a dedicated coprocessor with a wholly different programming model makes sense vs well designed vector extensions backed by a high bandwidth memory subsystem.