4 ms·

If I'm understanding it correctly, they're not actually using the 512 bit (ZMM) registers, because using them can cause overall system slowdown. It seems to me

If I'm understanding it correctly, they're not actually using the 512 bit (ZMM) registers, because using them can cause overall system slowdown. It seems to me they're only really useful if you're doing an AVX-512 intensive workload. And do those really exist? For something like bulk matrix multiplications, GPGPU is going to be much better, both in throughput and in operations per joule. I'm remaining to be convinced that the ecological niche occupied by SIMD is significant, let alone expanding.

chrisseaton 9y ago
> For something like bulk matrix multiplications, GPGPU is going to be much better, both in throughput and in operations per joule. But you have to get all of your matrix out onto the system bus, and over to the GPU, then start the kernel, and then copy it all the way back again, to use that. ZMM is just a register. You can operate on it immediately and stream data from memory while you do the multiply.
- equalunique 9y ago
  AMD's HSA is intended to enable memory sharing between the CPU and GPU.
justincormack 9y ago
They only cause slowdown on some models, if you pay Intel enough they wont. The advantage of AVX512 is you can use it on every core (rather than trying to work out how to partition a GPU), and it is useful for lots of general purpose computations.
- Dylan16807 9y ago
  What models don't slow down? Are you claiming the slowdown isn't actually necessary to keep the chip stable?
  - justincormack 9y ago
    compare these https://en.wikichip.org/wiki/intel/xeon_silver/4116#Frequencies https://en.wikichip.org/wiki/intel/xeon_silver/4116#Frequenc... https://en.wikichip.org/wiki/intel/xeon_gold/6154#Frequencies https://en.wikichip.org/wiki/intel/xeon_gold/6154#Frequencie... https://en.wikichip.org/wiki/intel/xeon_platinum/8180m#Frequencies https://en.wikichip.org/wiki/intel/xeon_platinum/8180m#Frequ... So platinum can run avx512 on 28 cores at 2.3GHz, while silver runs 12 cores at 1.4GHz
    - brigade 9y ago
      From your links, gold/plat has a ~15% clock speed hit when using ymm muls, then another ~20% hit when going to zmm muls. So for 512-bit wide to not slow things down on Intel's chips, you need ~30% runtime of all cores to already be in AVX.
  - jabl 9y ago
    Xeon Phi, maybe? Since the entire raison d'être of that chip is doing avx-512 computation. Or if you're a glass half empty kind of person, you'd argue that it doesn't speed up when running scalar code. :)
stagger87 9y ago
Define significant. I write software that couldn't exist without SIMD, as do all my competitors. (Note: it could exist but wouldn't offer nearly the same level of capability, i.e. it would be a different product) By specifying the last several generations of Intel hardware I can guarantee customers can run my software (AVX1 minimum) with consumer level desktops/laptops. AVX-512 hardware support hasn't reached the average consumer yet, but I am ready to take advantage of it when it does. (signal/video processing)
- mycall 9y ago
  Have you looked at RISC-V Vector proposal yet? Same person who did AVX-512 for Intel.
  - clarry 9y ago
    https://riscv.org/wp-content/uploads/2015/06/riscv-vector-workshop-june2015.pdf https://riscv.org/wp-content/uploads/2015/06/riscv-vector-wo... https://riscv.org/wp-content/uploads/2016/12/Wed0930-RISC-V-Vectors-Asanovic-UC-Berkeley-SiFive.pdf https://riscv.org/wp-content/uploads/2016/12/Wed0930-RISC-V-... EDIT: This looks really nice after skimming a little.
- raphlinus 9y ago
  I don't want to overstate it; I've written a lot of SIMD in my life too. If you've got audio processing that needs to run in a realtime thread, I can see how SIMD is appealing, because we don't really have mechanisms to achieve realtime on GPU (yet). However, for a lot of seriously heavy computation, it seems like doing it on the GPU is a win. For the cases where memory transfer back and forth to the GPU is expensive, even integrated GPU should offer significantly more computational resources than even AVX-512. One of the use cases I'm thinking of is font rendering, where I published a SIMD-heavy prototype a couple years ago, and it's blown out of the water by a newer GPU-based approach: http://pcwalton.github.io/blog/2017/02/14/pathfinder/ http://pcwalton.github.io/blog/2017/02/14/pathfinder/
steve_musk 9y ago
I work in an HPC lab (computational chemistry) and we have a hand-coded AVX-512 codepath. I can't give any specifics (because I don't know them) but I know there is a non-trivial speedup versus the standard codepath (or AVX2). However, GPUs blow it out of the water for much a lower price since we only need FP32. I think the main reason we invested time adding supports is for the Xeon Phi cards. I guess is could be worthwhile for some FP64 pipelines from a cost/performance perspective.
- m_mueller 9y ago
  If you look at Skylake-SP architecture vs. recent GPUs, the chip design at first glance doesn't seem so different anymore between these two, CPUs are just much less focused, which pays a 2x price in theoretical performance for the same die space, even using Intel's superior process technology. Now that being said, I think the GPU/SIMT model of vector computing is just much smarter. Why let me jump through all these hoops of masking and compiler optimizations if all I want is a branch and an early exit for a specific set of values? GPU schedulers and drivers make this easy to use and with somewhat predictable performance results. Furthermore (and probably more importantly), why is Intel putting this amount of compute power on a CPU without significantly upgrading memory bandwidth? A 28 core Skylake-SP using full vectorization now has 3x (!) the FLOP/Byte system balance compared to NVIDIA P100. Seriously? System balance was once an argument against GPUs, but not anymore apparently...
  - jabl 9y ago
    > Now that being said, I think the GPU/SIMT model of vector computing is just much smarter. I'm not sure. For an argument in favor of vectors, see https://riscv.org/wp-content/uploads/2015/06/riscv-vector-workshop-june2015.pdf https://riscv.org/wp-content/uploads/2015/06/riscv-vector-wo... > Why let me jump through all these hoops of masking and compiler optimizations if all I want is a branch and an early exit for a specific set of values? GPU schedulers and drivers make this easy to use and with somewhat predictable performance results. If the underlying hw is SIMD (vectors) and not SIMT anyway, as Nvidia hw apparently is, why should I have to go through the effort of rewriting my code in CUDA, and hope that some opaque driver will manage to turn that into efficient vector code? I mean, ideally I'd just like to write C/C++/Fortran/Julia/Haskell/whatever code, and the compiler would autovectorize it. > Furthermore (and probably more importantly), why is Intel putting this amount of compute power on a CPU without significantly upgrading memory bandwidth? Flops are cheap, bw expensive. But yeah, certainly the are many applications that would benefit from a much better bw/flops ratio. Then again, with the latest Teslas you have 16 GB with awesome bw, after that you're trying to feed the firehose through the PCIe straw.

chrisseaton 9y ago

> For something like bulk matrix multiplications, GPGPU is going to be much better, both in throughput and in operations per joule. But you have to get all of your matrix out onto the system bus, and over to the GPU, then start the kernel, and then copy it all the way back again, to use that. ZMM is just a register. You can operate on it immediately and stream data from memory while you do the multiply.

equalunique 9y ago
AMD's HSA is intended to enable memory sharing between the CPU and GPU.

justincormack 9y ago

They only cause slowdown on some models, if you pay Intel enough they wont. The advantage of AVX512 is you can use it on every core (rather than trying to work out how to partition a GPU), and it is useful for lots of general purpose computations.

Dylan16807 9y ago
What models don't slow down? Are you claiming the slowdown isn't actually necessary to keep the chip stable?
- justincormack 9y ago
  compare these https://en.wikichip.org/wiki/intel/xeon_silver/4116#Frequencies https://en.wikichip.org/wiki/intel/xeon_silver/4116#Frequenc... https://en.wikichip.org/wiki/intel/xeon_gold/6154#Frequencies https://en.wikichip.org/wiki/intel/xeon_gold/6154#Frequencie... https://en.wikichip.org/wiki/intel/xeon_platinum/8180m#Frequencies https://en.wikichip.org/wiki/intel/xeon_platinum/8180m#Frequ... So platinum can run avx512 on 28 cores at 2.3GHz, while silver runs 12 cores at 1.4GHz
  - brigade 9y ago
    From your links, gold/plat has a ~15% clock speed hit when using ymm muls, then another ~20% hit when going to zmm muls. So for 512-bit wide to not slow things down on Intel's chips, you need ~30% runtime of all cores to already be in AVX.
- jabl 9y ago
  Xeon Phi, maybe? Since the entire raison d'être of that chip is doing avx-512 computation. Or if you're a glass half empty kind of person, you'd argue that it doesn't speed up when running scalar code. :)

stagger87 9y ago

Define significant. I write software that couldn't exist without SIMD, as do all my competitors. (Note: it could exist but wouldn't offer nearly the same level of capability, i.e. it would be a different product) By specifying the last several generations of Intel hardware I can guarantee customers can run my software (AVX1 minimum) with consumer level desktops/laptops. AVX-512 hardware support hasn't reached the average consumer yet, but I am ready to take advantage of it when it does. (signal/video processing)

mycall 9y ago
Have you looked at RISC-V Vector proposal yet? Same person who did AVX-512 for Intel.
- clarry 9y ago
  https://riscv.org/wp-content/uploads/2015/06/riscv-vector-workshop-june2015.pdf https://riscv.org/wp-content/uploads/2015/06/riscv-vector-wo... https://riscv.org/wp-content/uploads/2016/12/Wed0930-RISC-V-Vectors-Asanovic-UC-Berkeley-SiFive.pdf https://riscv.org/wp-content/uploads/2016/12/Wed0930-RISC-V-... EDIT: This looks really nice after skimming a little.
raphlinus 9y ago
I don't want to overstate it; I've written a lot of SIMD in my life too. If you've got audio processing that needs to run in a realtime thread, I can see how SIMD is appealing, because we don't really have mechanisms to achieve realtime on GPU (yet). However, for a lot of seriously heavy computation, it seems like doing it on the GPU is a win. For the cases where memory transfer back and forth to the GPU is expensive, even integrated GPU should offer significantly more computational resources than even AVX-512. One of the use cases I'm thinking of is font rendering, where I published a SIMD-heavy prototype a couple years ago, and it's blown out of the water by a newer GPU-based approach: http://pcwalton.github.io/blog/2017/02/14/pathfinder/ http://pcwalton.github.io/blog/2017/02/14/pathfinder/

steve_musk 9y ago

I work in an HPC lab (computational chemistry) and we have a hand-coded AVX-512 codepath. I can't give any specifics (because I don't know them) but I know there is a non-trivial speedup versus the standard codepath (or AVX2). However, GPUs blow it out of the water for much a lower price since we only need FP32. I think the main reason we invested time adding supports is for the Xeon Phi cards. I guess is could be worthwhile for some FP64 pipelines from a cost/performance perspective.

m_mueller 9y ago
If you look at Skylake-SP architecture vs. recent GPUs, the chip design at first glance doesn't seem so different anymore between these two, CPUs are just much less focused, which pays a 2x price in theoretical performance for the same die space, even using Intel's superior process technology. Now that being said, I think the GPU/SIMT model of vector computing is just much smarter. Why let me jump through all these hoops of masking and compiler optimizations if all I want is a branch and an early exit for a specific set of values? GPU schedulers and drivers make this easy to use and with somewhat predictable performance results. Furthermore (and probably more importantly), why is Intel putting this amount of compute power on a CPU without significantly upgrading memory bandwidth? A 28 core Skylake-SP using full vectorization now has 3x (!) the FLOP/Byte system balance compared to NVIDIA P100. Seriously? System balance was once an argument against GPUs, but not anymore apparently...
- jabl 9y ago
  > Now that being said, I think the GPU/SIMT model of vector computing is just much smarter. I'm not sure. For an argument in favor of vectors, see https://riscv.org/wp-content/uploads/2015/06/riscv-vector-workshop-june2015.pdf https://riscv.org/wp-content/uploads/2015/06/riscv-vector-wo... > Why let me jump through all these hoops of masking and compiler optimizations if all I want is a branch and an early exit for a specific set of values? GPU schedulers and drivers make this easy to use and with somewhat predictable performance results. If the underlying hw is SIMD (vectors) and not SIMT anyway, as Nvidia hw apparently is, why should I have to go through the effort of rewriting my code in CUDA, and hope that some opaque driver will manage to turn that into efficient vector code? I mean, ideally I'd just like to write C/C++/Fortran/Julia/Haskell/whatever code, and the compiler would autovectorize it. > Furthermore (and probably more importantly), why is Intel putting this amount of compute power on a CPU without significantly upgrading memory bandwidth? Flops are cheap, bw expensive. But yeah, certainly the are many applications that would benefit from a much better bw/flops ratio. Then again, with the latest Teslas you have 16 GB with awesome bw, after that you're trying to feed the firehose through the PCIe straw.