5 ms·

Francois is a smart guy, but there's a range of opinions on the matters he raises. We argue a good deal on Twitter ("constructive confrontation" between former

Francois is a smart guy, but there's a range of opinions on the matters he raises. We argue a good deal on Twitter ("constructive confrontation" between former Intel Principal Engineers :-) ). He put together the video pretty quickly so don't dismiss it just for lack of polish. Personally, I argue with him a lot on AVX-512. I think AVX-512 is a Good Thing (or will be shortly - the first instantiation in Skylake Server - SKX - isn't great). The biggest meta-problem with AVX-512 is that due to process issues, the pause button got hit just as it appeared in a server-only, 'first cut' form. AVX2 had downclocking issues when it first appeared too - but these were rapidly mitigated and forgotten. I personally feel that SIMD is underutilized in general purpose workloads (partly due to Intel's fecklessness in promoting SIMD - including gratuitous fragmentation - e.g. a new Comet Lake machine with Pentium branding doesn't even suport AVX2 and has it fused off). Daniel Lemire and I built simdjson (https://github.com/lemire/simdjson https://github.com/lemire/simdjson) to show the power of SIMD in a non-traditional domain, and a lot of the benefits of Hyperscan (https://github.com/intel/hyperscan https://github.com/intel/hyperscan) were also due to SIMD. Notably, both use SIMD in a range of ways that aren't necessarily all that amenable to the whole idea of "just do it all on an accelerator/GPU" - choppy mixes of SIMD and GPR-based computing. Not everything looks like matrix multplication, or AI (but I repeat myself :-) ). AVX-512 is a combination of fascinating and frustrating. I enthuse about recent developments on my blog (https://branchfree.org/2019/05/29/why-ice-lake-is-important-a-bit-bashers-perspective/ https://branchfree.org/2019/05/29/why-ice-lake-is-important-...) so I won't repeat why it's so interesting, but the learning curve is steep and there's plenty of frustrations and hidden traps. Intel collectively tends to be more interested in Magic Fixes (autovec) and Special Libraries for Important Benchmarks rather than building first-rate material to improve the general level of expertise with SIMD programming (one of my perpetual complaints). You more or less have to teach yourself and feel your way through the process of getting good at SIMD. That being said, the benefits are often huge - not only because SIMD is fast, but because you can do fundamentally different things that you can't do quickly in GPR ("General Purpose Register") land (look, for example, at everyone's favorite instruction PSHUFB - you can't do the equivalent on a GPR without going to memory).

glangdale 6y ago
Oh, I should note that an interest in large SIMD is not confined to Intel - the ARM world is (slowly) heading towards SVE (or so my friends from that quarter assure me). So it's not just a partisan pro-Intel or pro-x86 thing on my part.
- pantalaimon 6y ago
  RISC-V is also adding Vector extensions. The cool thing about those is that they are variable length, so code can take advantage of newer hardware / degrade gracefully on less capable one. https://gms.tf/riscv-vector.html https://gms.tf/riscv-vector.html
  - glangdale 6y ago
    I think going for variable length is a Bold Move, and am not sure I like it for an ISA that otherwise is trying to be conservative. I'll be more optimistic about it when I see SVE (a) on more than one HPC machine and (b) implemented in a variety of widths. I'll be impressed if it works, but I'm not going to be impressed ahead of time with it while it's an aspirational design because it has aspirations.
  - PaulHoule 6y ago
    The small, medium and large sizes built into arms neon accomplishes some scalability without going back to what vector units were like in the big hair 1980s.
  - Const-me 6y ago
    > so code can take advantage of newer hardware / degrade gracefully on less capable one I never user variable-length SIMD on CPUs, but I have substantial experience with current SIMD up to AVX2, and GPGPUs. I’m not sure I like that variable-length stuff. 1. Variable-length SIMD is only suitable for vertical operations. Many useful algorithms are only possible to vectorize with shuffles and such. 2. With registers size unknown to compilers, it’s impossible to create a calling convention to pass data in registers, as opposed to RAM. RAM is slow on modern hardware, and is becoming slower over time (in relation to compute speed; nanoseconds are fine). GPUs are OK with that because they are massively multithreaded, they switch threads instead of waiting for data from memory. CPUs can’t do that, they only have 2 hardware threads per core.

glangdale 6y ago

Oh, I should note that an interest in large SIMD is not confined to Intel - the ARM world is (slowly) heading towards SVE (or so my friends from that quarter assure me). So it's not just a partisan pro-Intel or pro-x86 thing on my part.

pantalaimon 6y ago
RISC-V is also adding Vector extensions. The cool thing about those is that they are variable length, so code can take advantage of newer hardware / degrade gracefully on less capable one. https://gms.tf/riscv-vector.html https://gms.tf/riscv-vector.html
- glangdale 6y ago
  I think going for variable length is a Bold Move, and am not sure I like it for an ISA that otherwise is trying to be conservative. I'll be more optimistic about it when I see SVE (a) on more than one HPC machine and (b) implemented in a variety of widths. I'll be impressed if it works, but I'm not going to be impressed ahead of time with it while it's an aspirational design because it has aspirations.
- PaulHoule 6y ago
  The small, medium and large sizes built into arms neon accomplishes some scalability without going back to what vector units were like in the big hair 1980s.
- Const-me 6y ago
  > so code can take advantage of newer hardware / degrade gracefully on less capable one I never user variable-length SIMD on CPUs, but I have substantial experience with current SIMD up to AVX2, and GPGPUs. I’m not sure I like that variable-length stuff. 1. Variable-length SIMD is only suitable for vertical operations. Many useful algorithms are only possible to vectorize with shuffles and such. 2. With registers size unknown to compilers, it’s impossible to create a calling convention to pass data in registers, as opposed to RAM. RAM is slow on modern hardware, and is becoming slower over time (in relation to compute speed; nanoseconds are fine). GPUs are OK with that because they are massively multithreaded, they switch threads instead of waiting for data from memory. CPUs can’t do that, they only have 2 hardware threads per core.