4 ms·

Amazing, manual optimizing with algorithms like this can boost performance dramatically indeed. I wonder whether Intel provides AVX-512 optimized libraries for

Amazing, manual optimizing with algorithms like this can boost performance dramatically indeed. I wonder whether Intel provides AVX-512 optimized libraries for basic algorithms like sorting and searching, or for matrix multiplication? Without those libraries, and auto-vectorization still in research, we will have to hand craft algorithms with intrinsics, which is time consuming.

bmismyname 4y ago
There are already libraries that make it easier to do vectorization, which are equivalent to compilers that take generic instructions and convert them into SIMD equivalents (though it's somewhat more nuanced than this). The issue is not so much the compilers, but rather having a standard language for describing how to do the maths on vectors. It's a lot like the difference between writing C or assembly. Even today, sometimes you need asm to make the code go real quick because what the compiler spits out isn't always optimal (but for the general case, it's quite good).
Thaxll 4y ago
The thing is that Intel removed AVX512 in most CPUs. Only the servers one have or will have it. https://www.tomshardware.com/news/intel-nukes-alder-lake-avx-512-now-fuses-it-off-in-silicon https://www.tomshardware.com/news/intel-nukes-alder-lake-avx...
- janwas 4y ago
  Removal from Alder Lake was indeed regrettable but AVX-512 is still included in several recent client (non-server) CPUs: Icelake, Rocket Lake, Tiger Lake.
  - Thaxll 4y ago
    At the cost of reduced frequency if I'm not mistaken.
    - aseipp 4y ago
      The frequency scaling problems most prevalent in Skylake are significantly better as of e.g. Ice Lake. It's complex to summarize but basically those instructions request a higher "power license" for higher power delivery when used, broadly speaking. For single/dual core workloads on my laptop there is no frequency scaling loss IIRC, and even at full bore on all 4 cores I think the drop is from 3.6 to 3.5 nominal max clock. Ice Lake Client only has 1x FMA unit however, I don't know of any benchmarks of Ice Lake SP where there's 2x FMA units. You can reliably handle full datapath AVX-512 usage in client workloads these days, on Ice Lake or later, IMO. (Of course when you're on a laptop, the difference of 3.5GHz vs 3.6GHz on draining your battery doesn't feel too different...) Wide-datapath designs will generally have some tradeoffs like needing to ramp up power delivery, so there will be things like initial instruction latency for wide-datapath instructions, etc. That's kind of inevitable; I suspect the same will be true of the new fancy "variable length" vector ISAs if the underlying implementation and vector usage is wide enough, too. Also: you don't need to use wide 512-bit vectors with AVX-512! You can use the instruction set with 128-or-256/bit vectors just fine.
      - deleted 4y ago
        [deleted]
    - stagger87 4y ago
      You are mistaken.
      - mcronce 4y ago
        Not entirely. With Skylake, there was a clock speed penalty for "heavy" 256b instructions and "light" 512b instructions, and a steep clock speed penalty for "heavy" 512b instructions. With Ice Lake, there is a very small single-core clock speed penalty for 512b instructions. There is no clock speed penalty after Ice Lake. (Which, for non-server CPUs, is currently a list containing one generation: Rocket Lake.)
        stagger87 4y ago
- deleted 4y ago
  [deleted]
janwas 4y ago
Indeed a nice speedup! As to libraries: I'm the main author of github.com/google/highway which provides 'portable intrinsics' so you only have to write your code once for all platforms. It includes ready to use sorting and searching algorithms in hwy/contrib.
- synergy20 4y ago
  "Supported targets: scalar, S-SSE3, SSE4, AVX2, AVX-512, AVX3_DL (~Icelake, requires opt-in by defining HWY_WANT_AVX3_DL), NEON (ARMv7 and v8), SVE, SVE2, WASM SIMD, RISC-V V." wow this looks very interesting, I noticed the release passed 1.0 milestone, how did you unify all those intrinsics? I'm particularly interested in RVV1.0.
  - janwas 4y ago
    :) We (several engineers) put together a table [1] of instructions that several architectures natively support. There was enough overlap that I thought a wrapper would make sense. With a lot of the JPEG XL code written, when SVE and RVV were being introduced/discussed, I realized that pretty much the same code would work there, too: we just needed to replace a constexpr VecClass::kLanes with Lanes(), which is what Highway now does. Beyond the initial set of efficient-everywhere ops, we've added some (such as AbsDiff) that help on say Arm without hurting other platforms, nor can user code do better itself. There are also a few (ReorderWidenMulAccumulate) that define a non-obvious relaxation of the interface which is more efficient for all platforms than holding to any one platform's interface. For RVV, one remaining concern is the VSETVLI - the revised intrinsics now require an avl argument, which Highway creates in each function. It's not yet clear whether the compiler will be smart enough to (peephole-?)optimize away the duplicates. 1: https://github.com/google/highway/blob/master/g3doc/instruction_matrix.pdf https://github.com/google/highway/blob/master/g3doc/instruct...

bmismyname 4y ago

There are already libraries that make it easier to do vectorization, which are equivalent to compilers that take generic instructions and convert them into SIMD equivalents (though it's somewhat more nuanced than this). The issue is not so much the compilers, but rather having a standard language for describing how to do the maths on vectors. It's a lot like the difference between writing C or assembly. Even today, sometimes you need asm to make the code go real quick because what the compiler spits out isn't always optimal (but for the general case, it's quite good).

Thaxll 4y ago

The thing is that Intel removed AVX512 in most CPUs. Only the servers one have or will have it. https://www.tomshardware.com/news/intel-nukes-alder-lake-avx-512-now-fuses-it-off-in-silicon https://www.tomshardware.com/news/intel-nukes-alder-lake-avx...

janwas 4y ago
Removal from Alder Lake was indeed regrettable but AVX-512 is still included in several recent client (non-server) CPUs: Icelake, Rocket Lake, Tiger Lake.
- Thaxll 4y ago
  At the cost of reduced frequency if I'm not mistaken.
  - aseipp 4y ago
    The frequency scaling problems most prevalent in Skylake are significantly better as of e.g. Ice Lake. It's complex to summarize but basically those instructions request a higher "power license" for higher power delivery when used, broadly speaking. For single/dual core workloads on my laptop there is no frequency scaling loss IIRC, and even at full bore on all 4 cores I think the drop is from 3.6 to 3.5 nominal max clock. Ice Lake Client only has 1x FMA unit however, I don't know of any benchmarks of Ice Lake SP where there's 2x FMA units. You can reliably handle full datapath AVX-512 usage in client workloads these days, on Ice Lake or later, IMO. (Of course when you're on a laptop, the difference of 3.5GHz vs 3.6GHz on draining your battery doesn't feel too different...) Wide-datapath designs will generally have some tradeoffs like needing to ramp up power delivery, so there will be things like initial instruction latency for wide-datapath instructions, etc. That's kind of inevitable; I suspect the same will be true of the new fancy "variable length" vector ISAs if the underlying implementation and vector usage is wide enough, too. Also: you don't need to use wide 512-bit vectors with AVX-512! You can use the instruction set with 128-or-256/bit vectors just fine.
    - deleted 4y ago
      [deleted]
  - stagger87 4y ago
    You are mistaken.
    - mcronce 4y ago
      Not entirely. With Skylake, there was a clock speed penalty for "heavy" 256b instructions and "light" 512b instructions, and a steep clock speed penalty for "heavy" 512b instructions. With Ice Lake, there is a very small single-core clock speed penalty for 512b instructions. There is no clock speed penalty after Ice Lake. (Which, for non-server CPUs, is currently a list containing one generation: Rocket Lake.)
      - stagger87 4y ago
deleted 4y ago
[deleted]

janwas 4y ago

Indeed a nice speedup! As to libraries: I'm the main author of github.com/google/highway which provides 'portable intrinsics' so you only have to write your code once for all platforms. It includes ready to use sorting and searching algorithms in hwy/contrib.

synergy20 4y ago
"Supported targets: scalar, S-SSE3, SSE4, AVX2, AVX-512, AVX3_DL (~Icelake, requires opt-in by defining HWY_WANT_AVX3_DL), NEON (ARMv7 and v8), SVE, SVE2, WASM SIMD, RISC-V V." wow this looks very interesting, I noticed the release passed 1.0 milestone, how did you unify all those intrinsics? I'm particularly interested in RVV1.0.
- janwas 4y ago
  :) We (several engineers) put together a table [1] of instructions that several architectures natively support. There was enough overlap that I thought a wrapper would make sense. With a lot of the JPEG XL code written, when SVE and RVV were being introduced/discussed, I realized that pretty much the same code would work there, too: we just needed to replace a constexpr VecClass::kLanes with Lanes(), which is what Highway now does. Beyond the initial set of efficient-everywhere ops, we've added some (such as AbsDiff) that help on say Arm without hurting other platforms, nor can user code do better itself. There are also a few (ReorderWidenMulAccumulate) that define a non-obvious relaxation of the interface which is more efficient for all platforms than holding to any one platform's interface. For RVV, one remaining concern is the VSETVLI - the revised intrinsics now require an avl argument, which Highway creates in each function. It's not yet clear whether the compiler will be smart enough to (peephole-?)optimize away the duplicates. 1: https://github.com/google/highway/blob/master/g3doc/instruction_matrix.pdf https://github.com/google/highway/blob/master/g3doc/instruct...