4 ms·

> In my line of work, AVX512 is useless. This won't change until at least 25% market penetration on clients, which may or may not happen. Why not use it where

• Const-me 5y ago
  > Why not use it where available? AFAIK things like that (highway, OpenMP 4.0+, automatic vectorizers) are only good for vertical operations. When the only thing you’re doing is billions of vertical operations, computing that thing on CPU is often a poor choice because GPUs are way faster at that, and way more power efficient too. Therefore, when I optimize CPU-running things, that code is usually not a good fit for GPUs. Sometimes the data size is too small (PCIe latency gonna eat all the profit from GPU), sometimes too many branches, but most typical reason is horizontal operations. An example is multiplying 6x24 by 24x1 matrices. A good approach is a loop with 12 iterations, in the loop body 2x _mm256_broadcast_sd, then _mm256_blend_pd (these 3 instructions are making 3 vectors [v1,v1,v1,v1], [v1,v1,v2,v2], [v2,v2,v2,v2] ), then 3x _mm256_fmadd_pd with memory source operands. Then after the loop add 12 values into 6 with shuffles and vector adds.
  • janwas 5y ago
    > things like that (highway, OpenMP 4.0+, automatic vectorizers) are only good for vertical operations. I agree autovec and #pragma omp simd struggle with any kind of shuffles, but Highway does not belong in that category - it offers more control, similar to intrinsics. There are about two dozen non-vertical ops (see https://github.com/google/highway/blob/master/g3doc/quick_reference.md#combine https://github.com/google/highway/blob/master/g3doc/quick_re... and "blockwise" and "swizzle") and it sounds like you want Set and ConcatUpperLower. In your experience, have you needed any non-vertical ops that are not in that list?
    • Const-me 5y ago
      > have you needed any non-vertical ops that are not in that list? Yes indeed. I rarely using SIMD for vertical-only ops, for such use cases GPUs are very often better than CPUs. I’ve already wrote an example in my previous comment. It’s possible to emulate with the stuff you have, however _mm256_blend_pd is very fast instruction, a single cycle of latency. The highway’s emulation is going to be way more expensive. You probably compiling your UpperHalf() into _mm256_extractf128_pd and Combine() into _mm256_insertf128_pd, that’s 2 instructions and (on Skylake) 6 cycles of latency instead of 1 cycle. 6 cycles instead of 1 cycle is a large overhead in that context. That particular small matrix multiplication is called rather often. I only optimizing code when the profiler tells me so. For the majority of CPU bound code in that project, Eigen’s implementation is actually good enough. I’ve searched the source code of that project (CAM/CAE software). Here’s the list of the shuffle intrinsics I use, some of them a lot: _mm256_blend_pd, _mm_blend_ps, _mm_blend_epi32, _mm256_permute2f128_pd, _mm256_permute_ps, _mm256_permute4x64_pd, _mm256_permutevar8x32_ps, _mm256_permutevar8x32_epi32, _mm_permute_ps, _mm_permute_pd, _mm_insert_ps, _mm_movehdup_ps, _mm_moveldup_ps, _mm_loaddup_pd, _mm_extract_ps, _mm_dp_ps, _mm_extract_epi32, _mm_extract_epi64, _mm_shuffle_epi32. A similar list for this project https://github.com/Const-me/Vrmac https://github.com/Const-me/Vrmac (a GPU-centric library for 3D and 2D graphics, not using any AVX): _mm_shuffle_epi8, _mm_alignr_epi8, _mm_shuffle_epi32, _mm_shuffle_ps, _mm_addsub_ps (that one is vertical but still missing from highway), _mm_insert_epi32, _mm_insert_ps, _mm_extract_ps, _mm_extract_epi16, _mm_movehdup_ps, _mm_dp_ps. BTW the project is portable between AMD64 and ARMv7, I have tons of #ifdef there to support NEON which differs substantially, there’s stuff like vrev64q_f32 and vextq_f32, 64-bit SIMD vectors, and quite a few other instructions missing on AMD64. Even if you expose all the missing horizontal stuff in highway — won’t be much better than intrinsics. Such code ain’t gonna use AVX512 when available. Only going to inflate the software complexity for no good reason, by adding an unneeded layer of abstraction between the application’s code and the actual hardware.
      • janwas 5y ago
        > _mm256_blend_pd is very fast instruction, a single cycle of latency. The highway’s emulation is going to be way more expensive. Actually it uses exactly the same instruction :) https://github.com/google/highway/blob/master/hwy/ops/x86_256-inl.h#L2704 https://github.com/google/highway/blob/master/hwy/ops/x86_25... Thanks for sharing the list! Out of curiosity, can you expand on what _mm256_permute4x64_pd is used for? > _mm_addsub_ps (that one is vertical but still missing from highway) Indeed, we have not needed complex-valued functions yet. AFAIK NEON does not have such an instruction but it could be emulated with an extra XOR+constant. Will add to wishlist, we implement them whenever there's a use case. > I have tons of #ifdef there Seems reasonable if you only target v7 and ASIMD; but #ifdef is increasingly infeasible now that SVE and Risc-V V are coming, no? > to support NEON which differs substantially, there’s stuff like vrev64q_f32 and vextq_f32 Those can actually be bridged just fine, they correspond to Shuffle2301 and CombineShiftRightLanes. > Even if you expose all the missing horizontal stuff in highway — won’t be much better than intrinsics. Such code ain’t gonna use AVX512 when available. Because of the hardcoded vector size? I agree that's best avoided, in your case perhaps by batching the matrix mul and applying SIMD over that dimension instead? Even if not, I'd rather have type-safety, friendlier function names and gaps in the ISA filled, but tastes differ.
• nikita 5y ago
  Are you looking for a job by any chance :)?