On Skylake-SP's AVX-512, instructions that previously were dispatched to port 0 or 1 get instead dispatched to ports 0 _and_ 1. So instructions like vpsrlq get

On Skylake-SP's AVX-512, instructions that previously were dispatched to port 0 or 1 get instead dispatched to ports 0 _and_ 1. So instructions like vpsrlq get zero net speedup from switching to AVX-512 from AVX2. Instructions that previously ran on ports 0,1,5 will now run on ports 0 and 5, for a speedup of at best 1.33. Multiplication will depend on whether the chip has one or two FMA units. If so, you can run vpmuludq on ports 0 and 5, which is a 2x speedup compared to AVX2's ports 0 and 1. This 8275CL Xeon does have 2 FMA units. Looking at the two inner loops, we have up: vmovdqa64 zmm0,ZMMWORD PTR [rdi+rax*4] # p23 add rax,0x10 # p0156 vpmuludq zmm1,zmm0,zmm4 # p05 or only p0 vpsrlq zmm2,zmm1,0x20 # p0 vpsrlq zmm1,zmm0,0x20 # p0 vpmuludq zmm1,zmm1,zmm4 # p05 or only p0 vpandd zmm1,zmm6,zmm1 # p05 vpord zmm1,zmm1,zmm2 # p05 vpsubd zmm0,zmm0,zmm1 # p05 vpsrld zmm0,zmm0,0x1 # p0 vpaddd zmm0,zmm0,zmm1 # p05 vpsrld zmm0,zmm0,xmm5 # p0+p5 vpaddd zmm3,zmm0,zmm3 # p05 cmp rax,rdx jb up up: vmovdqa ymm0,YMMWORD PTR [rdi+rax*4] # p23 add rax,0x8 # p0156 vpmuludq ymm1,ymm0,ymm4 # p01 vpsrlq ymm2,ymm1,0x20 # p01 vpsrlq ymm1,ymm0,0x20 # p01 vpmuludq ymm1,ymm1,ymm4 # p01 vpand ymm1,ymm1,ymm6 # p015 vpor ymm1,ymm1,ymm2 # p015 vpsubd ymm0,ymm0,ymm1 # p015 vpsrld ymm0,ymm0,0x1 # p01 vpaddd ymm0,ymm0,ymm1 # p015 vpsrld ymm0,ymm0,xmm5 # p01+p5 vpaddd ymm3,ymm0,ymm3 # p01 cmp rax,rdx jb up All other things being equal, we have on average, and counting only the differing instructions, a throughput of ~2.27 instructions per cycle on the AVX2 loop, whereas it is somewhere around ~1.45-1.60 for AVX-512, depending whether you have 1 or 2 FMA units to run multiplications on port 5. So based on this approximation, the AVX-512 code should probably run around 2*(1.5/2.27) ~ 1.33 times faster. Add to this that vpmuludq is actually one of the most thermally insensitive instructions around and will reduce your core's frequency by 100-200 MHz, and the small speedup you see is more or less explainable. (I actually do see some more noticeable speedup here when switching to AVX-512; 0.25 vs 0.21). PS: The Intel Icelake and later chips also manage to achieve a throughput of 1/2 divisions per cycle for 32-bit divisors, and 1/3 divisions per cycle for 64-bit divisors. PPS: Some of your functions could use blends. For example, __m256i libdivide_mullhi_u32_vec256_(__m256i a, __m256i b) { __m256i hi_product_0Z2Z = _mm256_srli_epi64(_mm256_mul_epu32(a, b), 32); __m256i a1X3X = _mm256_srli_epi64(a, 32); __m256i hi_product_Z1Z3 = _mm256_mul_epu32(a1X3X, b); return _mm256_blend_epi32(hi_product_0Z2Z, hi_product_Z1Z3, 0xAA); } __m512i libdivide_mullhi_u32_vec512_(__m512i a, __m512i b) { __m512i hi_product_0Z2Z = _mm512_srli_epi64(_mm512_mul_epu32(a, b), 32); __m512i a1X3X = _mm512_srli_epi64(a, 32); __m512i hi_product_Z1Z3 = _mm512_mul_epu32(a1X3X, b); return _mm512_mask_blend_epi32((__mmask16)0xAAAA, hi_product_0Z2Z, hi_product_Z1Z3); }