4 ms·

That explains what I observed while optimizing convolution functions in OpenBLAS, adding support for AVX512 while AVX2 was already there. On a 36-core Xeon Plat

by yalok 8y ago

That explains what I observed while optimizing convolution functions in OpenBLAS, adding support for AVX512 while AVX2 was already there. On a 36-core Xeon Platinum there was no measurable gain when moving to AVX512, maybe even a bit slower, when running intensive convolutions full of Fuse Multiply Add - FMLA. I was puzzled and thought it’s due to hitting the memory bandwidth limit (but pre-caching more didn’t help), or due to the pipeline - but again, unrolling the loop and reshuffling AVX512 instructions in between “lighter” instructions didn’t seem to help either. I should have monitored the CPU clock changes. But I had no idea there was this kind of dependency. I now wonder what should be the average ratio of heavy 512-bit was AVX512 instructions vs lighter 256-bit instructions to avoid getting into permanent L2 mode and stay at L1. Maybe paralleling one 512bit loop unroll with several 256bit unrolls may still yield some gains in heavy avx usage... Thank you for the article!

nkurz 8y ago
I now wonder what should be the average ratio of heavy 512-bit was AVX512 instructions vs lighter 256-bit instructions to avoid getting into permanent L2 mode and stay at L1. One of the co-authors talks more about the exact limits here: https://www.realworldtech.com/forum/?threadid=179654&curpostid=179654 https://www.realworldtech.com/forum/?threadid=179654&curpost... The quick answer is that on the W-2104 system he tested, you can sustain one FMA every 2 cycles while still remaining in the medium speed L1 state. His avx-turbo tool (https://github.com/travisdowns/avx-turbo https://github.com/travisdowns/avx-turbo) can be used to check the situation for your particular processor.
gnufx 8y ago
Does the OB work follow BLIS and libxsmm? There's been at least some discussion of avx512 trades-off around them and the authors would probably advise. (Not that there's a single "avx512"...)

by yalok 8y ago

nkurz 8y ago

I now wonder what should be the average ratio of heavy 512-bit was AVX512 instructions vs lighter 256-bit instructions to avoid getting into permanent L2 mode and stay at L1. One of the co-authors talks more about the exact limits here: https://www.realworldtech.com/forum/?threadid=179654&curpostid=179654 https://www.realworldtech.com/forum/?threadid=179654&curpost... The quick answer is that on the W-2104 system he tested, you can sustain one FMA every 2 cycles while still remaining in the medium speed L1 state. His avx-turbo tool (https://github.com/travisdowns/avx-turbo https://github.com/travisdowns/avx-turbo) can be used to check the situation for your particular processor.

gnufx 8y ago

Does the OB work follow BLIS and libxsmm? There's been at least some discussion of avx512 trades-off around them and the authors would probably advise. (Not that there's a single "avx512"...)