3 ms·

Is there the option to use AVX2 (256-bit) instead of AVX-512 (which can cause thermal throttling on basically every chip[a])? Now, if Intel can get AVX-512 work

Is there the option to use AVX2 (256-bit) instead of AVX-512 (which can cause thermal throttling on basically every chip[a])? Now, if Intel can get AVX-512 working well, that’s something else. Side question: do any Ryzen processors support AVX-512? AFAIK, they only support up to AVX2. [a]: IIRC, some tests show AVX2 code actually being faster than the equivalent AVX-512 code because AVX2 doesn’t cause the processor to throttle so hard

aseipp 6y ago
AVX2 also causes various degrees of thermal throttling depending on which chip you're using, such as Haswell. AVX-512 on Ice Lake (client) however is quite good and causes very little throttling versus its sustained speeds (e.g. my Ice Lake laptop @ 3.6GHz or whatever it is). Ultimately you have to do benchmarking yourself and if you care about inferences/sec you'll have to think about this stuff. That said, AVX2 support would also be cool simply because it's more readily available to use on more platforms, not just my laptop...
- banachtarski 6y ago
  AVX-512 causes just as much if not more throttling in my experience? In what case are you finding that AVX-512 runs at lower voltages than AVX2?
37ef_ced3 6y ago
No support for AVX2. AVX-512 is the first really nice (from the programmer's perspective) SIMD instruction set on x86-64 CPUs, and it's significantly different from what came before. AVX-512 is not just a wider version of AVX2 NN-512 explores the way Winograd and Fourier convolutions can be done when you have 32 512-bit vector registers. Four 8x8 Winograd tiles simultaneously, four 8x8 Fourier tiles interleaved to form a 16x16 tile for strided convolutions, and so on. These multi-tile operations don't work for AVX2, too few registers, each register too narrow GCC has only very recently come into a state where it properly supports AVX-512. For example, before GCC 9.1, GCC would split FNMADD into xor-negation followed by FMADD (doing an extra xor, using an extra register for the negation constant, instead of just using the FNMADD instruction). Eventually the hardware will mature, too
reitzensteinm 6y ago
There's no way an AVX2 version of this would run as fast even taking throttling in to account. The throttling is aggressive in mixed code paths, but not really all that bad if you're almost exclusively running vector code. AVX2 throttled a fair bit in the first implementation, and the penalty has generally reduced. I'd expect something similar here, it's just that Intel chips are barely changed in three years. And as the author says, 512 is a game changer, not just a wider AVX2. It's much more flexible.

aseipp 6y ago

AVX2 also causes various degrees of thermal throttling depending on which chip you're using, such as Haswell. AVX-512 on Ice Lake (client) however is quite good and causes very little throttling versus its sustained speeds (e.g. my Ice Lake laptop @ 3.6GHz or whatever it is). Ultimately you have to do benchmarking yourself and if you care about inferences/sec you'll have to think about this stuff. That said, AVX2 support would also be cool simply because it's more readily available to use on more platforms, not just my laptop...

banachtarski 6y ago
AVX-512 causes just as much if not more throttling in my experience? In what case are you finding that AVX-512 runs at lower voltages than AVX2?

37ef_ced3 6y ago

No support for AVX2. AVX-512 is the first really nice (from the programmer's perspective) SIMD instruction set on x86-64 CPUs, and it's significantly different from what came before. AVX-512 is not just a wider version of AVX2 NN-512 explores the way Winograd and Fourier convolutions can be done when you have 32 512-bit vector registers. Four 8x8 Winograd tiles simultaneously, four 8x8 Fourier tiles interleaved to form a 16x16 tile for strided convolutions, and so on. These multi-tile operations don't work for AVX2, too few registers, each register too narrow GCC has only very recently come into a state where it properly supports AVX-512. For example, before GCC 9.1, GCC would split FNMADD into xor-negation followed by FMADD (doing an extra xor, using an extra register for the negation constant, instead of just using the FNMADD instruction). Eventually the hardware will mature, too

reitzensteinm 6y ago

There's no way an AVX2 version of this would run as fast even taking throttling in to account. The throttling is aggressive in mixed code paths, but not really all that bad if you're almost exclusively running vector code. AVX2 throttled a fair bit in the first implementation, and the penalty has generally reduced. I'd expect something similar here, it's just that Intel chips are barely changed in three years. And as the author says, 512 is a game changer, not just a wider AVX2. It's much more flexible.