4 ms·
The problem with AVX2 accelerated code (and much of AVX) is that unless you have a lot of it to run you end up with a substantial speed hit that comes from the
by yoklov 8y ago
The problem with AVX2 accelerated code (and much of AVX) is that unless you have a lot of it to run you end up with a substantial speed hit that comes from the cost of switching to a different power bin (which often takes 1 or 2ms!) and then running at a lower clock speed.
This often still ends up being an improvement over scalar code (at the cost of higher power usage), but for occasional workloads that don't need to do multiple milliseconds of AVX instructions you tend to have better results from 4-wide vectors, which don't have this cost.
- jwr 8y agoSee also the excellent writeup at: https://gist.github.com/rygorous/32bc3ea8301dba09358fd2c64e02d774 https://gist.github.com/rygorous/32bc3ea8301dba09358fd2c64e0...
- stochastic_monk 8y agoSo in a sense, using wider vectors should be done at times when you’re certain to need heavy lifting, much like a GPU, but at a smaller timescale. I should update my vectorization library so that it doesn’t simply use the widest possible.
- puzzle 8y agoYou also have penalties from context switches, although you can reduce their impact by performing them in a lazy fashion. AVX512 is even worse, of course.
- jeffreyrogers 8y agoYou need a context switch to use AVX?
- puzzle 8y agoNo, unless you have a special setup where one process on the whole machine can use AVX. That might make sense for special, controlled environments. What I meant is that there are more registers to shuffle around at context switch time, when multiple processes use the extensions.
- jeffreyrogers 8y agoAh got it, thanks.
- deleted 8y ago[deleted]
- yoklov 8y agoThat's true, these are always costly though.
- rphlx 8y agoTransferring SIMD/FPU regs to/from memory is actually fast - a modest fraction of the typical context switch cost - on modern x86_64 thanks to XSAVE.
- deleted 8y ago[deleted]
- rphlx 8y agoFWIW Haswell - on which this was benchmarked - does not downclock on AVX2; it just boosts vcore and thermal throttles if the cooling solution can't keep the die below ~100C (which of course, the crappy TIM plus bundled desktop heatsink have essentially zero chance of doing with many extended workloads). To preclude thermal throttling the auto-downclock-on-AVX2 was added later, I think for Skylake.