4 ms·

I hear a lot about AVX-512 being really good. Is there any software that's commonly used that has a measurable performance boost with it? Or is it more special

by Jonnax 7y ago

I hear a lot about AVX-512 being really good. Is there any software that's commonly used that has a measurable performance boost with it? Or is it more specialised stuff?

devonkim 7y ago
Newer versions of the x265 encoder for h.265 / HEVC video standard get significant speed-ups. I believe Handbrake was recently updated for it.
- av3csr 7y ago
  I think RPCS3 also uses it in the JIT
KingMachiavelli 7y ago
From what I've read on AVX-512, the big disadvantage is that the AVX-512 instructions are very CPU intensive so the max clock speeds is reduced if you heavily use AVX-512 instructions. Another disadvantage is that you have to recompile your code to use AVX512 but it seems general enough that compilers will use the instructions (to an extent) without specialized code [1]. [1] https://www.phoronix.com/scan.php?page=news_item&px=GCC-8-AVX-512-Benchmarks https://www.phoronix.com/scan.php?page=news_item&px=GCC-8-AV...
- robocat 7y ago
  The problem appears to be that whether you get a performance increase or a performance penalty depends on the duty cycle of your AVX512 instructions. What is especially deceiving is profiling a function in a loop for more than say 50ms, when the normal function execution takes say 0.5ms. Long running functions get the most gain, while short running functions cause the most pain. That is because downclocking AVX512 lasts 2ms (with a 0.5ms setup). Certain instruction mixes will cause a general slowdown (10% degradation measured by CloudFlare under actual usage) even though the test profiling might predicts a performance gain. Single AVX512 instructions when the CPU is running at full speed have a counterintuitive perverse performance penalty - apparently running 4x slower than when the CPU changes to the slower L1 or L2 clocks. Sustained AVX512 usage has predictable performance. “ Intel made more aggressive use of AVX-512 instructions in earlier versions of the icc compiler, but has since removed most use unless the user asks for it with a special command line option.” is a strong indication that you need to be very careful about where you use the instructions. Running an encoder for 1 second - likely candidate. Occasional 1ms functions or single AVX512 instructions on a web server - likely penalty.
dragontamer 7y ago
> I hear a lot about AVX-512 being really good. Its a great instruction set, Absolutely great, AVX512 supports gather/scatter, a whole slew of efficient processing instructions, etc. etc. However, AVX512 has poor implementations right now. Skylake-X is one of the only implementations, and running it drops the clock-rate in ways that are difficult to predict. (One core running AVX512 drops the clock of other cores, slowing down the throughput of the entire server). Traditionally, the first implementation of these instruction sets always a degree of "emulated". For example, the gather/scatter instructions aren't much faster than load/stores in practice. So while the AVX512 instruction set could theoretically be efficiently implemented, it seem like Skylake-X's implementation leaves much to be desired. Hopefully future implementations will be better. ------ The other major implementation of AVX512 is Xeon Phi, which has been deemed end-of-life. I like the idea of Xeon Phi, but it just didn't seem to work out in practice.
- gameswithgo 7y ago
  the clock rate has nothing to do with a bad implementation, all the computation just makes a lot of heat. the performance increase is still massive, often more than 2x avx, which also throttles btw
  - DuskStar 7y ago
    The clock rate issue isn't the fact that it downclocks a core when moving to AVX512, it's that it downclocked all the other cores on the processor at the same time.
    - robocat 7y ago
      From what I have read, AVX512 only affects the one core (downclocking License level L1 or L2), it is older CPUs with AVX2 that affected all cores. Independently thermal throttling can occur which would affect all cores, although presumably the CPU is generating heat per numeric operation, so AVX512 is neutral versus other instructions per numeric operation.
      - 7y ago
3JPLW 7y ago
One challenge with AVX-512 is that it can actually _slow down_ your code. It's so power hungry that if you're using it on more than one core it almost immediately incurs significant throttling. Now, if everything you're doing is 512 bits at a time, you're still winning. But if you're interleaving scalar and vector arithmetic, the drop in clock speeds could slow down the scalar code quite substantially. https://blog.cloudflare.com/on-the-dangers-of-intels-frequency-scaling/ https://blog.cloudflare.com/on-the-dangers-of-intels-frequen...
- semi-extrinsic 7y ago
  How fast does the CPU step up and down the throttling caused by AVX-512? Basically, if you are interleaving like you suggest, does the processor detect this and reduce the throttling by the "duty cycle" of 512-bit operations? If not, could there be a way to tell the CPU to do this?
  - PeCaN 7y ago
    >How fast does the CPU step up and down the throttling caused by AVX-512? It's actually really, really slow. On newer (I think around Skylake-X, which is when AVX-512 was introduced) CPUs it takes up to 500 microseconds i.e. millions of cycles to activate AVX-512. This can't really be made faster because they actually need to give the voltage regulators time to adjust or the chip literally brows out. During this time AVX-512 instructions execute on the AVX-256 datapath¹. Once AVX-512 is activated the clock of that core is reduced by about 25% and it starts a 2ms timer which is reset whenever another AVX-512 instruction is issued. AFAIK Intel doesn't say how long it takes to raise the frequency again once the timer expires. (This is something of a simplification because there are actually two AVX power licenses, the first allowing AVX-256 and a limited set² of AVX-512 instructions, reducing clock by about 15%, and the second allowing everything. Also, executing a single AVX-512 instruction doesn't immediately request a higher power license, you have to execute a certain number of them.) ― This is actually the better version. On Haswell executing any AVX-256 instruction would reduce the frequency of every core by about 15-20%. But hey, at least it only takes about 150k cycles to activate (not much of a consolation, I know). Beats me how long it stays throttled for. (I don't know what exactly Broadwell did. I don't think it throttled all cores, but it didn't have the additional power license with reduced throttling that Skylake has.) ― ¹ Or the 128-bit datapath if the core is at the lowest power license (which still lets you use 128-bit SSE instructions, and basic AVX-256 instructions). ² Basically anything that doesn't execute on the floating-point unit, which means no floating point and no integer multiplication (which uses the FPU). This is actually kinda the saving grace of the whole thing, since it means you can vectorize things like memcmp and strlen without requiring the highest power license.
pkaye 7y ago
I've used it once but on Intel the clock is throttled with AVX-512 so the overall program performance improved only 1% because the non-SIMD code was running at a slower clock.

by Jonnax 7y ago

I hear a lot about AVX-512 being really good. Is there any software that's commonly used that has a measurable performance boost with it? Or is it more specialised stuff?

devonkim 7y ago

Newer versions of the x265 encoder for h.265 / HEVC video standard get significant speed-ups. I believe Handbrake was recently updated for it.

av3csr 7y ago
I think RPCS3 also uses it in the JIT

KingMachiavelli 7y ago

From what I've read on AVX-512, the big disadvantage is that the AVX-512 instructions are very CPU intensive so the max clock speeds is reduced if you heavily use AVX-512 instructions. Another disadvantage is that you have to recompile your code to use AVX512 but it seems general enough that compilers will use the instructions (to an extent) without specialized code [1]. [1] https://www.phoronix.com/scan.php?page=news_item&px=GCC-8-AVX-512-Benchmarks https://www.phoronix.com/scan.php?page=news_item&px=GCC-8-AV...

robocat 7y ago
The problem appears to be that whether you get a performance increase or a performance penalty depends on the duty cycle of your AVX512 instructions. What is especially deceiving is profiling a function in a loop for more than say 50ms, when the normal function execution takes say 0.5ms. Long running functions get the most gain, while short running functions cause the most pain. That is because downclocking AVX512 lasts 2ms (with a 0.5ms setup). Certain instruction mixes will cause a general slowdown (10% degradation measured by CloudFlare under actual usage) even though the test profiling might predicts a performance gain. Single AVX512 instructions when the CPU is running at full speed have a counterintuitive perverse performance penalty - apparently running 4x slower than when the CPU changes to the slower L1 or L2 clocks. Sustained AVX512 usage has predictable performance. “ Intel made more aggressive use of AVX-512 instructions in earlier versions of the icc compiler, but has since removed most use unless the user asks for it with a special command line option.” is a strong indication that you need to be very careful about where you use the instructions. Running an encoder for 1 second - likely candidate. Occasional 1ms functions or single AVX512 instructions on a web server - likely penalty.

dragontamer 7y ago

> I hear a lot about AVX-512 being really good. Its a great instruction set, Absolutely great, AVX512 supports gather/scatter, a whole slew of efficient processing instructions, etc. etc. However, AVX512 has poor implementations right now. Skylake-X is one of the only implementations, and running it drops the clock-rate in ways that are difficult to predict. (One core running AVX512 drops the clock of other cores, slowing down the throughput of the entire server). Traditionally, the first implementation of these instruction sets always a degree of "emulated". For example, the gather/scatter instructions aren't much faster than load/stores in practice. So while the AVX512 instruction set could theoretically be efficiently implemented, it seem like Skylake-X's implementation leaves much to be desired. Hopefully future implementations will be better. ------ The other major implementation of AVX512 is Xeon Phi, which has been deemed end-of-life. I like the idea of Xeon Phi, but it just didn't seem to work out in practice.

gameswithgo 7y ago
the clock rate has nothing to do with a bad implementation, all the computation just makes a lot of heat. the performance increase is still massive, often more than 2x avx, which also throttles btw
- DuskStar 7y ago
  The clock rate issue isn't the fact that it downclocks a core when moving to AVX512, it's that it downclocked all the other cores on the processor at the same time.
  - robocat 7y ago
    From what I have read, AVX512 only affects the one core (downclocking License level L1 or L2), it is older CPUs with AVX2 that affected all cores. Independently thermal throttling can occur which would affect all cores, although presumably the CPU is generating heat per numeric operation, so AVX512 is neutral versus other instructions per numeric operation.
    - 7y ago

3JPLW 7y ago

One challenge with AVX-512 is that it can actually _slow down_ your code. It's so power hungry that if you're using it on more than one core it almost immediately incurs significant throttling. Now, if everything you're doing is 512 bits at a time, you're still winning. But if you're interleaving scalar and vector arithmetic, the drop in clock speeds could slow down the scalar code quite substantially. https://blog.cloudflare.com/on-the-dangers-of-intels-frequency-scaling/ https://blog.cloudflare.com/on-the-dangers-of-intels-frequen...

semi-extrinsic 7y ago
How fast does the CPU step up and down the throttling caused by AVX-512? Basically, if you are interleaving like you suggest, does the processor detect this and reduce the throttling by the "duty cycle" of 512-bit operations? If not, could there be a way to tell the CPU to do this?
- PeCaN 7y ago
  >How fast does the CPU step up and down the throttling caused by AVX-512? It's actually really, really slow. On newer (I think around Skylake-X, which is when AVX-512 was introduced) CPUs it takes up to 500 microseconds i.e. millions of cycles to activate AVX-512. This can't really be made faster because they actually need to give the voltage regulators time to adjust or the chip literally brows out. During this time AVX-512 instructions execute on the AVX-256 datapath¹. Once AVX-512 is activated the clock of that core is reduced by about 25% and it starts a 2ms timer which is reset whenever another AVX-512 instruction is issued. AFAIK Intel doesn't say how long it takes to raise the frequency again once the timer expires. (This is something of a simplification because there are actually two AVX power licenses, the first allowing AVX-256 and a limited set² of AVX-512 instructions, reducing clock by about 15%, and the second allowing everything. Also, executing a single AVX-512 instruction doesn't immediately request a higher power license, you have to execute a certain number of them.) ― This is actually the better version. On Haswell executing any AVX-256 instruction would reduce the frequency of every core by about 15-20%. But hey, at least it only takes about 150k cycles to activate (not much of a consolation, I know). Beats me how long it stays throttled for. (I don't know what exactly Broadwell did. I don't think it throttled all cores, but it didn't have the additional power license with reduced throttling that Skylake has.) ― ¹ Or the 128-bit datapath if the core is at the lowest power license (which still lets you use 128-bit SSE instructions, and basic AVX-256 instructions). ² Basically anything that doesn't execute on the floating-point unit, which means no floating point and no integer multiplication (which uses the FPU). This is actually kinda the saving grace of the whole thing, since it means you can vectorize things like memcmp and strlen without requiring the highest power license.

pkaye 7y ago

I've used it once but on Intel the clock is throttled with AVX-512 so the overall program performance improved only 1% because the non-SIMD code was running at a slower clock.