3 ms·

Careful when invoking (some) AVX-512 instructions, because having a process using them on just one hardware thread can cripple your entire system and hurt overa

Careful when invoking (some) AVX-512 instructions, because having a process using them on just one hardware thread can cripple your entire system and hurt overall performance in workloads where the kernel or another process on the system is doing a lot of the work.

NullPrefix 6y ago
How do cloud vendors handle this on shared machines? Disable AVX?
- w0utert 6y ago
  Not sure how many cloud vendors are running CPU's with AVX-512 support, but I imagine that if they do you either have to pay for exclusive use of a single CPU (all cores), or thath they indeed disable AVX-512 instructions by patching the cpuid flags? Note that the downclocking is only really a problem with AVX-512 and much less so with AVX2 and (AFAIK) not at all with AVX.
- wtallis 6y ago
  The big cloud guys get custom parts with non-standard clock speeds, which may include changes to how AVX512 clocking is handled.
- BeeOnRope 6y ago
  The cloud vendors (at least the ones I checked) have SKUs with fairly limited downclocking, but more importantly the license based downclocking is per-core. So as long as you get at least a core to yourself, the noisy neighbor problem is reduced in that you affect your own CPU speed only. For more fine-grained sharing, sure, if the key before you was running AVX-512 you'll get a lower frequency for the initial part of your timeslice, but it should pop back to full speed in under 1 ms.
  - kristianp 6y ago
    Do the cloud vendors pin their VMs to a core to achieve this?
    - BeeOnRope 6y ago
      Something like that, for the ones that dedicate at least a core per tenant. This also reduces the chance of cross-tenant information leakage. Of course there are lots of scenarios where this isn't the case.
lorenzhs 6y ago
While that's true, I don't see how this is relevant to SIMDe? SIMDe lets you compile code using (e.g.) SSE/AVX intrinsics for ARM targets, using the target platform's SIMD intrinsics when possible. It doesn't even officially include support for AVX512 yet. Anyway, for anyone looking for details on what the OP mentioned, it's an implementation detail of Intel CPUs, see https://en.wikichip.org/wiki/intel/frequency_behavior#Base.2C_Non-AVX_Turbo.2C_and_AVX_Turbo https://en.wikichip.org/wiki/intel/frequency_behavior#Base.2...
- microcolonel 6y ago
  > it's an implementation detail of Intel CPUs While that may be technically true, since Intel is the only CPU vendor implementing AVX-512, the reason that severe downclocking is warranted is rooted in physics, and may not be resolvable given this ISA design. General vector ISAs like RISC-V "V" may have less trouble, since it is clearer how to design a vector unit not to interfere with scalar access to memory/caches. also, in the article they specifically mention adding shims for several AVX-512 ISAs: > We also have rapidly progressing implementations of many other extensions including NEON, AVX2, SVML, and several AVX-512 extensions (AVX-512F, AVX-512BW, AVX-512VL, etc.). if it's not obvious to you, this means that on machines with AVX-512 hardware will be invoking those instructions directly.
- gameswithgo 6y ago
  This is just an easy to understand fact about SIMD so people always post it.
- proverbialbunny 6y ago
  While it's not the most relevant, I don't know a lot about SIMD so I'm grateful to hear it.
- nemequ1729 6y ago
  It does include support for AVX-512, it's just still a work in progress. AVX-512 is enormous (IIRC ~ 4k functions), so it will be a while before it is fully supported. If you're targeting AVX-512 but don't have hardware that supports AVX-512 (like all AMD CPUs), SIMDe can be quite nice. The result is much faster than Intel's SDE and it's just native code that you can use your normal debugger on.

NullPrefix 6y ago

How do cloud vendors handle this on shared machines? Disable AVX?

w0utert 6y ago
Not sure how many cloud vendors are running CPU's with AVX-512 support, but I imagine that if they do you either have to pay for exclusive use of a single CPU (all cores), or thath they indeed disable AVX-512 instructions by patching the cpuid flags? Note that the downclocking is only really a problem with AVX-512 and much less so with AVX2 and (AFAIK) not at all with AVX.
wtallis 6y ago
The big cloud guys get custom parts with non-standard clock speeds, which may include changes to how AVX512 clocking is handled.
BeeOnRope 6y ago
The cloud vendors (at least the ones I checked) have SKUs with fairly limited downclocking, but more importantly the license based downclocking is per-core. So as long as you get at least a core to yourself, the noisy neighbor problem is reduced in that you affect your own CPU speed only. For more fine-grained sharing, sure, if the key before you was running AVX-512 you'll get a lower frequency for the initial part of your timeslice, but it should pop back to full speed in under 1 ms.
- kristianp 6y ago
  Do the cloud vendors pin their VMs to a core to achieve this?
  - BeeOnRope 6y ago
    Something like that, for the ones that dedicate at least a core per tenant. This also reduces the chance of cross-tenant information leakage. Of course there are lots of scenarios where this isn't the case.

lorenzhs 6y ago

While that's true, I don't see how this is relevant to SIMDe? SIMDe lets you compile code using (e.g.) SSE/AVX intrinsics for ARM targets, using the target platform's SIMD intrinsics when possible. It doesn't even officially include support for AVX512 yet. Anyway, for anyone looking for details on what the OP mentioned, it's an implementation detail of Intel CPUs, see https://en.wikichip.org/wiki/intel/frequency_behavior#Base.2C_Non-AVX_Turbo.2C_and_AVX_Turbo https://en.wikichip.org/wiki/intel/frequency_behavior#Base.2...

microcolonel 6y ago
> it's an implementation detail of Intel CPUs While that may be technically true, since Intel is the only CPU vendor implementing AVX-512, the reason that severe downclocking is warranted is rooted in physics, and may not be resolvable given this ISA design. General vector ISAs like RISC-V "V" may have less trouble, since it is clearer how to design a vector unit not to interfere with scalar access to memory/caches. also, in the article they specifically mention adding shims for several AVX-512 ISAs: > We also have rapidly progressing implementations of many other extensions including NEON, AVX2, SVML, and several AVX-512 extensions (AVX-512F, AVX-512BW, AVX-512VL, etc.). if it's not obvious to you, this means that on machines with AVX-512 hardware will be invoking those instructions directly.
gameswithgo 6y ago
This is just an easy to understand fact about SIMD so people always post it.
proverbialbunny 6y ago
While it's not the most relevant, I don't know a lot about SIMD so I'm grateful to hear it.
nemequ1729 6y ago
It does include support for AVX-512, it's just still a work in progress. AVX-512 is enormous (IIRC ~ 4k functions), so it will be a while before it is fully supported. If you're targeting AVX-512 but don't have hardware that supports AVX-512 (like all AMD CPUs), SIMDe can be quite nice. The result is much faster than Intel's SDE and it's just native code that you can use your normal debugger on.