5 ms·

Intel's AVX2 instructions don't crash the machine, but lead to an extreme increase in voltage and, thus, temperature. Sustained use of these instructions (say,

by woodson 10y ago

Intel's AVX2 instructions don't crash the machine, but lead to an extreme increase in voltage and, thus, temperature. Sustained use of these instructions (say, for example, in signal processing and iterative parameter estimation algorithms implemented using routines in Intel's MKL) requires cooling way beyond what the usual boxed cooler that comes with an i7 CPU can deliver in order to prevent throttling.

TazeTSchnitzel 10y ago
Probably part of why (the other obvious reason being die space) Ryzen lacks 256-bit SIMD hardware, and instead runs such instructions with a 128-bit unit over two cycles.
- informedperson 10y ago
  Well doing that tends to halve throughput for ALU-dense code, but obviously saves a bit of die space and more importantly reduces peak power draw, which has to be supported by the metallization on the chip. AVX2 might have also been just a bit too late to be integrated outside the microcode.
  - TazeTSchnitzel 10y ago
    I wonder how much worse it is in practice, because Intel's chips supposedly have to downclock a bit when doing AVX2 because of the heat.
    - informedperson 10y ago
      Desktop parts have no dedicated AVX clock (however, their base vs. all-core turbo pretty much does the same thing), while server SKUs (Socket R3 and later) basically have two distinct clocking ladders, one for non-AVX code and one for AVX code (the "mode switch" quantum is rather coarse, on the order of one millisecond). So both base clock and maximum turbo clocks are lower in AVX code compared to non-AVX code. As usual, actual achieved clock speeds depend on thermal performance and are not fixed. If cooling is completely insufficient, the clock frequency will drop below base clocks, i.e. thermal throttling. To clarify: This isn't a secret, but part of the specs and even in some Intel slides. (However, platform and processor specs, even at a basic (product D/S) level, tend to be a rather lengthy read, so most do away with reading all that).
stagger87 10y ago
That is disappointing to hear. Is this first hand knowledge, or do you have a discussion/link you could share on the topic?
- gcp 10y ago
  http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/performance-xeon-e5-v3-advanced-vector-extensions-paper.pdf http://www.intel.com/content/dam/www/public/us/en/documents/...
- DannyBee 10y ago
  It's documented and known that you can't execute AVX2 continuously without it clocking the processor down :(
  - semi-extrinsic 10y ago
    > clocking the processor down Nitpick, and certainly a matter of definition, but it's not clocking the processor down, it's just unable to turbo like it can with other loads.
    - DannyBee 10y ago
      Err, no, actually. it really is clocked down, and that's what intel says. There is the normal base frequency (which is, again, the one people see on the box. Intel calls it "marked TDP frequency") There is the avx base frequency, which is less. If you execute all avx workloads, it will clock down to the avx frequency, which is below the marked tdp frequency. They are very explicit about this: "Workloads using Intel AVX instructions may reduce processor frequency as far down as the AVX base frequency to stay within TDP limits." See http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/performance-xeon-e5-v3-advanced-vector-extensions-paper.pdf http://www.intel.com/content/dam/www/public/us/en/documents/... IE if you buy a 2.3ghz processor, and run all avx workloads on it, it may operate at 1.9ghz
antman 10y ago
Very interesting. Any links for further info?
magila 10y ago
This is the main reason why, for example, the i7-7700K comes with a "base clock" of 4.2 GHz even though for most workloads the CPU will operate at 4.4 or 4.5 GHz. That 4.2 GHz represents the maximum speed the CPU can maintain when running the worst case AVX2 workload without exceeding the specified 91W TDP.
- old-gregg 10y ago
  Interesting... do you mind me asking: how does overclocking works these days then? We have P-states, i.e. the CPU will go from 1200Mhz... 2400Mhz to 4Ghz, then there's "trubo" frequency range. So does modern overclocking apply some kind of multiplier to all these steps? Thanks.
  - wtallis 10y ago
    There's a base clock that's usually something like 100MHz and can usually be tweaked by a few percent. The base clock affects all power states, and usually the memory and I/O (which is usually the limiting factor). On top of that is a CPU frequency multiplier that is variable and the upper limit is unlocked on certain processor and motherboard combinations. If you have an unlocked multiplier you can set the maximum multiplier for each of the Turbo states (1-core, 2-core, through all-cores). You could configure them to all be the same multiplier, or just scale up the default behavior of running at a higher clock speed when fewer cores are active.
informedperson 10y ago
You're not wrong - the Vcore boost when both AVX pipes are powered up causes increased dissipation, but a large part of that is also caused by ... well, AVX processing huge amounts of data. For similar reasons server processors (Xeon E5, E7 series) throttle the core clock in AVX "mode".
Symmetry 10y ago
Sort of? If you can actually increase your performance architecturally by doubling your vector width then you double the power dissipated by the relevant bits of your execution and memory systems, true. But doubling performance by doubling power is really awesome. Things like your out of order window size will tend to increase power as the square of their size. And in your typical desktop cpu regime your power scales with cube of frequency. So if you're lucky enough to have code that can benefit form 256 bit vectors then AVX will double your performance and power then the thermal management system will throttle you back down to regular power and 80% of your regular clock speed for a net of merely a 60% increase in performance. Which is really nice if you happen to be doing all 256 bit wide vector math. If you're only infrequently using vectors your vector speedup will be smaller but so will the increase in power that needs to be overcome so it's still a net win.

by woodson 10y ago

TazeTSchnitzel 10y ago

Probably part of why (the other obvious reason being die space) Ryzen lacks 256-bit SIMD hardware, and instead runs such instructions with a 128-bit unit over two cycles.

informedperson 10y ago
Well doing that tends to halve throughput for ALU-dense code, but obviously saves a bit of die space and more importantly reduces peak power draw, which has to be supported by the metallization on the chip. AVX2 might have also been just a bit too late to be integrated outside the microcode.
- TazeTSchnitzel 10y ago
  I wonder how much worse it is in practice, because Intel's chips supposedly have to downclock a bit when doing AVX2 because of the heat.
  - informedperson 10y ago
    Desktop parts have no dedicated AVX clock (however, their base vs. all-core turbo pretty much does the same thing), while server SKUs (Socket R3 and later) basically have two distinct clocking ladders, one for non-AVX code and one for AVX code (the "mode switch" quantum is rather coarse, on the order of one millisecond). So both base clock and maximum turbo clocks are lower in AVX code compared to non-AVX code. As usual, actual achieved clock speeds depend on thermal performance and are not fixed. If cooling is completely insufficient, the clock frequency will drop below base clocks, i.e. thermal throttling. To clarify: This isn't a secret, but part of the specs and even in some Intel slides. (However, platform and processor specs, even at a basic (product D/S) level, tend to be a rather lengthy read, so most do away with reading all that).

stagger87 10y ago

That is disappointing to hear. Is this first hand knowledge, or do you have a discussion/link you could share on the topic?

gcp 10y ago
http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/performance-xeon-e5-v3-advanced-vector-extensions-paper.pdf http://www.intel.com/content/dam/www/public/us/en/documents/...
DannyBee 10y ago
It's documented and known that you can't execute AVX2 continuously without it clocking the processor down :(
- semi-extrinsic 10y ago
  > clocking the processor down Nitpick, and certainly a matter of definition, but it's not clocking the processor down, it's just unable to turbo like it can with other loads.
  - DannyBee 10y ago
    Err, no, actually. it really is clocked down, and that's what intel says. There is the normal base frequency (which is, again, the one people see on the box. Intel calls it "marked TDP frequency") There is the avx base frequency, which is less. If you execute all avx workloads, it will clock down to the avx frequency, which is below the marked tdp frequency. They are very explicit about this: "Workloads using Intel AVX instructions may reduce processor frequency as far down as the AVX base frequency to stay within TDP limits." See http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/performance-xeon-e5-v3-advanced-vector-extensions-paper.pdf http://www.intel.com/content/dam/www/public/us/en/documents/... IE if you buy a 2.3ghz processor, and run all avx workloads on it, it may operate at 1.9ghz

antman 10y ago

Very interesting. Any links for further info?

magila 10y ago

This is the main reason why, for example, the i7-7700K comes with a "base clock" of 4.2 GHz even though for most workloads the CPU will operate at 4.4 or 4.5 GHz. That 4.2 GHz represents the maximum speed the CPU can maintain when running the worst case AVX2 workload without exceeding the specified 91W TDP.

old-gregg 10y ago
Interesting... do you mind me asking: how does overclocking works these days then? We have P-states, i.e. the CPU will go from 1200Mhz... 2400Mhz to 4Ghz, then there's "trubo" frequency range. So does modern overclocking apply some kind of multiplier to all these steps? Thanks.
- wtallis 10y ago
  There's a base clock that's usually something like 100MHz and can usually be tweaked by a few percent. The base clock affects all power states, and usually the memory and I/O (which is usually the limiting factor). On top of that is a CPU frequency multiplier that is variable and the upper limit is unlocked on certain processor and motherboard combinations. If you have an unlocked multiplier you can set the maximum multiplier for each of the Turbo states (1-core, 2-core, through all-cores). You could configure them to all be the same multiplier, or just scale up the default behavior of running at a higher clock speed when fewer cores are active.

informedperson 10y ago

You're not wrong - the Vcore boost when both AVX pipes are powered up causes increased dissipation, but a large part of that is also caused by ... well, AVX processing huge amounts of data. For similar reasons server processors (Xeon E5, E7 series) throttle the core clock in AVX "mode".

Symmetry 10y ago

Sort of? If you can actually increase your performance architecturally by doubling your vector width then you double the power dissipated by the relevant bits of your execution and memory systems, true. But doubling performance by doubling power is really awesome. Things like your out of order window size will tend to increase power as the square of their size. And in your typical desktop cpu regime your power scales with cube of frequency. So if you're lucky enough to have code that can benefit form 256 bit vectors then AVX will double your performance and power then the thermal management system will throttle you back down to regular power and 80% of your regular clock speed for a net of merely a 60% increase in performance. Which is really nice if you happen to be doing all 256 bit wide vector math. If you're only infrequently using vectors your vector speedup will be smaller but so will the increase in power that needs to be overcome so it's still a net win.