4 ms·

"A tricky beast" definitely seems fair. These things have to scale per-core power consumption from ~13W down to ~3W on the fly to stay within their 280W limit.

"A tricky beast" definitely seems fair. These things have to scale per-core power consumption from ~13W down to ~3W on the fly to stay within their 280W limit. To my knowledge, AMD hasn't implemented any proactive measures that are as severe as Intel's AVX512 strategy, where the instructions get split and handled by the narrower vector units for a surprisingly long time while powering up the full-width vector units (and dropping CPU clocks). This AMD instability is only using AVX2: 256-bit SIMD rather than 512-bit. But spread across so many cores, that's still a lot of FPUs to be lighting up.

tedunangst 7y ago
That would suggest the instability might be resolved by adding threads one at a time, but does prime95 have an option like that?
- franzb 7y ago
  That's an interesting thought!
- Hello71 7y ago
  you can probably bodge it by running a bunch of copies with limited workers, then run your main copy, then kill the starters. or, you could fiddle with the CPU affinity.
- Filligree 7y ago
  That's something you can do outside of prime95; the schedutil scheduler does that by default, which I believe is why I had hard crashes with ondemand and not schedutil. (Ok, it actually ramps up frequencies slower rather than lock out cores, but the effect is the same.)
nottorp 7y ago
So basically Intel underclocks itself to do the AVX stuff while AMD underclocks itself ... less?
- wtallis 7y ago
  Core clock speeds are just some of the many variables that need to be adjusted on the fly. Everyone wants to provide as much performance as possible within the platform power limits. But a sudden change in the instruction stream can cause a big swing in a CPU core's power draw even if the core's clock speed and target voltage remain constant. And new instructions can show up a lot faster than the voltage regulators can respond. So handling transients is tricky, but it also looks like this case may be less about transients and more about a particular motherboard not being able to supply stable power for a workload that's tuned to push the limits as much as possible. Gigabyte may have missed their mark when considering what the worst-case power draw situation is that they need to design and test for.
- blattimwind 7y ago
  Intel CPUs have two different clock "ladders" so to speak, which define the range of frequencies the dynamic frequency selection (boost) can use. Specifically, as soon as an AVX2 instruction is encountered, the entire thing switches to the AVX2 clock ladder, which drops base and maximum frequencies by a GHz or so. Ladder selection is quite coarse in both cores affected; all cores for smaller CPUs; and time (I believe the y can only change like every 100 ms or so).
  - solarkraft 7y ago
    Why are the adjustments so coarse here? What's the limit to making them finer, which I assume would make sense?
    - blattimwind 7y ago
      I assume this is due to AVX2 being rare in general-purpose software, so the design "one core is running AVX2 -> switch large part of CPU to AVX2 mode" doesn't hinder use cases where you are hammering all cores with AVX2 code.
    - paulmd 7y ago
      You have to switch to a higher-voltage state to account for the voltage droop that you will cause when the AVX units fire off. https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html (specifically: https://travisdowns.github.io/assets/avxfreq1/fig-volts256-1.svg https://travisdowns.github.io/assets/avxfreq1/fig-volts256-1... ) > So one theory is that this type of transition represents the period between when the CPU has requested a higher voltage (because wider 256-bit instructions imply a larger worst-case current delta event, hence a worst-case voltage drop) and when the higher voltage is delivered. While the core waits for the change to take effect, throttling is in effect in order to reduce the worst-case drop: without throttling33 there is no guarantee that a burst of wide SIMD instructions won’t drop the voltage below the minimum voltage required for safe operation at this frequency.

tedunangst 7y ago

That would suggest the instability might be resolved by adding threads one at a time, but does prime95 have an option like that?

franzb 7y ago
That's an interesting thought!
Hello71 7y ago
you can probably bodge it by running a bunch of copies with limited workers, then run your main copy, then kill the starters. or, you could fiddle with the CPU affinity.
Filligree 7y ago
That's something you can do outside of prime95; the schedutil scheduler does that by default, which I believe is why I had hard crashes with ondemand and not schedutil. (Ok, it actually ramps up frequencies slower rather than lock out cores, but the effect is the same.)

nottorp 7y ago

So basically Intel underclocks itself to do the AVX stuff while AMD underclocks itself ... less?

wtallis 7y ago
Core clock speeds are just some of the many variables that need to be adjusted on the fly. Everyone wants to provide as much performance as possible within the platform power limits. But a sudden change in the instruction stream can cause a big swing in a CPU core's power draw even if the core's clock speed and target voltage remain constant. And new instructions can show up a lot faster than the voltage regulators can respond. So handling transients is tricky, but it also looks like this case may be less about transients and more about a particular motherboard not being able to supply stable power for a workload that's tuned to push the limits as much as possible. Gigabyte may have missed their mark when considering what the worst-case power draw situation is that they need to design and test for.
blattimwind 7y ago
Intel CPUs have two different clock "ladders" so to speak, which define the range of frequencies the dynamic frequency selection (boost) can use. Specifically, as soon as an AVX2 instruction is encountered, the entire thing switches to the AVX2 clock ladder, which drops base and maximum frequencies by a GHz or so. Ladder selection is quite coarse in both cores affected; all cores for smaller CPUs; and time (I believe the y can only change like every 100 ms or so).
- solarkraft 7y ago
  Why are the adjustments so coarse here? What's the limit to making them finer, which I assume would make sense?
  - blattimwind 7y ago
    I assume this is due to AVX2 being rare in general-purpose software, so the design "one core is running AVX2 -> switch large part of CPU to AVX2 mode" doesn't hinder use cases where you are hammering all cores with AVX2 code.
  - paulmd 7y ago
    You have to switch to a higher-voltage state to account for the voltage droop that you will cause when the AVX units fire off. https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html (specifically: https://travisdowns.github.io/assets/avxfreq1/fig-volts256-1.svg https://travisdowns.github.io/assets/avxfreq1/fig-volts256-1... ) > So one theory is that this type of transition represents the period between when the CPU has requested a higher voltage (because wider 256-bit instructions imply a larger worst-case current delta event, hence a worst-case voltage drop) and when the higher voltage is delivered. While the core waits for the change to take effect, throttling is in effect in order to reduce the worst-case drop: without throttling33 there is no guarantee that a burst of wide SIMD instructions won’t drop the voltage below the minimum voltage required for safe operation at this frequency.