13 ms·
Ice Lake AVX-512 Downclocking
- tarlinian 6y agoHopefully this behavior change will help improve AVX-512 uptake and end the somewhat ridiculous conception people have that the instructions are entirely useless. Intel's HotChips presentation on Icelake-SP also indicated that the behavior will be significantly better on server chips, but the behavior is more instruction dependent with 3 license levels and only 512-bit instructions that utilize the FMA unit being subject to downclocking (by ~15-17% as opposed to ~27-29% on SKX derived chips). (Intel Hotchips slide: https://images.anandtech.com/doci/15984/202008171757161.jpg https://images.anandtech.com/doci/15984/202008171757161.jpg)
- BeeOnRope 6y agoMakes sense - the main difference between ICL and ICL-X would seem to be 2 FMA units.
- Dylan16807 6y agoIt definitely helps. And it looks like they've reduced how often a single instruction will cause a lockup as the core shifts to a different power level. But until they've eliminated that issue, it's still scary to toss in a few AVX-512 instructions.
- FrankBooth 6y agoThere is no "lockup". Quit spreading misinformation.
- zlynx 6y agoIt is definitely a problem of defective hardware. But there are plenty of people out there with both Intel and Ryzen systems who disable every C-State and every Turbo setting because any change in power levels causes their system to crash. Lockup, bluescreen, kernel panic or just computation errors. Some of that is from overclocking. Some is from old and/or defective power supplies. Some is from motherboard VRMs. And some, like the original Ryzen 1700X, is from bad SOC-internal power management. At any rate, I have read forum posts reporting system failures caused by AVX. Either overcurrent or clockrate changes crashing it. So it seems to have support. I wouldn't call that "misinformation."
- theevilsharpie 6y ago> But there are plenty of people out there with both Intel and Ryzen systems who disable every C-State and every Turbo setting because any change in power levels causes their system to crash. There are plenty of people with both Intel and Ryzen systems that are straight-up broken and don't power on at all. Defects happen. Improperly designed systems happen. Misconfigurations happen. While those situations are unfortunate for the small percentage of people experiencing them, they shouldn't be used to judged the platform's capabilities as a whole. > So it seems to have support. I wouldn't call that "misinformation." Using anonymous, largely unverifiable anecdotes posted on web forums as evidence for a population-wide problem is a textbook case of selection bias. And if you're OK with that, let me throw in my own anecdote. All of my systems, which include: * Skylake, Coffee Lake, Haswell/Broadwell, Sandy Bridge/Ivy Bridge, and Westmere/Nehalem Intel processors, * Zen 2, K10, and K8 AMD processors, ...have been able to reliably execute supported vector instructions (SSE, AVX, etc.) for extended periods of time without any problems.
- Dylan16807 6y agohttps://travisdowns.github.io/blog/2020/01/17/avxfreq1.html https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html > There is a transition period (the rightmost of the two shaded regions, in orange14) of ~11 μs15 where the CPU is halted: no samples occur during this period16. For fun, I’ll call this a frequency transition. It stops executing for 35 thousand cycles. I call that a "lockup" "as it shifts".
- tarlinian 6y agoThis isn't the real problem though...the problem with the existing AVX-512 implementations is the "relaxation time" causes subsequent scalar code to be slow. From later in the same post: > Here, we have the worst case scenario of transitions packed as closely as possible, but we lose only ~20 μs (for 2 transitions) out of 760 μs, less than a 3% impact. The impact of running at the lower frequency is much higher: 2.8 vs 3.2 GHz: a 12.5% impact in the case that the lowered frequency was not useful (i.e., because the wide SIMD payload represents a vanishingly small part of the total work). Interestingly enough, this is another feature that is supposed to have been improved on server Icelake. The frequency transition halt time is now pretty much negligible. The "core frequency transition block time" goes from ~12 us on CLX (similar to the number quoted above) to ~0 us on ICX. (Slide with frequency transition info: https://images.anandtech.com/doci/15984/202008171754441.jpg https://images.anandtech.com/doci/15984/202008171754441.jpg)
- Dylan16807 6y agoEither way, it's a big problem that certain single instructions can cause this transition. When the transition is based on a usage threshold of heavy instructions, it's not so bad. And with this revision, more of the transitions are based on threshold. But there are still some instructions that cause an immediate frequency change, if I'm reading the articles right.
- tarlinian 6y agoNo...the whole point is that the single instruction induced halt for downclocking isn't a real issue. Even in the pathological case where you insert a single instruction spaced 760 us apart in order to induce the maximum number of clock shifts, the total performance degradation due to the clock halts is only 3% (the frequency drop that is induced by the use of these instructions has a much larger impact.) Furthermore, on Icelake-SP, the halted time due to frequency transitions is supposed to go to 0, which makes this aspect of the problem entirely irrelevant. Yes, if you insert a single 512-bit FMA that runs every so often in your code you will get a 15% performance hit from the lower frequency, but that's much less likely than the old case where people who were trying to use AVX-512 for memcpy and the like would slow down scalar code.
- The_rationalist 6y agoI generally agree but https://www.phoronix.com/scan.php?page=news_item&px=LLVM-Clang-10-AVX512-Change https://www.phoronix.com/scan.php?page=news_item&px=LLVM-Cla...
- gameswithgo 6y agocompilers are not at a point where they do a great job of leveraging SIMD in general, and definitely not where they leverage AVX-512, but hand written intrinsics with AVX-512 can attain amazing performance.
- CoolGuySteve 6y agoI don't think the book is closed. Thermal and TDP downclocking are still present. It would have been nice to see the vcore and thermal values graphed as part of the benchmark. Do they increase faster for AVX-512 vs the other instruction sets? I've had problems in the past with Sandybridge, where AVX hit thermal throttles before SSE. I ended up having to disable them in my build because of it. Presumably, the same behaviour would be seen here now that the vector unit is wider and there are more densely packed transistors flipping.
- wmf 6y agoDoing work faster is almost always going to consume more power and if you're already at the power or temperature limit (which is how most CPUs/GPUs operate now) then the frequency will have to reduce. This isn't automatically bad; ultimately what matters is performance. Did you really see lower performance with AVX than with SSE?
- CoolGuySteve 6y agoThe problem is that the downclocking affects other cores. So a performance improvement in this one task can hurt performance on other threads, which is what happened to me.
- wmf 6y agoYeah, I think Intel is trying to fix this with Speed Select but it's complex enough that no one will probably use it.
- gameswithgo 6y agoit does bot affect other cores on all cpus, most newer ones it doesn’t
- BeeOnRope 6y agoAt least on most new cores, the frequency is per-core. This isn't true on, for example, some Skylake client cores - but these don't have much SIMD related downclocking either.
- ksec 6y ago> end the somewhat ridiculous conception people have that the instructions are entirely useless. I don't think anyone ever said that. What people are really saying, AVX-512; in its current form, limited by Clockspeed, Thermals, and most importantly market segmentation. And given all these three things are only slightly improved in the current 3 years roadmap render the instruction useless. ( But latest HotChip information seems to suggest they are well aware of it. So Roadmap can change ) And even more importantly that is in reference to current stack of Intel issues.
- quietbritishjim 6y agoI think some subtlety has been lost on me. The parent comment said that people had the "ridiculous conception ... that the instructions are entirely useless". Then you said that no one thinks that, followed by some stuff I'll admit I didn't really understand, leading to the conclusion "... render the instruction useless". Are you disagreeing with the parent comment or not? Are you maybe saying that the instruction is useless in chips that are in computers right now but will be useful in chips that will be released in future?
- paulmd 6y ago> I don't think anyone ever said that. Yeah, Torvalds said pretty much exactly that, in his typically boisterous form. That it is "only useful in special cases", "only exists to make Intel look better on benchmarks", and that "the transistors would be better spent elsewhere". https://www.phoronix.com/scan.php?page=news_item&px=Linus-Torvalds-On-AVX-512 https://www.phoronix.com/scan.php?page=news_item&px=Linus-To...
- Const-me 6y ago> the instructions are entirely useless I mostly agree, but for me the frequency scaling is almost irrelevant. Here’s a link, expand “Other settings” and you’ll see: https://store.steampowered.com/hwsurvey https://store.steampowered.com/hwsurvey If you implement AVX2, the code will run on 76% of PCs. When you want good performance, such optimization is a good use of limited resources. AVX512 is only at 0.42%. That’s why it’s useless to implement AVX512 except for servers where you have full control over hardware. And with current state of Intel, not necessarily a good idea even for servers, it’s quite possible in a couple of years you’ll want to switch to AMD.
- killerstorm 6y agoEh? Compilers can generate two versions of code -- AVX and non-AVX - and dynamically choose depending on CPU capabilities. It was this way for ages.
- Const-me 6y ago> Compilers can generate two versions of code From what? Automatic vectorization is very limited, it’s only good for pure vertical algorithms. > and dynamically choose depending on CPU capabilities Only Intel compiler does that, the rest of them don’t. Sometimes I ship multiple versions of binaries. Other times I do runtime dispatch myself, it’s only couple lines of code, __cpuid() then cache some function pointers, or abstract class pointers. It’s even possible to compile these implementations from the same C++ source, using templates, macros, and/or something else.
- paulmd 6y agoGiven the process improvements in Tiger Lake - I wonder if this improves further, or at least all levels become somewhat faster?
- BooneJS 6y agoAt Hot Chips 32 this week, Intel mentioned that Tiger Lake Xeon with Sunny Cove core would only downclock if AVX-512 usage hit TDP limits.
- rbanffy 6y agoI wonder how many clock cycles that'd take ;-)
- BooneJS 6y agoThey made it sound like some instructions were more power hungry than others. The impression I got is that the unit can run some kinds of streams without reduction Of clock.
- donor20 6y agoThat is MUCH better it seems? Because then you don't randomly throw away lots of performance because some 512 items hit so there is less risk in using 512.
- williadc 6y agoTigerLake client uses Willow Cove cores. I would expect IceLake Xeon to use the Sunny Cove cores (same as IceLake client) and TigerLake Xeon to use Willow Cove, but I haven't seen evidence that is the case.
- ezekiel68 6y agoWell, other than precedent. We have never seen Intel suddenly surprise the market by using a different core type in a chip family's Xeon SKUs than was used in the mobile and desktop SKUs. And part of that (I conjecture, having been in the rodeo with them since 486 days) is that they like to use the consumer versions of the chips to suss out errata to be fixed before the Xeons are taped out and released. It is true that some features do differ, such as support for ECC RAM and perhaps activation of TSX or extra FMA unit, but these are really just variations on the same core family.
- 6y ago
- deleted 6y ago[deleted]
- jiggawatts 6y agoOne thing I really want to know is whether SQL Server's new vector-accelerated "Batch Mode" uses AVX2 only or if it also has AVX-512 code paths? I'd like to be able to recommend the right CPU to customers, but there just isn't any information out in public about this...
- rbanffy 6y agoThere are a lot of different factors that can affect performance. I'd advise you to always benchmark.
- jiggawatts 6y agoI can't benchmark with a CPU I don't have, and I can't advise a customer to go off and buy a $50K server just for a "quick benchmark". Even if I were to, say, test with some cloud VMs, even then there are confounding issues. The different VM size categories aren't just different in the CPU type only, there are other differences that'll make the benchmark difficult to interpret. Memory type, throttling, HT on/off, etc... Why is it so difficult for Microsoft to simply say "AVX-512 supported" somewhere in their documentation? This is like every TV vendor saying "HDMI" instead of "HDMI 2.1" or whatever. Just because the port looks the same doesn't mean that they're identical! Versions matter.
- rbanffy 6y ago"AVX-512 supported" doesn't say much either. They used to publish reference designs with specific servers and components.
- gameswithgo 6y agoyou could rent an avx-512 vm in the cloud for a few hours?
- foota 6y agoThis is a terrible suggestion, but you could always attach a debugger to find out
- rbanffy 6y agoDid the Xeon Phi also downclock when using AVX-512?
- th3typh00n 6y agoI haven't seen any numbers on that but there's literally zero reason to run a Xeon Phi without using AVX-512, so I'd assume no design considerations were taken to optimize the clock frequency for a non-AVX-512 use case.
- shaklee3 6y agoThere's almost zero reason to run a phi in general.
- rbanffy 6y agoI always suggested making software run well on a Phi would be valid research for making it run well on future Xeon Scalable and Core i9. If you can make your code so parallelizable it runs well on a Phi, it'll run extremely well on future CPUs because clocks won't get much higher, but core counts will.
- shaklee3 6y agoGood point, but from what I recall the phi had two different memory types, and you had to specify which you were targeting. This didn't necessarily translate to the server CPUs.
- dragontamer 6y agoThe Phi was an interesting computer. AVX512 on 60 cores back in 2015 was pretty nuts. CUDA wasn't quite as good as it is today (there have been HUGE advancements in CUDA recently). These days, we have a full-fat EPYC or Threadripper to use, and even then its only 256-bit vector units. CUDA is also way better and NVidia has advanced dramatically: proving that CUDA is easier to code than people once thought. (Back in 2015, it was still "common knowledge" that CUDA was too hard for normal programmers). Intel's Xeon Phi was a normal CPU processor. It could run normal Linux, and scale just like a GPU (Each PCIe x16 lane added another 60 Xeon Phi cores to your box). It was a commercial failure, but I wouldn't say it was worthless. NVidia just ended up making a superior product, by making CUDA easier-and-easier to use.
- jojobas 6y agoDoes the word "license" actually mean "even though the hardware is capable, you didn't pay enough to not downclock" or something else here?
- justaguy88 6y agoIf it does that's terrible
- jojobas 6y agoSun or SGI used to do approximately that. They sold downclocked/locked core chips that could later be unlocked for a fee (from memory, time-limited fee).
- segfaultbuserr 6y agoIntel did that on low-end desktop CPUs for once, but soon discontinued it after criticism. * Intel Upgrade Service https://en.wikipedia.org/wiki/Intel_Upgrade_Service https://en.wikipedia.org/wiki/Intel_Upgrade_Service
- twoodfin 6y agoA Google search doesn’t do much to define the term, but from context I believe it means a frequency allowance that’s baked into the CPU rather than one that’s enforced dynamically depending on power/thermals. I doubt Intel has validated the hardware outside its “license”. This appears to be a much earlier design constraint than anything they could reasonably expect to bin.
- jeffbee 6y agoThe license is an open-loop, predetermined safe operating area for AVX. The other limits, like RAPL and temperature, are closed-loop control systems. I think we can imagine that the static SOA exists for all these parts, but in the newer part it's so large that it doesn't tread much on the territory of the closed-loop systems. Eventually the SOA will be large enough to be irrelevant in practice.
- throwaway_pdp09 6y agoI don't know this area, but to clarify (or mess things up, depending on my understanding), the downclocking is for FP-heavy work. AFAIU it doesn't occur for any integer work (maybe an exception for div?), and 512-wide simd integer instructions could be very, very useful. Pls correct me if wrong.