8 ms·
Gathering Intel on Intel AVX-512 Transitions
- BeeOnRope 7y agoAuthor here, happy for any feedback or to answer any questions.
- shaklee3 7y agoNo questions, but really enjoyed your blog, and your comments on this website over the past few years.
- ksec 7y agoJust want to say Thank You. Do you know anything about on AMD's side of things? >Note: For the really short version, you can skip to the summary, but then what will do you for the rest of the day? Spending rest of the day on HN. /s
- BeeOnRope 7y agoI don't know specifically, e.g. if there are any such pauses on Zen. Also, Zen doesn't yet support AVX-512 so a big possible source of variation is moot. I don't know if any AMD chip has ever had different turbo speeds for any ISA. It should be noted that even without that, any chip can still run slower with heavier instructions because they hit some other limit: thermal, TDP, current, etc. AMD has used an interesting "adaptive clocking" scheme since steamroller, and apparently this is still in effect in Zen: https://www.realworldtech.com/steamroller-clocking/ https://www.realworldtech.com/steamroller-clocking/ This handles the same type of voltage droop worst case that Intel apparently handles by dispatch throttling. It would be interesting to test it, since the clock elongation should be visible when you measure instruction timing relative to a clock not affected by the adaptation.
- celrod 7y agoOn the desktop chips (x299) it's easy to adjust all the clock speeds in the bios. If the workloads I'm most interested in are all avx512-heavy (why I bought x299 instead of threadripper), do you think there'd be a reason to set the clock speeds to be equal, regardless of ISA? That is, if I currently have 4.6/4.3/4.1 GHz no-avx/avx(2)/avx512, when might it be worth setting all three of these to 4.1 GHz? I suspect "never" is the answer? I have the impression that Zen's clocking algorithm is much smarter than Intel's heuristic approach.
- BeeOnRope 7y agoI don't think it makes sense, because the maximum penalty due to the tranitions is fairly low (~30 us out of 650 us, and only under pretty much a malicious load that transitions at exactly the right points), and mostly you want the higher frequencies when you can get them: they quickly overwhelm the small transition periods. Also, someone indicated to me in private correspondence that even when the frequencies are manually set so no transition takes place, the throttling periods may still take place (which makes sense since the required voltage may still be higher).
- oddity 7y agoIt’s a real shame avx-512 has so many eccentricities when it’s a much nicer ISA than anything before it (in x86 land). I would almost prefer a more predictable, high-latency decomposition into 4x128 wide uops over what we have now.
- zwegner 7y agoBut what about instructions like vpermi2b, which does 64 parallel 128 byte table lookups? The AVX shuffle instructions were hamstrung by the split into two 16-byte halves...
- BeeOnRope 7y agoYeah, the shuffles are awesome in AVX-512.
- dman 7y agoIntel really botched the launch of AVX-512. By not making it available on client, and even on server having it be available on select chips meant that no one code coded / optimized for it. If you are proposing new instructions make sure they are widely available.
- xscott 7y agoAdditionally, it looks like each subset of AVX-512 past the foundation is an optional feature and needs to be tested for. With only a few exceptions (usually as a result of bickering between Intel and AMD), previous ISA extensions implied you had everything that came before it too. In practice this means you could pick a few entire subsets: Legacy+SSE2 is always there for 64 bit, maybe test up to SSE4.2 for another subset. Maybe switch everything from REX to VEX if it has AVX and AVX2. That's effectively three back ends, which is manageable. With AVX-512, everything beyond AVX-512F is a la carte, and that adds unwanted complexity for instruction selection in a compiler. Just look at all the separate AVX feature flags from CPUID: https://en.wikipedia.org/wiki/CPUID#EAX=7,_ECX=0:_Extended_Features https://en.wikipedia.org/wiki/CPUID#EAX=7,_ECX=0:_Extended_F... Between the performance problems and complexity, I think it'll be a while until AVX-512 is attractive.
- zone411 7y agoThe latest Microsoft Visual C++ has an option to generate AVX-512 code https://docs.microsoft.com/en-us/cpp/build/reference/arch-x64?view=vs-2019 https://docs.microsoft.com/en-us/cpp/build/reference/arch-x6...
- Flicki 7y agoNice work as usual BeeOnRope Now if only I could actually use avx512 in a desktop, been waiting what feels like 5+ years..
- deleted 7y ago[deleted]
- haecceity 7y agoWhat is this intended to be used for? This [1] article mentions compression, ML, scientific computing. Wouldn't people rather use GPU for those workloads though? [1] https://devblogs.microsoft.com/cppblog/microsoft-visual-studio-2017-supports-intel-avx-512/ https://devblogs.microsoft.com/cppblog/microsoft-visual-stud...
- yvdriess 7y agoOffloading to the GPU has a significant cost that needs to be amortized. DNNs work well because very little is moved over the PCIe bottleneck, and inputs can be buffered. Take modern compression, let's say H265, has a complex control flow combined with highly vectorizable work. I'm unsure where the threshold lies today, but you need a significant amount of work before offloading to and reading back from the GPU becomes interesting on a beefy Xeon.
- shaklee3 7y agoThis site seems to lock up my phone, but desktop is fine. Anyone else have that issue?
- BeeOnRope 7y agoThere are several large SVGs, I wonder if that is the issue. Can I ask what type of phone you have? Are you willing to help me diagnose the issue?
- shaklee3 7y agoI have a really old Nexus 6p. I'm definitely willing to help, but I bet it's just part of having a really old phone.
- BeeOnRope 7y agoWell that happens to be a phone I have too, although the battery is pretty dead so it's hard to use. It's still a pretty powerful phone though, weird that it would have issue rendering the page. I was able to reproducing freezing and hanging even on my Pixel 3, so I can probably look into it myself. Again, I have to guess the large SVGs are to blame.
- shaklee3 7y agoAwesome. Thanks!
- totalZero 7y agoAs a former owner of that model of device, whose phone bricked itself irreparably after updating to Oreo, I'm surprised it hasn't been destroyed by the bootloop of death. I'll never buy a Google hardware product again after the runaround they gave me.
- BeeOnRope 7y agoI managed to avoid that problem although I read about it plenty. It was my first Android phone, but I liked it except for the battery life issues, which were, frankly, brutal. Regularly shuts down at 50% battery. To replace the battery you are pretty much guaranteed to break parts of the camera assembly (the cover). I'm not sure how blame should be apportioned between Google and the manufacturer though.
- stolk 7y agoWould disabling “Intel Turbo Boost Technology” be advised to avoid this? If it never uses turbo, it should not suffer the transitions?
- BeeOnRope 7y agoNo, the transitions occur even without turbo. In fact, the chip I tested on has no turbo at all, just 3.2 GHz nominal speed. Also, disabling turbo would probably be a massive over-reaction, unless you really care out about 99.9th latency or something: the impact of these transitions is small (at worst a few %), while the benefit of turbo is large: 10s of %.