5 ms·
I was excited for AVX512 long ago but I've since heard that if you are jamming AVX512 instructions to every core you get a forcibly lower clockrate. In practice
by jchw 8y ago
I was excited for AVX512 long ago but I've since heard that if you are jamming AVX512 instructions to every core you get a forcibly lower clockrate. In practice this sounds like it'd suggest using an AVX512 algorithm could actually be slower even when it's faster. If that's the case, I wonder what kind of performance gain you'd have to hit to beat a scalar or SSE-based vectorized algorithm.
- zwaps 8y agoDepends on the task and CPU, but especially on smaller machines the slowdown is not so large compared to potentially huge performance gains, ain't it?
- zamadatix 8y agoDepends which AVX512 instructions and how quickly you expect to bounce between AVX and scalar workloads. Generally if you can expect to spend a significant portion of your algorithm time in vector optimized code then the small clock drop isn't enough to erode the throughput gains.
- shereadsthenews 8y agoYou also get a lower clock rate from running scalar code on many cores at once. Use of the 512-bit unit only makes the coefficient different. That said, the biggest mystery in this article is why you'd ever want to remove all whitespace from text. Why is that useful?
- taspeotis 8y agoI dunno, spam filtering? sig nup now for che ap viag ra And then search for key words like “viagra.”
- jasonzemos 8y agoIf one could remove spaces from JSON to create a so-called "Canonical JSON" one could obtain the same digest hash from the same data (even in combination with the hardware accelerated hashing offered by AVX!). Admittedly this is a strange case but I've run into it.
- jandrese 8y agoIsn't this a problem if you're fed these two inputs: {mystring:"Hello World!"} {mystring:"Hello World !"} They aren't the same JSON.
- marvy 8y agoalso: {mystring:"HelloWorld!"}
- amenghra 8y agoTypically, when you want a canonical JSON you don't want to touch strings and you want to sort maps.
- kijin 8y agoI don't usually want to remove all whitespace from text, but I do often want to remove weird Unicode whitespace characters (there are dozens of them), control characters, excessively repeated diacritics, and unbalanced bidirectional markers from user-submitted content. The code will have to be a lot more complicated in order to achieve this, and I'm not even sure I could use AVX512 to replace the ugly regex I'm currently relying on. But the idea is close enough, so maybe whitespace removal is meant to serve as a simple proof of concept.
- vardump 8y agoWhen your starting point is a regexp, I think you have a few layers of lesser abstraction to explore for optimization opportunities before trying to implement something in AVX512...
- pkaye 8y agoMaybe for the International Obfuscated C Code Contest.
- tgtweak 8y agoCompression routines regularly track then remove whitespace. Given the commonplace occurrence of both compression serverside and decompression clientside on the web, this improvement could be notable in every day computing.
- wmu 8y agoThis is a problem with base64 decoding. While there are vectorized algorithms for decoding and encoding, they do expect a text without spaces or newlines. However, by design, in emails the base64-encoded data must be split to lines not exceeding some length.
- throwaway2048 8y agoThere is also a huge delay for the core to shift into AVX512 mode, so interleaving it into regular non avx512 code can be a huge performance hit.
- BeeOnRope 8y agoWell if by huge you mean 10ish microseconds, then yeah. There is also hysteresis built in so the transitions can only occur about once per millisecond. This puts a hard bound on the cost of the transitions themselves of about 1%. So no, I don't think the transitions are a big problem for any throughput case, although if you are counting microseconds for some type of low latency thing then sure it can matter. The bigger problem is usually the frequency reduction.
- Twirrim 8y agoThe amount of impact is heavily dependent on the class of CPU you use from Intel (gee, thanks Intel. Awesome way to make optimising code a pain in the arse). Bronze you probably really don't want to do AVX512 unless it's all heavily AVX512. Silver is better, but not great (which is what Cloudflare were using and ran in to). Gold or Platinum you're not likely to see much degradation from using AVX-512, even sparsely.
- BeeOnRope 8y agoAre you talking specifically about the transition penalties which I was referring to? I am not aware of any strong variance in this behavior based on CPU class, and I would be surprised if it existed. I've always measured around 8 to 10 us for these transitions (there may be both a real time and "clock cycles" component to the transition time so it might vary a bit when measured in real time based on the CPU frequency). It sounds to be like you are talking more about AVX-512 downclocking. In any case, if your algorithm can be make heavy and effective use of AVX-512 it's going to be a win on almost any class of chip since you are looking at something like a 2x baseline speedup and more in many cases. If not, then yeah, proceed with caution.
- gwern 8y agoLemire has criticized this: https://lemire.me/blog/2018/08/13/the-dangers-of-avx-512-throttling-myth-or-reality/ https://lemire.me/blog/2018/08/13/the-dangers-of-avx-512-thr... OP doesn't seem to show any major problems in his microbenchmarking, which is a start.
- wmu 8y agoTBH I didn't think about throttling. My (mis)understanding is that a single program which completes in a fraction of second on an almost-idle machine doesn't suffer from AVX512 throttling. Daniel has reviewed this problem in series of posts: - https://lemire.me/blog/2018/08/15/the-dangers-of-avx-512-throttling-a-3-impact/ https://lemire.me/blog/2018/08/15/the-dangers-of-avx-512-thr... - https://lemire.me/blog/2018/08/25/avx-512-throttling-heavy-instructions-are-maybe-not-so-dangerous/ https://lemire.me/blog/2018/08/25/avx-512-throttling-heavy-i...
- BeeOnRope 8y agoOn most chips released to date, AVX-512 always causes a lower frequency (the so-called L1 license) as soon as any instruction uses it: basically the chip needs to drop from the L0 (fastest) license to L1 (middle) before it even starts any AVX-512 (probably to power up the upper lanes of the ALUs, register file, etc). Below L1, there is an even lower frequency tier, L2, which behaves as you say: it doesn't drop to L2 unless there is a sustained use of heavy AVX-512 instructions. On the Cannonlake chip you used, however, the i3-8121U, I don't measure any AVX-512 throttling at all - regardless of what instructions are running, up to and including dense 512-bit FMAs, the chip is running at the full 3.2 GHz (1 active core) or 3.1 GHz (2 active cores) turbo speed at all times. This behavior was common in the past on client parts: e.g., most Skylake client parts run at full speed when using 256-bit AVX/AVX2 operations, but server parts and some higher-end desktop parts based on them had varying speed tiers. All of the initially released SKX (Skylake X) parts were based on the Skylake-SP server part and had the speed tiers (downclocking) for AVX-512, but maybe many or most future client parts won't if CNL is any indication.
- wmu 8y ago