8 ms·
Transcoding Unicode with AVX-512: AMD Zen 4 vs. Intel Ice Lake
- snvzz 4y agoAVX-512 is wider, but also needs special instructions to leverage the hardware. This is unlike RISC-V V extension, where the same code will run and utilize the hardware, regardless of vector unit width.
- jonstewart 4y agoIs there a RISC-V chip with SIMD available for purchase with roughly comparable price/performance to current Intel/AMD offerings?
- snvzz 4y agoIf you specifically prefer SIMD over Vector, Andes has offerings with draft P extension. If you otherwise want vector (V extension), "right now" would limit you to pre-1.0 V extension implementations. If you need to license hardware IP, there are several very high performance implementations as of RISC-V Summit[0]. Actual hardware will pop up throughout 2023. 0. https://www.youtube.com/@RISCVInternational/videos https://www.youtube.com/@RISCVInternational/videos
- moonchild 4y agoMost of what's interesting about avx512 is the new instructions; the wider vectors are just icing on the cake. You would need to rewrite your code regardless.
- readonlybarbie 4y agoNoob question- must one write avx512 assembly directly by hand, or is this something a c compiler would do for you?
- habitue 4y agoPotentially both. Most compilers have vectorization optimizations if you compile for an architecture that supports it. However, a lot of software is compiled on one machine to be run on potentially many possible architectures, so they target a very lowest common denominator arch like x86-64. This will have some SIMD instructions but (I don't think) AVX-512. So if a developer wants to ensure those instructions are used if they're supported, they'll write two code paths. one path will explicitly call the avx512 instructions with compiler intrinsics and then the other path will just use the manual code and let the compiler decide how to turn it into x86-64 safe instructions.
- readonlybarbie 4y agothanks for that! so it sounds like, if i purchase a chip that supports avx512, and run an operating system and compiler that supports avx512, i can write "plain old c code" with a minimal amount of compiler arguments and compile that code on my machine (aka not just running someone else's binary). and then the full power of avx512 is right there waiting for me? :)
- janwas 4y agoA compiler turning C(++) code into SIMD instructions is called "autovectorization". In my experience this works for simple loops such as dot products (even that requires special compiler flags to enable FMA and reorders), but unfortunately the wheels often fall off for more complex code. Also, I haven't seen the compiler generate the more exotic instructions.
- brrrrrm 4y agotypically if it's available, compilers will use the avx512 register file. This means you'll see things like xmm25 and ymm25 (128 and 256 bit registers) and those are avx512 only. However, compilers using 512-wide instructions is kinda rare from what I've seen
- readonlybarbie 4y ago
- stabbles 4y agoI wonder to what extent compilers even emit avx512 instructions apart from the common ones (load, store, shuffle, arithmetic) in case you don’t want to manually optimize for sse / avx / avx2 / avx512.
- celrod 4y agoHow much code is compiled with `-march=native` or function multiversioning? I would guess the percentage is relatively small, at least when it comes to distributed binaries. Compiler autovectorizers also aren't very good at producing fast AVX512 code, so most of the benefit would probably come from using optimized libraries like Intel's MKL or simdjson.
- derefr 4y ago> How much code is compiled with `-march=native` Any installation of Gentoo is, presumably. (Otherwise, what's the point of compiling it all yourself?) More interestingly, possibly all OEM firmware-installed copies of ChromeOS are -march=native builds as well, given that ChromeOS is based off of a Gentoo upstream.
- celrod 4y agoTrue. I have never gone down the Gentoo rabbit hole. Might be fun to try sometime, but I'd seriously doubt that the time spent compiling would be won back from better performance. Clear Linux is probably a more practical alternative. I used it a couple years ago, and found that they had a lot of avx2 and avx512 versions of random libraries built, with the appropriate ones presumably being loaded based on the hardware. Random glibc math function calls, for example, were much faster on Clear Linux than Arch or Fedora. But development of Clear seems to have stopped, libraries like llvm aren't being updated anymore so the toolchains are outdated. I'd wanted to avoid the blood and sweat of managing my own toolchains, and ironically being on bleeding edge distros (Arch,Fedora,etc) was the way to keep that to a minimum. Next time I reinstall an OS, I'll look at Clear again. Or maybe Guix or Nix. Or maybe use spack for package management on top of some other distro.
- ryao 4y agoUsing the RISC-V V vector instructions means that the underlying hardware vector width can change and the code will automatically take advantage of the larger width. That said, many of the avx512 instructions are simply extended width AVX2/avx2 instructions. The interesting things about it are really the increased width and the additional registers. Not many of the new instructions that are not bitwidtg extended versions of the old ones are particularly interesting since Intel had already implemented most of the interesting things for smaller vector widths.
- kolbe 4y agoI've only scratched the surface of the avx512 instructions, but they are much more broad and useful. Masked gather, scatter, double precision exp and mantissa extraction, and floating point to integer conversions are all new and all proving useful to me.
- chris6f 4y agoVector length agnostic programming has its own share of problems. I'm not familiar with the RISC-V V extension, but I assume it's similar to ARM's SVE. There's a good critical look at SVE and VLA here: https://gist.github.com/zingaburga/805669eb891c820bd220418ee3f0d6bd#fixed-width-vector-workloads-on-variable-width-vectors https://gist.github.com/zingaburga/805669eb891c820bd220418ee...
- snvzz 4y agoV extension and SVE2 are very different. Here is a quite recent introduction to RISC-V Vector[0]. 0. https://erikexplores.substack.com/p/grokking-risc-v-vector-processing https://erikexplores.substack.com/p/grokking-risc-v-vector-p...
- janwas 4y agoI'm curious why you say they are very different? From where I sit, RVV also supports mask-like predication, and adds two concepts: LMUL (in-HW unrolling of each instruction) plus the ability to limit operations to a given number of elements. The former is nifty, though intended for single-issue machines, and the latter seems redundant because masks can also do that.
- brigade 4y agoVectorizing text handling is likely to use generalized register permutes (and I spy some _mm512_shuffle_epi8 here), which are the bugaboo in length-agnostic SIMD. Fundamentally, the maximum index you can read from in a register depends on the register size. So yeah, even in RISC-V V, vrgather has explicitly different per-element operation depending on VLMAX, which obviously depends on the HW's VLEN. So depending on the table size, you have to assume constraints on VLEN or execute different permute sequences.
- sylware 4y ago512bits is 64 bytes, a cache line on x86_64.
- svnpenn 4y agoI saw this once before, and both times, it's pretty shocking. Is this really something that needs to be inside the CPU itself? I don't want my CPU doing this. I would rather just take the performance hit and keep the CPU "dumb".
- wtallis 4y agoAre you saying you don't want your CPU to have SIMD/vector capabilities, or you want it to have a limited SIMD instruction set without the extra flexibility that AVX-512 brings, or have you wildly misinterpreted the headline (twice?) to assume that AVX-512 adds special-purpose instructions for Unicode conversions?
- tedunangst 4y agoAny instructions beyond NAND are clearly bloat.
- stingraycharles 4y agoI think the parent must not understand what SIMD/vector operations are, and probably thinks there is native Unicode support in the CPU. At least, that’s my most favorable interpretation of his critique.
- titzer 4y ago> keep the CPU "dumb". a 240+ entry reorder buffer has entered the chat.
- ClumsyPilot 4y ago> keep the CPU "dumb" Users started dumb, develooers became dumb, now you want CPUs dumb too? Who will deal with the consequences??
- dotnet00 4y agox86 CPUs have been essentially black magic for a while now. If anything, what's described here is on the dumber side for things an x86 CPU from the past decade or two is doing behind the scenes to operate as fast as it does.
- ajross 4y ago> These results suggest that AMD Zen 4 is matching Intel Ice Lake in AVX-512 performance. Given that the Zen 4 microarchitecture is the first AMD attempt at supporting AVX-512 commercially, it is a remarkable feat. Um... Ice Lake shipped over three years ago. I mean, there's a real question as to whether or not "senselessly wide SIMD" is a good or bad feature in a datacenter part, and how or whether AMD should attempt to implement it and within which market sectors. And surely there's discussion to be had about the design tradeoffs to be made chasing after this nonsense. But, no, performance parity with chips that are nearing end of life has to be viewed as table stakes here. It's certainly not "remarkable".
- sakras 4y agoThe Ice Lake chips being benchmarked against are server chips, while the 7950X Zen 4 chip used is a consumer chip. So while Ice Lake has been out for a while, it’s also several times more expensive. It’s also worth noting that it took Intel several generations of trying AVX512 to get it working well, so AMD doing it first try really is impressive (even if they did cheat by just having AVX512 be double pumped AVX2).
- moloch-hai 4y agoDouble-pumped is fine: what matters is the new semantics implemented that cannot be expressed efficiently in previous ISAs.
- Dalewyn 4y agoWhether it's comparing latest gen architectures against old architectures or comparing consumer CPUs against enterprise CPUs (or an unholy combination of both), it's all insincere hogwash. Comparing apples to oranges is not how you determine how good a peach is.
- adrian_b 4y agoUntil the Intel Sapphire Rapids server CPUs will be launched later this month, the Ice Lake/Tiger Lake/Rocket Lake microarchitecture is the best AVX-512 implementation available from Intel, after 15 years since this ISA has been publicly disclosed. Those differ only in the clock frequency and in a few details that are irrelevant for this particular benchmark. So the comparison normalized by clock frequency, as done here, is legit, no other better comparison is possible for now. Even if someone had a Sapphire Rapids sample, they would not be allowed to publish any benchmark yet.
- eddiewithzato 4y agoWonder if this piece by Linus about AVX512 is still relevant https://news.ycombinator.com/item?id=23809335 https://news.ycombinator.com/item?id=23809335
- stingraycharles 4y agoIt’s not, as long as you’re using a recent CPU architecture. I think the slowdown problem was mostly related to Intel’s first version of AVX512, and the underlying issue has since been addressed (at least to the point that it’s not nearly as much a problem as it used to be). This is also why it’s impressive that this is only AMD’s first attempt: it appears to work really well, where it took Intel multiple attempts to get it working well.
- adgjlsfhk1 4y agoAVX-512 one major problem and a bunch of minor ones. The major problem is that most computers still don't have it. Intel tried to segment their lineup and only put AVX-512 in their high end server CPUs for the first 2 generations that had it, but as a result normal programmers didn't have access to it, compiler devs didn't have access to it, and users didn't have access to it. As a result, most compilers don't do a good job generating AVX-512 code, and most programmers think AVX-512 isn't useful. AVX-512 is great. The new instructions are incredibly useful for a wide variety of applications, but the fragmentation and segmentation by Intel has made it a total mess.
- janwas 4y agoBefore we consider the technical merits, let us note that Linus admits to "irrational hatred" and "bias" on this topic. It's also not clear to me how much experience he has developing and testing AVX-512. As others mentioned, throttling is basically nonexistent on Icelake (and AMD Genoa). It can hurt on Xeon Silver (so let's not use those?) and if you only sporadically use SIMD instructions (again, don't do that). I claim that just about any reasonable code which sustains SIMD instructions over several milliseconds would still be a net win even with throttling. Un-nuanced concerns about throttling are outdated and unhelpful. Perhaps I'll write up a paper on this.
- deleted 4y ago[deleted]
- qalmakka 4y ago> However, we have two popular Unicode formats: UTF-8 and UTF-16 The fact we are still using UTF-16 still irks me to this day. UTF-16 (which is actually two different encodings, not one, hence the need for a BOM) is basically a way to salvage all those platforms that hurried on the UCS-2 (aka, the original "UNICODE") bandwagon in the '90s hoping that by just doing s/char/wchar_t/g all their internationalization problems would be solved. It did not go well, to say the least. UTF-16 and 32 are objectively worse than UTF-8, because they still are multibyte encodings, you still have to do normalization, ... while also having to deal with "char16_t" and the likes (I will not enter into the whole "TCHAR" fiasco). Spoiler alert, the world is still full of allegedly "UTF-16 compliant" platforms out there are not, indeed, UTF-16 compliant, they just use 16 bit chars and hope for the best. The whole idea "1 char = 1 character" is arguably a terrible idea in Unicode, though. You not only can have multibyte characters, but you can also have multirune characters, where multiple codepoints are normalized into a single displayed character (just think about ` + e = è). It's a mess and it's bound to be broken, and there's no real way to "fix that up" - ASCII's assumption of "1 value = 1 char" was the broken concept here, and it unfortunately flawed how every developer (me included) thinks about strings. Sigh.
- dralley 4y agoUCS-32 is at least directly indexable, even though it's ludicrously space-inefficient.
- natanbc 4y agoOnly in codepoints, but it still has the problem GP mentions of ` + e = è being two codepoints (so two elements in UCS-32), but being logically one character https://manishearth.github.io/blog/2017/01/14/stop-ascribing-meaning-to-unicode-code-points/ https://manishearth.github.io/blog/2017/01/14/stop-ascribing...
- qalmakka 4y agoThis, it's pointless to have char32_t if you still need to pull several megabytes of ICU to normalize the string first in order to remove characters spanning over multiple codepoints. UTF32 is arguably dangerous because of this, it's yet another attempt to replicate ASCII but with Unicode. The only sane encoding out there is UTF-8, and that's it. If you have to always assume your string is not really splittable without a library, you won't do dangerous stuff such as assuming `wcslen(L"menù") == 4`.