4 ms·
SIMD on x86 has become a ridiculous minefield of standards. I honestly pity the compiler writers who have to navigate this nonsense. It was stressful enough try
by cmrdporcupine 4y ago
SIMD on x86 has become a ridiculous minefield of standards. I honestly pity the compiler writers who have to navigate this nonsense. It was stressful enough trying to just manage the compiler flagging in an application (music synthesizer) I wrote recently.
The core amd64 ISA is stable and that's good. I wouldn't call it elegant or nice, but it's there. The patchwork of which processor supports which SIMD extension on top of it, though, has gotten out of control and is unpleasant.
I'm sure AVX512 is great. Except as an application author I can't rely on it being there.
I am not a MacOS user or MacOS developer, but I envy the consistency of that platform now that they control their own ISA.
- paulmd 4y ago> SIMD on x86 has become a ridiculous minefield of standards. people make the AVX-512 compatibility thing more complex than it has to be. there's consumer AVX-512 and server AVX-512 and within those series there has always been strictly increasing compatibility, with a common core between them. and AMD supports the complete thing with Zen4 so there is no differentiation for them. intel didn't want to put bfloat16 ML instructions in consumer chips and people turned that into a whole thing because they like to meme about AVX-512, but that's literally the only difference. it's just not that complex a story though, if you don't turn it into a thing. intel's consumer and server teams have been operating quasi-independently and some of them are on "newer generations" than the other team. currently that's the client team that's ahead of the server team. so client got a couple gens ahead of server, and server has this one extra instruction in their versions. that's it, that's the whole deal. frankly it's actually a simpler story than SSE4 where AMD just kinda went and did its own thing for a while and Intel went and did AVX and there truly was total divergence between what the companies were doing. you can probably turn SSE into a real ugly venn diagram if you try to capture every nuance too! After all, some of them were introduced in K8, K9, Phenom... and there's lots of sub-versions of each. xeon phi is also its own thing but you'll know if you're targeting that. it has a weird 4-wide AVX thing to go along with its 4-wide SMT, and it's its own specific thing.
- hajile 4y agoBased on Wikipedia's compatibility chart, it sure looks like a minefield to me. https://en.wikipedia.org/wiki/AVX-512#CPUs_with_AVX-512 https://en.wikipedia.org/wiki/AVX-512#CPUs_with_AVX-512
- paulmd 4y agoI did this a while back, here's that chart reordered by series+date rather than just date and with the order of the fields switched around. https://i.imgur.com/idAjB1X.png https://i.imgur.com/idAjB1X.png so you can see within each server/consumer series it's monotonically increasing, and the server gets bfloat16 and client gets VPOPCNT, IFMA, and VBMI. so it's really pretty straightforward: are you targeting server, client, or both, and how recent? that will tell you how much of an overlap you've got. like is that really that bad a chart overall? it's just "newer chips have more stuff", that's not news to anyone, and barely anyone has cannon lake or even rocket lake. ice lake and tiger lake are what you want to target, mostly. Alder lake actually supports everything, but Intel pulled AVX-512 from that series... and yes, it's annoying that server got stuck in 2017 for 5+ years, ice lake-SP (cooper lake) barely has any penetration either, but the client platform moved right along and supports basically everything except bfloat16. sapphire rapids is cooper lake's successor and the first installations (HPC) were promised in like 2018... they will come online early in 2023 apparently. So they are 5+ years behind schedule because of node problems, things are basically comically bad at Intel and the server division is still not executing/iterating properly.
- janwas 4y ago> Except as an application author I can't rely on it being there. I'm surprised to keep hearing this concern. We can write code once in a vector-length agnostic way, compile it for multiple targets, and use whatever is there at runtime.
- p0nce 4y agoYes, but if you write consumer software you need to optimize more for the low-spec machines. It is those users that need the speed.
- janwas 4y agoAgreed. Do you have any example of an optimization of generic/cross-platform vector code, such that it would run better on SSE2 or SSSE3? One example might be reducing the number of live registers to avoid spilling, but you can already do that in your portable code, without necessarily requiring a separate codepath for the low-spec machines.
- dzaima 4y agoWhen writing "generic" code you still want to consider the target; for example, while it may semantically be nicer to use masking everywhere, doing so is pretty bad for perf pre-AVX-512 (especially for loads/stores, which just don't have masking pre-AVX2, and iirc AVX2's masks on AMD can still fault for masked out elements). A pretty big portion of AVX-512 is that it makes more things possible/doable a lot nicer, but that's useless if you have to also support things that don't. (another example may be using a vector-wide byte shuffle; SSE and AVX512 have instructions for that, but on AVX2 it needs to be emulated with two 16×u8 shuffles, which you'd want to avoid needing if possible; in general, if you're not considering exactly the capabilities of your target arch & bending data formats and wanted results around it, you're losing performance)
- janwas 4y ago> while it may semantically be nicer to use masking everywhere, doing so is pretty bad for perf Agreed. It's best if applications pad data structures. > iirc AVX2's masks on AMD can still fault for masked out elements). Unfortunately yes. I haven't seen a CPU use that latitude, though. > it makes more things possible/doable a lot nicer, but that's useless if you have to also support things that don't. hm, is it truly useless? I've found that even emulating missing vector functionality is usually still faster than entirely scalar code. > if you're not considering exactly the capabilities of your target arch & bending data formats and wanted results around it, you're losing performance That's fair. We're also losing performance if we don't port to every single arch we might be running on. It seems to me that generic code (written as you say with an eye to platform capabilities) is a good and practical compromise, especially because we can still specialize per-arch where that is worthwhile.
- p0nce 4y agoI sell consumer software and you can get bad reviews from people not having SSSE3 (2008). So not ony I hardly need larger vectors (vs 128-bit) vs memory throughput, but it would take 10+ years before being able to use that. The larger the SIMD vector, the more useless it is in a way that Linus describes.
- kierank 4y agoThis solves all the problems: https://blogs.gnome.org/rbultje/2017/07/14/writing-x86-simd-using-x86inc-asm/ https://blogs.gnome.org/rbultje/2017/07/14/writing-x86-simd-...
- janwas 4y agohm, nifty - templates for x86 assembly language. Doesn't solve our requirement for portability across SVE/RISC-V, though :)