8 ms·
AVX10/128 is a silly idea
- ashvardanian 3y agoGreat writeup! It sounds like the author had quite the ordeal with Intel Knights Landing; often, the most informative articles come from genuine experiences (and sometimes, pain). I'm in the trenches with AMX right now, and it's not without its challenges. When Nvidia rolled out the `wmma` intrinsics for tensor cores years ago, they felt a bit unconventional. But as technology evolves, these new extensions seem to be increasingly idiosyncratic, challenging to navigate, and hyper-specialized. Hardware improvements significantly outpace compiler optimizations. So hand-written kernels are becoming the norm again :)
- deleted 3y ago[deleted]
- hnthrowaway0315 3y agoCan you please elaborate the last paragraph? Sounds interesting that we can now legitimately write raw assemblies! I always want to do that but it's still in my backlog.
- jandrewrogers 3y agoI think they mean writing code using compiler intrinsics, which are one layer above assembly (and how you should write most “assembly” these days). Many of the modern intrinsics in silicon are sufficiently complex in how they manipulate the data that it is effectively impossible for a compiler to detect applications of those instructions to normal idiomatic code. If you want to use those instructions, you will have to use them directly because the compiler can’t do it for you. The nice thing about intrinsics is that they look like C functions so they are easy to use. There is definitely a growing cottage industry for writing high-performance kernels using this increasingly rich set of intrinsics. I find myself writing a lot more intrinsics code than I used to.
- Const-me 3y agoWriting assembly is rarely a good choice in modern days. Whether assembly code is correct or not depends not just on the target CPU, but also on the target OS. See that bug for the example what gonna happen https://bugs.chromium.org/p/chromium/issues/detail?id=1218384 https://bugs.chromium.org/p/chromium/issues/detail?id=121838... TLDR: Windows ABI conventions say xmm6-xmm15 vector registers are non-volatile. When a function uses them, it needs to backup old data, and restore before the return. Linux ABI conventions say all vector registers are volatile and don’t need to be preserved. Chromium has non-trivial pieces written in assembly, and crashes. When writing C or C++ with intrinsics, registers are managed by the compiler. Modern ones follow these ABI conventions very carefully.
- TinkersW 3y agoI agree, but so is AVX10/256, get rid of that also thanks:)
- MobiusHorizons 3y agowhy? As I understand it 256 bit avx has been standard on x86 CPUs for generations. avx10 in this context just changes the number of instructions that can be done with 256bit registers. seems like it would be pretty silly to remove 256bit avx that is already useful.
- aleph_minus_one 3y ago> why? As I understand it 256 bit avx has been standard on x86 CPUs for generations. The Atom-derived cores did not support AVX for quite a long time. Even the Tremont microarchitecture (i.e. Elkhart Lake launched in September 2020 and Jasper Lake launched January 2021) does not support AVX. Only from the Gracemont microarchitecture on (launched in November 2021), there exist Intel Atom processors that support AVX/AVX2.
- Am4TIfIsER0ppos 3y agoAVX was also unavailable on Pentium and Celeron branded processors but I think that has changed.
- aleph_minus_one 3y agoThese are based on the same microarchitecture as the Atom cores, so it was also a problem up to Tremont. With the Gracemont microarchitecture, this "principally" changed, but until now Intel has not announced any processor of this microarchitecture that it branded "Pentium" or "Celeron": > https://en.wikipedia.org/w/index.php?title=Gracemont_(microarchitecture)&oldid=1170238307 https://en.wikipedia.org/w/index.php?title=Gracemont_(microa...
- Am4TIfIsER0ppos 3y ago
- snvzz 3y agoHopefully RISC-V will take off in high performance, before any of this is relevant.
- comex 3y agoEven if it does, this is also highly relevant to desktop CPUs.
- snvzz 3y ago>highly relevant It is a SIMD extension, despite Vector era (RISC-V Vector 1.0, ARM SVE2). Far from relevant. >desktop CPUs Will too be RISC-V. Not far from now. Some chips (likely from Intel and AMD) will offer "x86 acceleration" to run legacy software, for a transitional period lasting a few years.
- neonsunset 3y agoBefore making bold claims, it might be useful to get more familiar with a mess that are myriad of RISC-V extensions and seemingly insane/backwards decisions it makes completely ignoring the hard lessons learned by both ARM and x86.
- justahuman74 3y agoJust wait for RISC-VI, it'll be great
- snvzz 3y ago>it might be useful to get more familiar with a mess that are myriad of RISC-V extensions and seemingly insane/backwards decisions it makes completely ignoring the hard lessons learned by both ARM and x86. FUD unless you give specific examples and evidence backing them. Many "seemingly insane/backwards decisions" in RISC-V are good decisions backed by large bodies of data. Refer to Computer Architecture: A Quantitative Approach[0]. 0. ISBN-13 978-0128119051
- jiggawatts 3y agoIntel: "We've unified the many different vector instruction sets into a single standard!" Me: "Page one of the unified standard lists its many variants, which seem to be the same number that existed before?" Intel: "We've renamed them all! See! Unified!"
- deleted 3y ago[deleted]
- snvzz 3y agoAnd vector is a misnomer. ARM SVE2 and RISC-V Vector 1.0 are vector extensions. AVX is just SIMD with marketing.
- foota 3y agoIt's not?
- kmeisthax 3y agoI'm confused as to what the difference is? I thought SIMD = vector.
- wmf 3y agoTraditionally "vectors" are variable size while SIMD is fixed size. Almost no one uses those definitions any more.
- RaisingSpear 3y agoTo add on: SVE/RVV also have fixed size vectors. It's just that the size is specified by the hardware, whilst for AVX, software specifies the size.
- dzaima 3y agoThis is at least somewhat wrong at best - SVE2 doesn't have a dynamic vector length like RVV (only non-fixed hardware register width, and masking), and Cray-1, said to be the first vector processor, has a fixed hardware register size (64 elements, each 64 bits). So either SVE2 isn't vector due to lack of VL, or AVX-512/AVX10 is vector if VL isn't necessary (the only difference between AVX and SVE2 being the scalable registers, which Cray-1 doesn't have).
- johnklos 3y agoSo much of this, such as the backpedaling on support of AVX512, seems to be at least indirectly related to the desire to "solve" the problem that occurs when Intel wants to have little (efficiency) cores that might not have the same feature set as the big (performance) cores. Two decades ago I put a Pentium III and a Celeron on to the same dual processor motherboard (using a Celery to Nintendo cartridge adapter), and said, "Look! Different processors can run at the same time in the same system. Perhaps we should have a way for processes to express a desire to use specific CPU features, and if they request features not available on all CPUs, we'll keep them on the ones that do." Sigh. Intel should've told OS writers years ago that heterogenous CPU features would be a thing soon, so prepare for it. Then we could've easily had AVX512 on big cores and AVX256 on little cores, and it'd've all be just fine.
- Veliladon 3y agoNo it wouldn't because promotion of a process from an E core to a P core isn't free. Microsoft and Intel could literally do this today on existing hardware by trapping unsupported instructions and then having the OS scheduler promote the process but they don't because it takes so stupidly long and requires an immense burst of resources. All this AVX10/128 is a necessary evil because the far better option is for an E core to be able to at least run the code, even if it's not performant.
- dralley 3y agoOr, intel could do like AMD and just design P cores that are space efficient enough to stuff them into a chip like E cores while still getting good performance.
- vlovich123 3y agoHow long is "stupidly long"? I get that it's expensive to transition back and forth repeatedly, but if it's on the order of even tens of milliseconds, you only need to do it once and then forever pin that process to P cores. And you could easily have a preamble in the init that checks the instructions you need to trigger that on boot if you wanted to front-load that cost. Heck, you could have it be a filesystem attribute that's automatically set when the promotion is noticed so that you just do it with 0 overhead after the very first execution.
- universal_sinc 3y agoThis article highly underestimates the value of keeping 128b vector performance high. Most code doesn't get recompiled or compiled with the appropriate flags. There is significant overhead involved in supporting 1x512b operations, 2x256b operations, or 4x128b operations per cycle with the same datapaths, forwarding network, and register files. Until 128b vector performance gets deprecated this tension incentives narrow implementations.
- RaisingSpear 3y ago> There is significant overhead involved Current CPUs already do this, and have been doing so for quite some time. And AVX10/128 doesn't alleviate it either.
- Symmetry 3y agoUnless Intel is planning on deprecating SSE then this is already baked in and AVX10/128 existing or not existing won't be changing anything.
- anonymoushn 3y agoFor string parsing and formatting or codec workloads (or for swiss tables, but really these are slower than alternatives so nobody should use them) it may be better to target 32x 128b registers than to target 16x 32b registers. An implementation that aliases the two or has high runtime costs for switching between the full set of 128b registers and the reduced set of 32b registers would be exceedingly annoying though.
- meling 3y agoI attended this db seminar at Berkeley earlier today… Helena Caminal (Google) – “Vector architectures for data-parallel workloads” https://sky.cs.berkeley.edu/events/database-seminar-helena-caminal-google-vector-architectures-for-data-parallel-workloads/ https://sky.cs.berkeley.edu/events/database-seminar-helena-c... Very interesting work about associative processing: “The Associative Processing (AP) paradigm leverages content-addressable memories to realize bit-serial arithmetic and logic operations, via sequences of search and update memory operations over multiple memory rows in parallel.”
- diimdeep 3y agoInteresting, I have seen claim by Adobe senior principal scientist, from 2012[1] and then 10 years later in 2022[2] that if you are writing single threaded scaler code in vanilla C++ you can only get up to 0.25% of maximum GFLOPS of typical machine. Looks like in 10 years nothing really changed. In my understanding this is real problem that needs to be immediately solved, not just for C++ but any other language. [1] Sean Parent: "Now What? A vignette in three parts" https://www.youtube.com/watch?v=iGenpw2NeKQ&t=1382s https://www.youtube.com/watch?v=iGenpw2NeKQ&t=1382s [2] Keynote: The Tragedy of C++, Acts One & Two - Sean Parent - CppNorth 2022 https://youtu.be/kZCPURMH744?t=2924 https://youtu.be/kZCPURMH744?t=2924
- KingLancelot 3y ago[dead]
- pjmlp 3y agoThis is why having JIT/AOT toolchains is great, let the runtime optmize to what is actually available, instead of shipping multiple dlls/so with exponential complexity of ISA mappings. Every Android phone and ChromeOS device is its own special snowflake in hardware configuration, regarding RISC, ARM and Intel CPUs, or GPUs for that matter.
- unnah 3y agoSpeculating a bit here: Perhaps one could even create a JIT that generates different code for Intel P-cores and E-cores in the same process. However, that would be very challenging to get working if the OS can switch the process from one core to another at any time. The runtime would basically have to detect which AVX-512 instruction has failed at any point in the compiled code, map the processor state to AVX-2 registers and jump to an equivalent AVX2 code path. It would probably be easier to build the implementation on software transactions that can be restarted at coarse granularity, i.e. something like STM.
- pjmlp 3y agoHow do you imagine Android works across high and lower power ARM cores? However ART is also a great piece of engineering at JVM and CLR level, even though Dalvik was quite lame, even when compared against Nokia and Sony-Ericson J2ME implementations. ART is another level. Nowadays you have an handwritten Assembly interpreter for faster startup, a JIT compiler, that also gathers PGO data, and an AOT compiler that picks up that data when the device is idle taking all the required optimizations. To take this even furter, those PGO files are uploaded into the Play Store, and shared across compatible devices when applications are downloaded, not only allows jumping straight into JIT with PGO data, all devices collaborate gathering PGO data that covers all common use cases across the application audience. And yes this also takes into account the ARM architectures. On the JVM side, IBM and Azul have similar offerings with cloud based JITs for their implementations.
- unnah 3y agoI thought ARM big.little implementations supported the exact same instruction set on both big and little cores. Don't they?
- sylware 3y ago64bytes is a sweet spot, a cache line on x86_64, going above seems "not appropriate".
- Tringi 3y agoI'm always fascinated when engineers who make my year's salary in a week come up with something as dumb as this.