7 ms·
AVX2 is slower than SSE2-4.x under Windows ARM emulation
- iberator 8mo agoAVX2 should be banned anyway. Only expensive CPUs have it, ruining mininum games requirements and making hardware obsolete. Most of the world lives of 300$ per month
- SecretDreams 8mo agoAlmost every x86 cpu made in the last decade should have avx2. Maybe you're thinking of avx512 or avx10?
- jorvi 8mo agoYeah, sounds like they're confusing AVX2 for AVX512. AVX2 has been common for a decade at least and greatly accelerates performance. AVX512 is so kludgy that it usually leads to a detriment in performance due to the extreme power requirements triggering thermal throttling.
- badgersnake 8mo agoI think that's slightly old information as well, AVX512 works well on Zen5.
- SecretDreams 8mo agoAgree. It's only recently with modern architectures in the server space that avx512 has shown some benefit. But avx2 is legit and has been for a long time.
- kimixa 8mo agoAMD's implementation very much doesn't have that issue - it throttles slightly, maybe, but it's still a net benefit. The problem with Intel's implementation is that the throttling was immediate - and took noticeable time to then settle and actually start processing again - from any avx512 instruction, so the "occasional" avx512 instruction (in autovectorized code, or something like the occasional optimized memcpy or similar) was a net negative in performance. This meant that it only benefitted large chunks of avx512-heavy code, so this switching penalty was overcome. But there's plenty in avx512 the really helps real algorithms outside the 512-wide registers - I think it would be perceived very differently if it was initially the new instructions on the same 256-wide registers - ie avx10 - in the first place, then extended to 512 as the transistor/power budgets allowed. AVX512 was just tying too many things together too early than "incremental extensions".
- otherjason 8mo agoSee this correct comment above: https://news.ycombinator.com/item?id=47061696 https://news.ycombinator.com/item?id=47061696 AVX512 leading to thermal throttling is a common myth that from what I can tell traces its origins to a blog post about clock throttling on a particular set of low-TDP SKUs from the first generation of Xeon CPUs that supported it (Skylake-X), released over a decade ago: https://blog.cloudflare.com/on-the-dangers-of-intels-frequency-scaling/ https://blog.cloudflare.com/on-the-dangers-of-intels-frequen... The results were debated shortly after that by well-known SIMD authors that were unable to duplicate the results: https://lemire.me/blog/2018/08/25/avx-512-throttling-heavy-instructions-are-maybe-not-so-dangerous/ https://lemire.me/blog/2018/08/25/avx-512-throttling-heavy-i... In practice, this has not been an issue for a long time, if ever; clock frequency scaling for AVX modes has been continually improved in subsequent Intel CPU generations (and even more so in AMD Zen 4/5 once AVX512 support was added).
- adrian_b 8mo agoThat was true only for the 14-nm Intel Skylake derivatives, which had very bad management of the clock frequency and supply voltage, so they scaled down the clock prophylactically, for fear that they would not be able to prevent overheating fast enough. All AMD Zen 4 and Zen 5 and all of the Intel CPUs since Ice Lake that support AVX-512, benefit greatly from using it in any application. Moreover the AMD Zen CPUs have demonstrated clearly that for vector operations the instruction-set architecture really matters a lot. Unlike the Intel CPUs, the AMD CPUs use exactly the same execution units regardless whether they execute AVX2 or AVX-512 instructions. Despite this, their speed increases a lot when executing programs compiled for AVX-512 (in part for eliminating bottlenecks in instruction fetching and decoding, and in part because the AVX-512 instruction set is better designed, not only wider).
- corysama 8mo agoIn gamedev it takes 7-10 years before you can require a new tech without getting a major backlash. AMD came out with AVX2 support in 2015. And, the (vocal minority) petitions to get AVX2 requirements removed from major games and VR systems are only now starting to quiet down. So, in order to make use of users new fancy hardware without abandoning other users old and busted hardware, you have to support multiple back-ends. Same as it ever was. Actually, a lot easier than it ever was today. Doom 3 famously required Carmack to reimplement the rendering 6 times to get the same results out of 6 different styles of GPUs that were popular at the time. ARB Basic Fallback (R100) Multi-pass Minimal effects, no specular. NV10 GeForce 2 / 4 MX, 5 Passes, Used Register Combiners. NV20 GeForce 3 / 4 Ti, 2–3 Passes, Vertex programs + Combiners. R200 Radeon 8500–9200, 1 Pass, Used ATI_fragment_shader. NV30 GeForce FX Series, 1 Pass, Precision optimizations (FP16). ARB2 Radeon 9500+ / GF 6+, 1 Pass, Standard high-end GLSL-like assembly. https://community.khronos.org/t/doom-3/37313 https://community.khronos.org/t/doom-3/37313
- winstonwinston 8mo agoNot really, Intel Celeron/Pentium/Atom (apollo lake) that was made in the last decade does not have AVX. These CPUs were very popular for low-cost, low-tdp quad-core machines such as Intel NUC mini PC. Edit. Furthermore, i think that none of these (pre-2020) low budget CPUs support AVX2, until Tiger lake released in 2020.
- nwellnhof 8mo agoI think the last Intel CPUs that didn't support AVX were 10th gen (Comet Lake) Pentiums and Celerons, released in 2019. Edit: That's wrong. Jasper Lake from 2021 also came without AVX support.
- my123 8mo agoIt took until Alder Lake-N for the atom-grade stuff to have AVX2 across the board.
- thrtythreeforty 8mo agoAu contraire: AVX2 is the vector ISA for x86 that doesn't suck. And it's basically ubiquitous at this point.
- Tuldok 8mo agoI, too, hate progress. By the way, the AMD Athlon 3000G system I helped build for a friend has AVX2. Even the old HP T630 thin client (https://www.parkytowers.me.uk/thin/hp/t630/ https://www.parkytowers.me.uk/thin/hp/t630/) I bought for $15 as a home network router has AVX2.
- jsheard 8mo agohttps://store.steampowered.com/hwsurvey https://store.steampowered.com/hwsurvey Other Settings > AVX2 > 95.11% supported (+0.30% this month)
- cogman10 8mo agoAny x86 CPU manufactured in the last 10 year has AVX2. Here's a laptop for $350 which has a CPU with AVX2 support. https://ebay.us/m/yoznZ1 https://ebay.us/m/yoznZ1
- wiredpancake 8mo ago[dead]
- Aissen 8mo agoSpoiler is in the conclusion: > Yes, it is absolutely key to build your app as ARM, not to rely on Windows ARM emulation.
- okanat 8mo agoIs this actually surprising? Once you use stuff like vectorization you want to get as much performance out of a system. If you're not natively compiling for a system, you won't get any performance. Using AVX2 and using an emulator have contradictory goals. Of course there can be a better emulator or actually matching hardware design (since both Apple and Microsoft actually exploit the similar register structure between ARM64 and x86_64). However, this means you have increased complexity and reduced reliability / predictability.
- vintagedave 8mo agoAuthor here - have to say, thanks for reading all the way to the end, you don't always see people do that ;) I put a spoiler at the top too, to avoid trying to make people read the whole thing. The real bit is that chart, which I think is quite an amazing result. You're right re building. We're a compiler vendor, so we have a natural interest in what people should be targeting. But even for us the results here were not what we expected ahead of time.
- Aissen 8mo agoHaving written an emulator, the conclusion was a bit less surprising. It's also probably not definitive, as it might depend on the specific hardware (and future emulator optimizations); you even say in your blog that the hardware you use is not the hardware Microsoft targeted.
- qingcharles 8mo agoIs Chrome for Windows compiled in ARM too or does it use the Windows under emulation? The reason I ask is that I believe Windows Chrome is (like many Windows binaries) compiled with lots of the advanced CPU features disabled (e.g. AVX512) because they're not available on older PCs. Is that true?
- kbolino 8mo agoI suspected this was because the vector units were not wide enough, and it seems that is the case. AVX2 is 256-bit, ARM NEON is only 128-bit. The big question then is, why are ARM desktop (and server?) cores so far behind on wider SIMD support? It's not like Intel/AMD came up with these extensions for x86 yesterday; AVX2 is over 15 years old.
- phonon 8mo agoWell, you can always use a Fujitsu A64FX...let me check eBay.. :-)
- jsheard 8mo agoSVE was supposed to be the next step for ARM SIMD, but they went all-in on runtime variable width vectors and that paradigm is still really struggling to get any traction on the software side. RISC-V did the same thing with RVV, for better or worse.
- kbolino 8mo agoYeah, the extensions exist, and as pointed out by a sibling comment to yours, have been implemented in supercomputer cores made by Fujitsu. However, as far as I know, neither Apple nor Qualcomm have made any desktop cores with SVE support. So the biggest reason there's no desktop software for it is because there's no hardware support.
- jsheard 8mo agoARMs Neoverse IP does support SVE, so it's at least already relevant in cloud applications. Apparently AWS Graviton3 had 256bit SVE, but Graviton4 regressed back to 128bit SVE for some reason? https://ashvardanian.com/posts/aws-graviton-checksums-on-neon-vs-sve/ https://ashvardanian.com/posts/aws-graviton-checksums-on-neo...
- camel-cdr 8mo agoThe problem with SVE is that ARM vendors need to make NEON as fast as possible to stay competitive, so there is little incentive to implement SVE with wider vectors. Graviton3 has 256-bit SVE vector registers but only four 128-bit SIMD execution units, because NEON needs to be fast. Intel previously was in such a dominant market position that they could require all performance-critical software to be rewritten thrice.
- crest 8mo agoI wouldn't be surprised for SSE4 to be the fastest cause it's easiest to map to NEON as both use 128 bit registers and offer a fairly simlar feature set.
- vintagedave 8mo agoAuthor here - agreed, we have been speculating that too.
- TheJoeMan 8mo agoI tried searching "SSE2-4.x" and this is the top result in DDG and Google, so I was initially confused what instruction set the article is referring to. However, this appears to be shorthand for SSE2 through SSE4? Perhaps a rephrasing of the article title could be helpful.
- vintagedave 8mo agoAuthor here - yes, it's shorthand for the set of SSE2, SSE3, SSSE3 (not a typo), and SSE4 including SSE 4.1 and SSE 4.2. My bad for confusion! That set matches the x86-64-v2 x64 microarchitecture level. Most of the articles uses 'v2' or 'v3' or 'x86-64-v2', but I thought that more people would be familiar with the names of the instruction sets than that x64 was versioned. The versions only appeared quite recently (2020) and are rather retroactive.
- cogman10 8mo agoI read it as SSE2->4.x. Generally speaking, when working with SSE instructions you'll end up using a mix of instructions from 2->4 as they are all effectively just additional operations on the SSE2 registers.
- targettracker 8mo ago[flagged]
- Mickell 8mo ago[flagged]
- mtklein 8mo agoIf I remember correctly, the AVX2 feature set is a fairly direct upscale of SSE4.1 to 256 bit. Very few instructions even allowed interaction between the top and bottom 128 bits, I assume to make implementation on existing 128 bit vector units easier. And the most notable new things that AVX2 added beyond that widening, fp16 conversion and FMA support, are also present in NEON, so I wouldn't expect that to be the issue either. So I'd bet the issue is either newness of the codebase, as the article suggests, or perhaps that it is harder to schedule the work in 256 bit chunks than 128. It's got to be easier when you've got more than enough NEON q registers to handle the xmms, harder when you've got only exactly enough to pair up for handling ymms?
- ack_complete 8mo agoThere are also mode switching and calling convention issues. The way that the vector registers were extended to 256-bit causes problems when legacy 128-bit and 256-bit ops are mixed. Doing so puts the CPU into a mode where all legacy 128-bit ops are forced to blend the high half, which can reduce throughput of existing SSE2-based library routines to as low as 1/4 throughput. For this reason, AVX code has to aggressively use the VZEROUPPER instruction to ensure that the CPU is not left in AVX 256-bit vector mode before possibly returning to any library or external code that uses SSE2. VZEROUPPER sets a flag to zero the high half of all 256-bit registers, so it's cheap on modern x86 CPUs but can be expensive to emulate without hardware support. The other problem is that only the low 128 bits of vector registers are preserved across function calls due to the Windows x64 calling convention and the VZEROUPPER issue. This means that practically any call to external code forces the compiler to spill all AVX vectors to memory. Ideally 256-bit vector usage is concentrated in leaf routines so this isn't an issue, but where used in non-leaf routines, it can result in a lot of memory traffic.
- spacecadet_ 8mo ago> Very few instructions even allowed interaction between the top and bottom 128 bits That would be plain AVX, AVX2 has shuffles across the 128-bit boundary. To me that seems like the main hurdle for emulation with 128-bit vectors, in my experience compilers are very eager to emit shuffle instructions if allowed, and emulating a 256-bit shuffle with 128-bit operations would require 2 shuffles and a blend for each half of the emulated register. EDIT: I just noticed that the benchmark in the article is pure math which probably wouldn't hit this particular issue, so this doesn't explain the performance difference...
- Const-me 8mo ago> AVX2 level includes FMA (fast multiply-add) FMA acronym is not fast multiply add, it’s fused multiply add. Fused means the instruction computes the entire a * b + c expression using twice as many mantissa bits, only then rounds the number to the precision of the arguments. It might be the Prism emulator failed to translate FMA instructions into a pair of two FMLA instructions (equally fused ARM64 equivalent), instead it did some emulation of that fused behaviour, which in turn what degraded the performance of the AVX2 emulation.
- vintagedave 8mo agoAuthor here - thanks - my bad. Fixed 'fast' -> 'fused' :) I don't have insight into how Prism works, but I have wondered if the right debugger would see the ARM code and let us debug exactly what was going on for sure.
- Const-me 8mo agoYou’re welcome. Sadly, I don’t know how to observe ARM assembly produced by Prism. And one more thing. If you test on an AMD processor, you will probably see much less profit from FMA. Not because it’s slower, but because SSE4 version will runs much faster. On Intel processors like your Tiger Lake, all 3 operations addition, multiplication and FMA compete for the same execution units. On AMD processors however, multiplication and FMA do as well but addition is independent, e.g. on Zen4 multiplication and FMA run on execution units FP0 or FP1 while addition runs on execution units FP2 or FP3. This way replacing multiply/add combo with FMA on AMD doesn’t substantially improve throughput in FLOPs. The only win is L1i cache and instruction decoder.
- malkia 8mo agoYou can ... to a degree - Google for "XtaCache"
- LeoNatan25 8mo agoAny equivalent look at Apple's Rosetta 2? Perhaps if author has time and availability of hardware, they can have a similar look. Rosetta 2 is going away next year, and it's a shame, even if from a purely technical reason. Apple will never open source it.
- vintagedave 8mo agoMy daily driver is a M2 Mac, and we added the same set of optimisations to ARM on Mac as we did to ARM Windows (at the same time as Intel Windows we measured emulation of in this blog.) More info: https://blogs.remobjects.com/2026/01/26/fast-math-in-six-languages-what-we-did-and-why-it-works/ https://blogs.remobjects.com/2026/01/26/fast-math-in-six-lan... We did not try to especially optimise Intel Mac, but it's very tempting to do so in order to look at it as you ask. I wish Rosetta was open sourced too. Same with Prism. I think any and all translation tech could only benefit everyone.
- LeoNatan25 8mo agoWell, Prism is likely to be with us for a decade, if not more, since Microsoft actually cares about backwards compatibility, whereas Apple much less so, and I guess, to them, we're at a "good enough" state. So Microsoft releasing it is less likely, but Apple could, especially after it is done. But I suspect some asshole sees a "competitive advantage" somewhere, and won't sign off a source release. What a gut punch for the team that worked on it. Looking forward to a future look at Rosetta 2. Thanks!
- rbanffy 8mo agoWhen doing feature detection for execution path selection, it’s sometimes useful to run some quick benchmarks to see which path is objectively best. Now we have two-ish implementations or x86, but back in the 1980s and 1990s we had quite a few, some with wildly different performance characteristics. And, if we talk about ARM and RISC-V, we’ll have an order of magnitude more.