6 ms·

I write a lot of SIMD and I don't really agree with this.. Flaw1:fixed width I prefer fixed width as it makes the code simpler to write, size is known as comp

I write a lot of SIMD and I don't really agree with this.. Flaw1:fixed width I prefer fixed width as it makes the code simpler to write, size is known as compile time so we know the size of our structures. Swizzle algorithms are also customized based on the size. Flaw2:pipelining no CPU I care about is in order so mostly irrelevant, and even scalar instructions are pipelined Flaw3: tail handling I code with SIMD as the target, and have special containers that pad memory to SIMD width, no need to mask or run a scalar loop. I copy the last valid value into the remaining slots so it doesn't cause any branch divergence.

aengelke 1y ago
I agree; and the article seems to have also quite a few technical flaws: - Register width: we somewhat maxed out at 512 bits, with Intel going back to 256 bits for non-server CPUs. I don't see larger widths on the horizon (even if SVE theoretically supports up to 2048 bits, I don't know any implementation with ~~>256~~ >512 bits). Larger bit widths are not beneficial for most applications and the few applications that are (e.g., some HPC codes) are nowadays served by GPUs. - The post mentions available opcode space: while opcode space is limited, a reasonably well-designed ISA (e.g., AArch64) has enough holes for extensions. Adding new instructions doesn't require ABI changes, and while adding new registers requires some kernel changes, this is well understood at this point. - "What is worse, software developers often have to target several SIMD generations" -- no way around this, though, unless auto-vectorization becomes substantially better. Adjusting the register width is not the big problem when porting code, making better use of available instructions is. - "The packed SIMD paradigm is that there is a 1:1 mapping between the register width and the execution unit width" -- no. E.g., AMD's Zen 4 does double pumping, and AVX was IIRC originally designed to support this as well (although Intel went directly for 256-bit units). - "At the same time many SIMD operations are pipelined and require several clock cycles to complete" -- well, they are pipelined, but many SIMD instructions have the same latency as their scalar counterpart. - "Consequently, loops have to be unrolled in order to avoid stalls and keep the pipeline busy." -- loop unroll has several benefits, mostly to reduce the overhead of the loop and to avoid data dependencies between loop iterations. Larger basic blocks are better for hardware as every branch, even if predicted correctly, has a small penalty. "Loop unrolling also increases register pressure" -- it does, but code that really requires >32 registers is extremely rare, so a good instruction scheduler in the compiler can avoid spilling. In my experience, dynamic vector sizes make code slower, because they inhibit optimizations. E.g., spilling a dynamically sized vector is like a dynamic stack allocation with a dynamic offset. I don't think SVE delivered any large benefits, both in terms of performance (there's not much hardware with SVE to begin with...) and compiler support. RISC-V pushes further into this direction, we'll see how this turns out.
- cherryteastain 1y ago
  Fujitsu A64FX used in the Fugaku supercomputer uses SVE with 512 bit width
  - aengelke 1y ago
    Thanks, I misremembered. However, the microarchitecture is a bit "weird" (really HPC-targeted), with very long latencies (e.g., ADD (vector) 4 cycles, FADD (vector) 9 cycles). I remember that it was much slower than older x86 CPUs for non-SIMD code, and even for SIMD code, it took quite a bit of effort to get reasonable performance through instruction-level parallelism due to the long latencies and the very limited out-of-order capacities (in particular the just 2x20 reservation station entries for FP).
- camel-cdr 1y ago
  > we somewhat maxed out at 512 bits Which still means you have to write your code at least thrice, which is two times more than with a variable length SIMD ISA. Also there are processors with larger vector length, e.g. 1024-bit: Andes AX45MPV, SiFive X380, 2048-bit: Akeana 1200, 16384-bit: NEC SX-Aurora, Ara, EPI > no way around this You rarely need to rewrite SIMD code to take advantage of new extensions, unless somebody decides to create a new one with a larger SIMD width. This mostly happens when very specialized instructions are added. > In my experience, dynamic vector sizes make code slower, because they inhibit optimizations. Do you have more examples of this? I don't see spilling as much of a problem, because you want to avoid it regardless, and codegen for dynamic vector sizes is pretty good in my experience. > I don't think SVE delivered any large benefits Well, all Arm CPUs except for the A64FX were build to execute NEON as fast as possible. X86 CPUs aren't built to execute MMX or SSE or the latest, even AVX, as fast as possible. Anyway, I know of one comparison between NEON and SVE: https://solidpixel.github.io/astcenc_meets_sve https://solidpixel.github.io/astcenc_meets_sve > Performance was a lot better than I expected, giving between 14 and 63% uplift. Larger block sizes benefitted the most, as we get higher utilization of the wider vectors and fewer idle lanes. > I found the scale of the uplift somewhat surprising as Neoverse V1 allows 4-wide NEON issue, or 2-wide SVE issue, so in terms of data-width the two should work out very similar.
cjbgkagh 1y ago
I have similar thoughts, I don't understand the push for variable width SIMD. Possibly due to ignorance but I think it's an abstraction that can be specialized for different hardware so the similar tradeoffs between low level languages and high level languages apply. Since I already have to be aware of hardware level concepts such as 256bit shuffle not working across 128bit lanes and different instructions having very different performance characteristics on different CPUs I'm already knee deep in hardware specifics. While in general I like abstractions I've largely given up waiting for a 'sufficiently advanced compiler' that would properly auto-vectorize my code. I think AGI AI is more likely to happen sooner. At a guess it seems to be that SIMD code could work on GPUs but GPU code has different memory access costs so the code there would also be completely different. So my view is either create a much better higher level SIMD abstraction model with a sufficiently advanced compiler that knows all the tricks or let me work closely at the hardware level. As an outsider who doesn't really know what is going on it does worry me a bit that it appears that WASM is pushing for variable width SIMDs instead of supporting ISAs generally supported by CPUs. I guess it's a portability vs performance tradeoff - I worry that it may be difficult to make variable as performant as fixed width and would prefer to deal with portability by having alternative branches at code level. >> Finally, any software that wants to use the new instruction set needs to be rewritten (or at least recompiled). What is worse, software developers often have to target several SIMD generations, and add mechanisms to their programs that dynamically select the optimal code paths depending on which SIMD generation is supported. Why not marry the two and have variable width SIMD as one of the ISA options and if in the future variable width SIMD become more performant then it would just be another branch to dynamically select.
- kevingadd 1y ago
  Part of the motive behind variable width SIMD in WASM is that there's intentionally-ish no mechanism to do feature detection at runtime in WASM. The whole module has to be valid on your target, you can't include a handful of invalid functions and conditionally execute them if the target supports 256-wide or 512-wide SIMD. If you want to adapt you have to ship entire modules for each set of supported feature flags and select the correct module at startup after probing what the target supports. So variable width SIMD solves this by making any module using it valid regardless of whether the target supports 512-bit vectors, and the VM 'just' has to solve the problem of generating good code. Personally I think this is a terrible way to do things and there should have just been a feature detection system, but the horse fled the barn on that one like a decade ago.
camel-cdr 1y ago
> I prefer fixed width Do you have examples for problems that are easier to solve in fixed-width SIMD? I maintain that most problems can be solved in a vector-length-agnostic manner. Even if it's slightly more tricky, it's certainly easier than restructuring all of your memory allocations to add padding and implementing three versions for all the differently sized SIMD extensions your target may support. And you can always fall back to using a variable-width SIMD ISA in a fixed-width way, when necessary.
- jcranmer 1y ago
  There's a category of autovectorization known as Superword-Level Parallelism (SLP) which effectively scavenges an entire basic block for individual instruction sequences that might be squeezed together into a SIMD instruction. This kind of vectorization doesn't work well with vector-length-agnostic ISAs, because you generally can't scavenge more than a few elements anyways, and inducing any sort of dynamic vector length is more likely to slow your code down as a result (since you can't do constant folding). There's other kinds of interesting things you can do with vectors that aren't improved by dynamic-length vectors. Something like abseil's hash table, which uses vector code to efficiently manage the occupancy bitmap. Dynamic vector length doesn't help that much in that case, particularly because the vector length you can parallelize over is itself intrinsically low (if you're scanning dozens of elements to find an empty slot, something is wrong). Vector swizzling is harder to do dynamically, and in general, at high vector factors, difficult to do generically in hardware, which means going to larger vectors (even before considering dynamic sizes), vectorization is trickier if you have to do a lot of swizzling. In general, vector-length-agnostic is really only good for SIMT-like codes, which you can express the vector body as more or less independent f(index) for some knowable-before-you-execute-the-loop range of indices. Stuff like DAXPY or BLAS in general. Move away from this model, and that agnosticism becomes overhead that doesn't pay for itself. (Now granted, this kind of model is a large fraction of parallelizable code, but it's far from all of it).
  - camel-cdr 1y ago
    The SLP vectorizer is a good point, but I think it's, in comparison with x86, more a problem of the float and vector register files not being shared (in SVE and RVV). You don't need to reconfigure the vector length; just use it at the full width. > Something like abseil's hash table If I remember this correctly, the abseil lookup does scale with vector length, as long as you use the native data path width. (albeit with small gains) There is a problem with vector length agnostic handling of abseil, which is the iterator API. With a different API, or compilers that could eliminate redundant predicated load/stores, this would be easier. > good for SIMT-like codes Certainly, but I've also seen/written a lot of vector length agnostic code using shuffles, which don't fit into the SIMT paradigm, which means that the scope is larger than just SIMT. --- As a general comparison, take AVX10/128, AVX10/256 and AVX10/512, overlap their instruction encodings, remove the few instructions that don't make sense anymore, and add a cheap instruction to query the vector length. (probably also instructions like vid and viota, for easier shuffle synthesization) Now you have a variable-length SIMD ISA that feels familiar. The above is basically what SVE is.
tonetegeatinst 1y ago
AFAIK about every modern CPU uses out of order von Neumann architecture. The only people who don't are the handful of researchers and people who work with the government research into non van Neumann designed systems.
- luyu_wu 1y ago
  Low power RISC cores (both ARM and RISC-V) are typically in-order actually! But any core I can think of as 'high-performance' is OOO.
  - whaleofatw2022 1y ago
    MIPS as well as Alpha AFAIR. And technically itanium, otoh It seems to me a bit like a niche for any performance advantages...
    - mattst88 1y ago
      Alpha 21264 is out-of-order.
    - pezezin 1y ago
      I would not call neither MIPS, Alpha nor Itanium "high-performance" in 2025...
    - p_l 1y ago
      Alpha was out of order starting with EV7, but most importantly the entire architecture was designed with eye for both pipeline hazards and out of order execution, unlike VAX that it replaced which made it pretty much impossible
- IshKebab 1y ago
  Microcontrollers are often in-order.
  - Narishma 1y ago
    Are there any microcontrollers that aren't in-order?
PaulHoule 1y ago
In AVX-512 we have a platform that rewards the assembly language programmer like few platforms have since the 6502. I see people doing really clever things that are specific to the system and one level it is really cool but on another level it means SIMD is the domain of the specialist, Intel puts out press releases about the really great features they have for the national labs and for Facebook whereas the rest of us are 5-10 years behind the curve for SIMD adoption because the juice isn't worth the squeeze. Just before libraries for training neural nets on GPUs became available I worked on a product that had a SIMD based neural network trainer that was written in hand-coded assembly. We were a generation behind in our AVX instructions so we gave up half of the performance we could have got, but that was the least of the challenges we had to overcome to get the product in front of customers. [1] My software-centric view of Intel's problems is that they've been spending their customers and shareholders money to put features in chips that are fused off or might as well be fused off because they aren't widely supported in the industry. And that they didn't see this as a problem and neither did their enablers in the computing media and software industry. Just for example, Apple used to ship the MKL libraries which like a turbocharger for matrix math back when they were using Intel chips. For whatever reason, Microsoft did not do this with Windows and neither did most Linux distributions so "the rest of us" are stuck with a fraction of the performance that we paid for. AMD did the right thing in introducing double pumped AVX-512 because at least assembly language wizards have some place where their code runs and the industry gets closer to the place where we can count on using an instruction set defined 12 years ago. [1] If I'd been tasked with updating the to next generation I would have written a compiler (if I take that many derivatives by hand I'll get one wrong.) My boss would have ordered me not to, I would have done it anyway and not checked it in.
- bee_rider 1y ago
  It is kind of a bummer that MKL isn’t open sourced, as that would make inclusion in Linux easier. It is already free-as-in-beer, but of course that doesn’t solve everything. Baffling that MS didn’t use it. They have a pretty close relationship… Agree that they are sort of going after hard-to-use niche features nowadays. But I think it is just that the real thing we want—single threaded performance for branchy code—is, like, incredibly difficult to improve nowadays.
derf_ 1y ago
> I code with SIMD as the target, and have special containers that pad memory to SIMD width... I think this may be domain-specific. I help maintain several open-source audio libraries, and wind up being the one to review the patches when people contribute SIMD for some specific ISA, and I think without exception they always get the tail handling wrong. Due to other interactions it cannot always be avoided by padding. It can roughly double the complexity of the code [0], and requires a disproportionate amount of thinking time vs. the time the code spends running, but if you don't spend that thinking time you can get OOB reads or writes, and thus CVEs. Masked loads/stores are an improvement, but not universally available. I don't have a lot of concrete suggestions. I also work with a lot of image/video SIMD, and this is just not a problem, because most operations happen on fixed block sizes, and padding buffers is easy and routine. I agree I would have picked other things for the other two in my own top-3 list. [0] Here is a fun one, which actually performs worst when len is a multiple of 8 (which it almost always is), and has 59 lines of code for tail handling vs. 33 lines for the main loop: https://gitlab.xiph.org/xiph/opus/-/blob/main/celt/arm/celt_neon_intr.c#L171 https://gitlab.xiph.org/xiph/opus/-/blob/main/celt/arm/celt_...
- deleted 1y ago
  [deleted]
- jandrewrogers 1y ago
  > Masked loads/stores are an improvement, but not universally available. Traditionally we’ve worked around this with pretty idiomatic hacks that efficiently implement “masked load” functionality in SIMD ISAs that don’t have them. We could probably be better about not making people write this themselves every time.
- ack_complete 1y ago
  It depends on how integrated your SIMD strategy is into the overall technical design. Tail handling is much easier if you can afford SIMD-friendly padding so a full vector load/store is possible even if you have to manually mask. That avoids a lot of the hassle of breaking down memory accesses just to avoid a page fault or setting off the memory checker. Beyond that -- unit testing. I don't see enough of it for vectorized routines. SIMD widths are small enough that you can usually just test all possible offsets right up against a guard page and brute force verify that no overruns occur.
Const-me 1y ago
> I prefer fixed width Another reason to prefer fixed width, compilers may pass vectors to functions in SIMD registers. When register size is unknown at compile time, they have to pass data in memory. For complicated SIMD algorithms the performance overhead gonna be huge.
- kjs3 1y ago
  Back in the day, you had Cray style vector registers, and you had CDC style[1] 'vector pipes' (I think I remember that's what they called them) that you fed from main memory. So you would (vastly oversimplifying) build your vectors in consecutive memory locations (up to 64k as I recall), point to a result destination in memory and execute a vector instruction. This works fine if there's a close match between cpu speed and memory access speed. The compilers were quite good, and took care of handling variable sized vectors, but I have no idea what was going on under the hood except for some hi-level undergrad compiler lectures. As memory speed vs cpu speed divergence became more and more pronouced, it quickly became obvious that vector registers were the right performance answer, basically everyone jumped that way, and I don't think anyone has adopted a memory-memory vector architecture since the '80s. [1] from CDC STAR-100 and followons like the CDC Cyber 180/990, Cyber 200 series & ETA-10.

aengelke 1y ago

I agree; and the article seems to have also quite a few technical flaws: - Register width: we somewhat maxed out at 512 bits, with Intel going back to 256 bits for non-server CPUs. I don't see larger widths on the horizon (even if SVE theoretically supports up to 2048 bits, I don't know any implementation with ~~>256~~ >512 bits). Larger bit widths are not beneficial for most applications and the few applications that are (e.g., some HPC codes) are nowadays served by GPUs. - The post mentions available opcode space: while opcode space is limited, a reasonably well-designed ISA (e.g., AArch64) has enough holes for extensions. Adding new instructions doesn't require ABI changes, and while adding new registers requires some kernel changes, this is well understood at this point. - "What is worse, software developers often have to target several SIMD generations" -- no way around this, though, unless auto-vectorization becomes substantially better. Adjusting the register width is not the big problem when porting code, making better use of available instructions is. - "The packed SIMD paradigm is that there is a 1:1 mapping between the register width and the execution unit width" -- no. E.g., AMD's Zen 4 does double pumping, and AVX was IIRC originally designed to support this as well (although Intel went directly for 256-bit units). - "At the same time many SIMD operations are pipelined and require several clock cycles to complete" -- well, they are pipelined, but many SIMD instructions have the same latency as their scalar counterpart. - "Consequently, loops have to be unrolled in order to avoid stalls and keep the pipeline busy." -- loop unroll has several benefits, mostly to reduce the overhead of the loop and to avoid data dependencies between loop iterations. Larger basic blocks are better for hardware as every branch, even if predicted correctly, has a small penalty. "Loop unrolling also increases register pressure" -- it does, but code that really requires >32 registers is extremely rare, so a good instruction scheduler in the compiler can avoid spilling. In my experience, dynamic vector sizes make code slower, because they inhibit optimizations. E.g., spilling a dynamically sized vector is like a dynamic stack allocation with a dynamic offset. I don't think SVE delivered any large benefits, both in terms of performance (there's not much hardware with SVE to begin with...) and compiler support. RISC-V pushes further into this direction, we'll see how this turns out.

cherryteastain 1y ago
Fujitsu A64FX used in the Fugaku supercomputer uses SVE with 512 bit width
- aengelke 1y ago
  Thanks, I misremembered. However, the microarchitecture is a bit "weird" (really HPC-targeted), with very long latencies (e.g., ADD (vector) 4 cycles, FADD (vector) 9 cycles). I remember that it was much slower than older x86 CPUs for non-SIMD code, and even for SIMD code, it took quite a bit of effort to get reasonable performance through instruction-level parallelism due to the long latencies and the very limited out-of-order capacities (in particular the just 2x20 reservation station entries for FP).
camel-cdr 1y ago
> we somewhat maxed out at 512 bits Which still means you have to write your code at least thrice, which is two times more than with a variable length SIMD ISA. Also there are processors with larger vector length, e.g. 1024-bit: Andes AX45MPV, SiFive X380, 2048-bit: Akeana 1200, 16384-bit: NEC SX-Aurora, Ara, EPI > no way around this You rarely need to rewrite SIMD code to take advantage of new extensions, unless somebody decides to create a new one with a larger SIMD width. This mostly happens when very specialized instructions are added. > In my experience, dynamic vector sizes make code slower, because they inhibit optimizations. Do you have more examples of this? I don't see spilling as much of a problem, because you want to avoid it regardless, and codegen for dynamic vector sizes is pretty good in my experience. > I don't think SVE delivered any large benefits Well, all Arm CPUs except for the A64FX were build to execute NEON as fast as possible. X86 CPUs aren't built to execute MMX or SSE or the latest, even AVX, as fast as possible. Anyway, I know of one comparison between NEON and SVE: https://solidpixel.github.io/astcenc_meets_sve https://solidpixel.github.io/astcenc_meets_sve > Performance was a lot better than I expected, giving between 14 and 63% uplift. Larger block sizes benefitted the most, as we get higher utilization of the wider vectors and fewer idle lanes. > I found the scale of the uplift somewhat surprising as Neoverse V1 allows 4-wide NEON issue, or 2-wide SVE issue, so in terms of data-width the two should work out very similar.

cjbgkagh 1y ago

I have similar thoughts, I don't understand the push for variable width SIMD. Possibly due to ignorance but I think it's an abstraction that can be specialized for different hardware so the similar tradeoffs between low level languages and high level languages apply. Since I already have to be aware of hardware level concepts such as 256bit shuffle not working across 128bit lanes and different instructions having very different performance characteristics on different CPUs I'm already knee deep in hardware specifics. While in general I like abstractions I've largely given up waiting for a 'sufficiently advanced compiler' that would properly auto-vectorize my code. I think AGI AI is more likely to happen sooner. At a guess it seems to be that SIMD code could work on GPUs but GPU code has different memory access costs so the code there would also be completely different. So my view is either create a much better higher level SIMD abstraction model with a sufficiently advanced compiler that knows all the tricks or let me work closely at the hardware level. As an outsider who doesn't really know what is going on it does worry me a bit that it appears that WASM is pushing for variable width SIMDs instead of supporting ISAs generally supported by CPUs. I guess it's a portability vs performance tradeoff - I worry that it may be difficult to make variable as performant as fixed width and would prefer to deal with portability by having alternative branches at code level. >> Finally, any software that wants to use the new instruction set needs to be rewritten (or at least recompiled). What is worse, software developers often have to target several SIMD generations, and add mechanisms to their programs that dynamically select the optimal code paths depending on which SIMD generation is supported. Why not marry the two and have variable width SIMD as one of the ISA options and if in the future variable width SIMD become more performant then it would just be another branch to dynamically select.

kevingadd 1y ago
Part of the motive behind variable width SIMD in WASM is that there's intentionally-ish no mechanism to do feature detection at runtime in WASM. The whole module has to be valid on your target, you can't include a handful of invalid functions and conditionally execute them if the target supports 256-wide or 512-wide SIMD. If you want to adapt you have to ship entire modules for each set of supported feature flags and select the correct module at startup after probing what the target supports. So variable width SIMD solves this by making any module using it valid regardless of whether the target supports 512-bit vectors, and the VM 'just' has to solve the problem of generating good code. Personally I think this is a terrible way to do things and there should have just been a feature detection system, but the horse fled the barn on that one like a decade ago.

camel-cdr 1y ago

> I prefer fixed width Do you have examples for problems that are easier to solve in fixed-width SIMD? I maintain that most problems can be solved in a vector-length-agnostic manner. Even if it's slightly more tricky, it's certainly easier than restructuring all of your memory allocations to add padding and implementing three versions for all the differently sized SIMD extensions your target may support. And you can always fall back to using a variable-width SIMD ISA in a fixed-width way, when necessary.

jcranmer 1y ago
There's a category of autovectorization known as Superword-Level Parallelism (SLP) which effectively scavenges an entire basic block for individual instruction sequences that might be squeezed together into a SIMD instruction. This kind of vectorization doesn't work well with vector-length-agnostic ISAs, because you generally can't scavenge more than a few elements anyways, and inducing any sort of dynamic vector length is more likely to slow your code down as a result (since you can't do constant folding). There's other kinds of interesting things you can do with vectors that aren't improved by dynamic-length vectors. Something like abseil's hash table, which uses vector code to efficiently manage the occupancy bitmap. Dynamic vector length doesn't help that much in that case, particularly because the vector length you can parallelize over is itself intrinsically low (if you're scanning dozens of elements to find an empty slot, something is wrong). Vector swizzling is harder to do dynamically, and in general, at high vector factors, difficult to do generically in hardware, which means going to larger vectors (even before considering dynamic sizes), vectorization is trickier if you have to do a lot of swizzling. In general, vector-length-agnostic is really only good for SIMT-like codes, which you can express the vector body as more or less independent f(index) for some knowable-before-you-execute-the-loop range of indices. Stuff like DAXPY or BLAS in general. Move away from this model, and that agnosticism becomes overhead that doesn't pay for itself. (Now granted, this kind of model is a large fraction of parallelizable code, but it's far from all of it).
- camel-cdr 1y ago
  The SLP vectorizer is a good point, but I think it's, in comparison with x86, more a problem of the float and vector register files not being shared (in SVE and RVV). You don't need to reconfigure the vector length; just use it at the full width. > Something like abseil's hash table If I remember this correctly, the abseil lookup does scale with vector length, as long as you use the native data path width. (albeit with small gains) There is a problem with vector length agnostic handling of abseil, which is the iterator API. With a different API, or compilers that could eliminate redundant predicated load/stores, this would be easier. > good for SIMT-like codes Certainly, but I've also seen/written a lot of vector length agnostic code using shuffles, which don't fit into the SIMT paradigm, which means that the scope is larger than just SIMT. --- As a general comparison, take AVX10/128, AVX10/256 and AVX10/512, overlap their instruction encodings, remove the few instructions that don't make sense anymore, and add a cheap instruction to query the vector length. (probably also instructions like vid and viota, for easier shuffle synthesization) Now you have a variable-length SIMD ISA that feels familiar. The above is basically what SVE is.

tonetegeatinst 1y ago

AFAIK about every modern CPU uses out of order von Neumann architecture. The only people who don't are the handful of researchers and people who work with the government research into non van Neumann designed systems.

luyu_wu 1y ago
Low power RISC cores (both ARM and RISC-V) are typically in-order actually! But any core I can think of as 'high-performance' is OOO.
- whaleofatw2022 1y ago
  MIPS as well as Alpha AFAIR. And technically itanium, otoh It seems to me a bit like a niche for any performance advantages...
  - mattst88 1y ago
    Alpha 21264 is out-of-order.
  - pezezin 1y ago
    I would not call neither MIPS, Alpha nor Itanium "high-performance" in 2025...
  - p_l 1y ago
    Alpha was out of order starting with EV7, but most importantly the entire architecture was designed with eye for both pipeline hazards and out of order execution, unlike VAX that it replaced which made it pretty much impossible
IshKebab 1y ago
Microcontrollers are often in-order.
- Narishma 1y ago
  Are there any microcontrollers that aren't in-order?

PaulHoule 1y ago

In AVX-512 we have a platform that rewards the assembly language programmer like few platforms have since the 6502. I see people doing really clever things that are specific to the system and one level it is really cool but on another level it means SIMD is the domain of the specialist, Intel puts out press releases about the really great features they have for the national labs and for Facebook whereas the rest of us are 5-10 years behind the curve for SIMD adoption because the juice isn't worth the squeeze. Just before libraries for training neural nets on GPUs became available I worked on a product that had a SIMD based neural network trainer that was written in hand-coded assembly. We were a generation behind in our AVX instructions so we gave up half of the performance we could have got, but that was the least of the challenges we had to overcome to get the product in front of customers. [1] My software-centric view of Intel's problems is that they've been spending their customers and shareholders money to put features in chips that are fused off or might as well be fused off because they aren't widely supported in the industry. And that they didn't see this as a problem and neither did their enablers in the computing media and software industry. Just for example, Apple used to ship the MKL libraries which like a turbocharger for matrix math back when they were using Intel chips. For whatever reason, Microsoft did not do this with Windows and neither did most Linux distributions so "the rest of us" are stuck with a fraction of the performance that we paid for. AMD did the right thing in introducing double pumped AVX-512 because at least assembly language wizards have some place where their code runs and the industry gets closer to the place where we can count on using an instruction set defined 12 years ago. [1] If I'd been tasked with updating the to next generation I would have written a compiler (if I take that many derivatives by hand I'll get one wrong.) My boss would have ordered me not to, I would have done it anyway and not checked it in.

bee_rider 1y ago
It is kind of a bummer that MKL isn’t open sourced, as that would make inclusion in Linux easier. It is already free-as-in-beer, but of course that doesn’t solve everything. Baffling that MS didn’t use it. They have a pretty close relationship… Agree that they are sort of going after hard-to-use niche features nowadays. But I think it is just that the real thing we want—single threaded performance for branchy code—is, like, incredibly difficult to improve nowadays.

derf_ 1y ago

> I code with SIMD as the target, and have special containers that pad memory to SIMD width... I think this may be domain-specific. I help maintain several open-source audio libraries, and wind up being the one to review the patches when people contribute SIMD for some specific ISA, and I think without exception they always get the tail handling wrong. Due to other interactions it cannot always be avoided by padding. It can roughly double the complexity of the code [0], and requires a disproportionate amount of thinking time vs. the time the code spends running, but if you don't spend that thinking time you can get OOB reads or writes, and thus CVEs. Masked loads/stores are an improvement, but not universally available. I don't have a lot of concrete suggestions. I also work with a lot of image/video SIMD, and this is just not a problem, because most operations happen on fixed block sizes, and padding buffers is easy and routine. I agree I would have picked other things for the other two in my own top-3 list. [0] Here is a fun one, which actually performs worst when len is a multiple of 8 (which it almost always is), and has 59 lines of code for tail handling vs. 33 lines for the main loop: https://gitlab.xiph.org/xiph/opus/-/blob/main/celt/arm/celt_neon_intr.c#L171 https://gitlab.xiph.org/xiph/opus/-/blob/main/celt/arm/celt_...

deleted 1y ago
[deleted]
jandrewrogers 1y ago
> Masked loads/stores are an improvement, but not universally available. Traditionally we’ve worked around this with pretty idiomatic hacks that efficiently implement “masked load” functionality in SIMD ISAs that don’t have them. We could probably be better about not making people write this themselves every time.
ack_complete 1y ago
It depends on how integrated your SIMD strategy is into the overall technical design. Tail handling is much easier if you can afford SIMD-friendly padding so a full vector load/store is possible even if you have to manually mask. That avoids a lot of the hassle of breaking down memory accesses just to avoid a page fault or setting off the memory checker. Beyond that -- unit testing. I don't see enough of it for vectorized routines. SIMD widths are small enough that you can usually just test all possible offsets right up against a guard page and brute force verify that no overruns occur.

Const-me 1y ago

> I prefer fixed width Another reason to prefer fixed width, compilers may pass vectors to functions in SIMD registers. When register size is unknown at compile time, they have to pass data in memory. For complicated SIMD algorithms the performance overhead gonna be huge.

kjs3 1y ago
Back in the day, you had Cray style vector registers, and you had CDC style[1] 'vector pipes' (I think I remember that's what they called them) that you fed from main memory. So you would (vastly oversimplifying) build your vectors in consecutive memory locations (up to 64k as I recall), point to a result destination in memory and execute a vector instruction. This works fine if there's a close match between cpu speed and memory access speed. The compilers were quite good, and took care of handling variable sized vectors, but I have no idea what was going on under the hood except for some hi-level undergrad compiler lectures. As memory speed vs cpu speed divergence became more and more pronouced, it quickly became obvious that vector registers were the right performance answer, basically everyone jumped that way, and I don't think anyone has adopted a memory-memory vector architecture since the '80s. [1] from CDC STAR-100 and followons like the CDC Cyber 180/990, Cyber 200 series & ETA-10.