6 ms·

Using SIMD for Parallel Processing in Rust

• eachro 2y ago
  This is cool that simd primitives exist in the std lib of rust. I've wanted wanted to mess around a bit more with simd in python but I don't think that native support exists. Or your have to go down to C/C++ bindings to actually mess around with it (last I checked at least, please correct me if I'm wrong).
  • mroche 2y ago
    Quick search turned up this: SIMD in Pure Python https://www.da.vidbuchanan.co.uk/blog/python-swar.html https://www.da.vidbuchanan.co.uk/blog/python-swar.html Don't let the "SIMD in Python" section fool you, it's a short stop on Numpy before putting it aside.
  • bardak 2y ago
    I feel like most languages could use simd in the standard library. We have all this power in the vector units of our CPUs that compilers struggle to use but yet we also don't make it easy to do manually
    • neonsunset 2y ago
      C# is the language that is doing this exact thing, with the next two close options being Swift and, from my understanding, Mojo. Without easy to use SIMD abstraction, many* of .NET's CoreLib functions would have been significantly slower. * UTF-8 validation, text encoding/decoding, conversion to/from hex bytes, copying data, zeroing, various checksum and hash functions, text/element counting, searching, advanced text search with multiple algorithms under SearchValues type used by Regex engine, etc.
      • pjmlp 2y ago
        D as well.
  • Calavar 2y ago
    What would native SIMD support entail in a language without first party JIT or AOT compilation?
    • runevault 2y ago
      At some point bytecode still turns into CPU instructions, so if you added syntax or special functions that went to parts of the interpreter that are SIMD you could certainly add it to a purely interpreted language.
      • Calavar 2y ago
        If we're talking low level SIMD, like opcode level, I'm really struggling to see the use case for interpreted bytecode. The cost of type checking operands to dynamically dispatch down a SIMD path would almost certainly outweigh the savings of the SIMD path itself. JIT is different because in function-level JIT, you can check types just once at the opening of the function, then you stay on the SIMD happy path for the rest of the function. And in AOT, you may able to elide the checks entirely. There is certainly a space for higher level SIMD functionality. Numpy is one example.
• thomashabets2 2y ago
  The portable SIMD is quite nice. We can't really trust a "sufficiently smart compiler" to make the best SIMD decisions, since it may not see through what you're actually doing. https://blog.habets.se/2024/04/Rust-is-faster-than-C.html https://blog.habets.se/2024/04/Rust-is-faster-than-C.html and code at https://github.com/ThomasHabets/zipbrute/blob/master/rust/src/main.rs https://github.com/ThomasHabets/zipbrute/blob/master/rust/sr... showed me getting 3x faster using portable SIMD, on my first attempt.
• nbrempel 2y ago
  Thanks for reading everyone. I’ve gotten some feedback over on Reddit as well that the example is not effectively showing the benefits of SIMD. I plan on revising this. One of my goals of writing these articles is to learn so feedback is more than welcome!
  • KineticLensman 2y ago
    Great read! > One of my goals of writing these articles is to learn so feedback is more than welcome! When I went into the Rust playground to see the assembly output for the Cumulative Sum example, I could only get it to show the compiler warnings, not the actual assembly. I'm probably doing something wrong, but for me this was a barrier that detracted from the article. I'd suggest incorporating the assembly directly into the article, although keeping the playground link for people who are more dedicated / competent than I am.
    • the8472 2y ago
      The function has to be made pub so it doesn't get optimized out as unusued private function. Godbolt is a better choice for looking at asm anyway. https://rust.godbolt.org/z/3Y9ovsoz9 https://rust.godbolt.org/z/3Y9ovsoz9
      • KineticLensman 2y ago
        Ah, that worked, thanks! Although I can now see why he didn't include the output directly.
      • hayley-patton 2y ago
        Narrator: "The code did not, in fact, auto-vectorise." (There's only addsd/movsd instructions, which are add/move scalar-double; we want addpd/movpd which are add/move packed-double in vectorised code.)
  • devit 2y ago
    Are you really writing them? Seems written by an LLM for the most part.
• oconnor663 2y ago
  There are a lot of factors that go into how fast a hash function is, but the case we're showing in the big red chart at https://github.com/BLAKE3-team/BLAKE3 https://github.com/BLAKE3-team/BLAKE3 is almost entirely driven by SIMD. It's a huge deal.
• anonymousDan 2y ago
  The interesting question for me is whether Rust makes it easier for the compiler to extract SIMD parallelism automatically given the restrictions imposed by its type system.
  • pcwalton 2y ago
    The main thing I can think of that would help here is the fact that Rust has stricter alignment requirements than C++ does. Any live reference can more or less be assumed to point to validly-aligned memory at all times, which isn't true in C++. As to whether LLVM actually takes advantage of this effectively, I don't know. I know that we do supply the necessary attributes to LLVM in most cases, but I haven't looked at the individual transform and optimization passes to see whether they take advantage of this (e.g. emitting movdqa vs. falling back to movdqu).
  • PoignardAzur 2y ago
    Aside from aliasing restrictions, you can use chunked iterators which IIRC make it easier for the compiler to auto-vectorize your loop. The actual code changes very little.
• neonsunset 2y ago
  If you like SIMD and would like to dabble in it, I can strongly recommend trying it out in C# via its platform-agnostic SIMD abstraction. It is very accessible especially if you already know a little bit of C or C++, and compiles to very competent codegen for AdvSimd, SSE2/4.2/AVX1/2/AVX512, WASM's Packed SIMD and, in .NET 9, SVE1/2: https://github.com/dotnet/runtime/blob/main/docs/coding-guidelines/vectorization-guidelines.md https://github.com/dotnet/runtime/blob/main/docs/coding-guid... Here's an example of "checked" sum over a span of integers that uses platform-specific vector width: https://github.com/dotnet/runtime/blob/main/src/libraries/System.Linq/src/System/Linq/Sum.cs#L125-L150 https://github.com/dotnet/runtime/blob/main/src/libraries/Sy... Other examples: CRC64 https://github.com/dotnet/runtime/blob/main/src/libraries/System.IO.Hashing/src/System/IO/Hashing/Crc64.Vectorized.cs https://github.com/dotnet/runtime/blob/main/src/libraries/Sy... Hamming distance https://github.com/dotnet/runtime/blob/main/src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/TensorPrimitives.HammingDistance.cs#L111-L141 https://github.com/dotnet/runtime/blob/main/src/libraries/Sy... Default syntax is a bit ugly in my opinion, but it can be significantly improved with helper methods like here where the code is a port of simdutf's UTF-8 code point counting: https://github.com/U8String/U8String/blob/main/Sources/U8String/Shared/U8Searching.cs#L629-L638 https://github.com/U8String/U8String/blob/main/Sources/U8Str... There are more advanced scenarios. Bepuphysics2 engine heavily leverages SIMD to perform as fast as PhysX's CPU back-end: https://github.com/bepu/bepuphysics2/blob/master/BepuPhysics/CollisionDetection/CollisionTasks/CapsuleBoxTester.cs#L43 https://github.com/bepu/bepuphysics2/blob/master/BepuPhysics... Note that practically none of these need to reach out to platform-specific intrinsics (except for replacing movemask emulation with efficient ARM64 alternative) and use the same path for all platforms, varied by vector width rather than specific ISA.
  • runevault 2y ago
    Funny you mention c#, I started to look at this and I made the mistake of wanting to do string comparison via SIMD, except you can't do it externally because it relies on private internals (note, the built in comparison for c# already does SIMD, you just can't easily reimplement it against the built in string type).
    • 2y ago
• IshKebab 2y ago
  Minor nit: RISC-V Vector isn't SIMD. It's actually like ARM's Scalable Vector Extension. Unlike traditional SIMD the code is agnostic to the register width and different hardware can run the same code with different widths. There is also a traditional SIMD extension (P I think?) but it isn't finished. Most focus has been on the vector extension. I am wondering how and if Rust will support these vector processing extensions.
  • Findecanor 2y ago
    RISC-V's vector extension will have at least 128 bits in application processors, so I think you could set VLEN=128 and just use SIMD algorithms. The P extension is intended more for embedded microcontrollers for which the V extension would be too expensive. It reuses the GPRs at whatever width they are at (32 or 64 bits).
    • camel-cdr 2y ago
      That or you can detect the vector length and specialize for it, just like it's already done on x86 with VLEN 128, 256, and 512 for sse, avx, and avx512.
  • camel-cdr 2y ago
    > RISC-V Vector isn't SIMD Isn't SIMD a subset of vector processors? To that matter, can anybody here provide a proper and useful distinction between the two, that is SIMD and vector ISAs? You imply it's because it's vector length agnostic, but you could take e.g. the SSE encoding, and apart from a few instructions, make it operate on SIMD registers of any length. Wouldn't that also be vector length agnostic, as long as software can query the vector length? I think most people wouldn't call this a vector ISA, and how is this substantially different from dispatching to different implementations for SSE AVX and AVX512? I've also seen people say it's about the predication, which would make AVX512 a vector isa. I've seen others say it's about resource usage and vector chaining, but that is just an implementation detail and can be used or not used on traditional SIMD ISAs to the same extend as on vector ISAs.
    • 2y ago
• ww520 2y ago
  Zig actually has a very nice abstraction for SIMD in the form of vector programming. The size of the vector is agnostic to the underlying cpu architecture. The compiler or LLVM will generate code for using SIMD128, 256, or 512 registers. And you are just programming straight vectors.
  • jvanderbot 2y ago
    Yeah, the article overlooked library support for SIMD. nalgebra had a decent writeup on their ability to squeeze out autovectorization for their vector and matrix types.
  • hansvm 2y ago
    I do generally like their approach. It's especially well suited given how easy comptime allows metaprogramming against the target register size. I wish it had a few more builtins for commonly supported operations without me having to write inline assembly (e.g., runtime LUTs are basically untenable for implementing something like bolt [0] without inline asm), but otherwise the abstraction level is about where I'd like it to be. I usually prefer it to gcc intrinsics, fully inline asm, and other such shenanigans. [0] https://arxiv.org/abs/1706.10283 https://arxiv.org/abs/1706.10283
  • pcwalton 2y ago
    Rust has that too, with nalgebra if you want arbitrary-sized tensors as scientific computing wants, or with glam and similar crates if your needs are more modest as in graphics. In all cases they're SIMD-accelerated.
  • ladyanita22 2y ago
    But zig lacks the intrinsics support, and not ever single simd spec is exposed on the abstraction.
  • ladyanita22 2y ago
    Isn't that what std:simd is for Rust?
• brundolf 2y ago
  std::simd is a delight. I'd never done SIMD before in any language, and it was very easy and natural (and safe!) to introduce to my code, and just automatically works cross-platform. Can't recommend it enough