5 ms·
Faster substring search with SIMD in Zig
- lokeg 1y agoWhat about the worst case? I.e. something like searching for 1000 'a's in a long string of 'a's interspersed with 'b's every 500-1000 steps? Seems accidentally quadradic unfortunately in the absence of some KMP-like fallback
- expenses3 1y agoHow is it quadratic? You do 1000 checks every character in the haystack but that's still O(n)
- burntsushi 1y agoThe worst case is that `std.mem.eql`[1] is executed at every position in the haystack, which gives you `O(n*m)` time complexity. Several substring search algorithms are `O(n)`. [1]: https://github.com/aarol/substr/blob/9392f9557de735929dfb79efa4fc88115341c65d/src/main.zig#L82 https://github.com/aarol/substr/blob/9392f9557de735929dfb79e...
- MattPalmer1086 1y agoWorst case for these types of search is O(mn), m length of needle, n length of haystack. It is not linear in n. The absolute worst case is when both the needle and haystack are both composed of the same byte repeated (e.g. all zero).
- codethief 1y agoNice article! Also, this might be a stupid question (I'm a Zig newbie) but… instead of calling std.mem.eql() in the while loop to look at each potential match individually, couldn't you repeat the same trick as before? That is, use SIMD to search for the second and second-to-last character of the needle, then third and third-to-last, and so on, and finally take a bitwise AND of all the resulting bit masks? This way, one would avoid looking at each potential match one by one, and instead look at all of them at the same time. Even if that doesn't work for some reason and you still need to loop over all potential matches individually, couldn't you use SIMD inside the while loop to replace std.mem.eql and thereby speed up string comparison? My understanding was that std.mem.eql loops over bytes one by one and compares them?
- ncruces 1y agoKnowing little about zig, std.mem.eql very likely already uses SIMD. This is about using SIMD to avoid even calling std.mem.eql for 99% of the possible attempts.
- llimllib 1y agostd.mem.eql is here, super easy to read: https://github.com/ziglang/zig/blob/master/lib/std/mem.zig#L702-L704 https://github.com/ziglang/zig/blob/master/lib/std/mem.zig#L... My read is it would use SIMD if T is @Vector, and not otherwise? But I'm neither a zig nor SIMD expert
- ozgrakkurt 1y agoPretty sure that compiles into assembly for any primitive type like integers, floats etc.
- IshKebab 1y agoWhat do you mean it "compiles into assembly"?
- ozgrakkurt 1y agoI meant to write SIMD
- suddenlybananas 1y ago>The difference between 4μs vs 1μs is extremely small, but it’s slightly faster nonetheless. Put that in a loop and its an enormous speed-up.
- jiehong 1y agoNice! But, does that work with non-ascii characters? (aka Unicode).
- codethief 1y agoI suppose generalizing the approach to UTF-32 should be straightforward, but variable-length encodings like UTF-8 and UTF-16 might be more involved(?) Either way, I'm sure BurntSushi found a solution and built it into ripgrep.
- burntsushi 1y agoripgrep always deals with UTF-8. When it sees a different encoding, like UTF-16, ripgrep first transcodes to UTF-8 and then searches. This is absolutely in part because of all of the byte oriented optimizations that are baked into ripgrep (and its regex engine). Note that I said a part. Making ripgrep (and its regex engine) work on things other than a sequence of bytes is far more difficult than just porting a bunch of SIMD algorithms. There are also many optimizations and architectural constraints in the code based on the alphabet size. That is, with 8-bit integers, its alphabet size is 256. With 16-bit integers, the alphabet size is 65,536.
- tialaramex 1y agoI think this is the right choice because in practice UTF-8 "won" just like how the two's complement machine integer won. It's pretty good, Wikipedia has a brief section explaining how Ken Thompson for example made it self-synchronizing, which seems like a "duh" feature today but the concept before Ken touched it didn't have this. It's a Best Common Practice for the Internet, it's the default in most modern systems and places such as Java's virtual machine or Windows which can't easily "just" use UTF-8 have nevertheless gradually shifted toward being very friendly toward it.
- llimllib 1y agoKind of! This script is assuming that you're dealing with a byte slice, which means you've already encoded your unicode data. If you just encoded your string to bytes naïvely, it will probably-mostly still work, but it will get some combining characters wrong if they're represented differently in the two sources you're comparing. (eg, e-with-an-accent-character vs. accent-combining-character+e) If you want to be correct-er you'll normalize your UTF string[1], but note that there are four different defined ways to do this, so you'll need to choose the one that is the best tradeoff for your particular application and data sources. [1]: https://en.wikipedia.org/wiki/Unicode_equivalence#Normalization https://en.wikipedia.org/wiki/Unicode_equivalence#Normalizat...
- ashvardanian 1y agoI like that more people are getting involved with SIMD, and there have been several posts lately on both memmem-like and memcpy-like operations implemented in SIMD in different programming languages. In most cases, though, these still focus on AVX/NEON instructions from over 10 years ago, rather than newer and more powerful AVX-512 variations, SVE & SVE2, or RVV. These newer ISAs can noticeably change how one would implement a state-of-the-art substring search or copy/move operation. In my projects, such as StringZilla, I often use mask K registers (https://github.com/ashvardanian/StringZilla/blob/2f4b1386ca2ed3c4178c1e4f467a2e78a911f3b2/include/stringzilla/stringzilla.h#L4977-L5072 https://github.com/ashvardanian/StringZilla/blob/2f4b1386ca2...) and an input-dependent mix of temporal and non-temporal loads and stores (https://github.com/ashvardanian/StringZilla/blob/2f4b1386ca2ed3c4178c1e4f467a2e78a911f3b2/include/stringzilla/stringzilla.h#L4771-L4847 https://github.com/ashvardanian/StringZilla/blob/2f4b1386ca2...). In typical cases, the difference between the suggested SIMD kernels and the state-of-the-art can be as significant as 50% in throughput. As SIMD becomes more widespread, it would be beneficial to focus more on delivering software and bundling binaries, rather than just the kernels.
- moregrist 1y agoI’m not as familiar with the NEON side, but AVX512 support is pretty variable on new processors. Alder Lake omits it entirely. So we’re still in a world where AVX2 is the lowest common denominator for a system library that wants wide support.
- debugnik 1y agoEven that is too high of a requirement if your target user runs low end hardware. Most Intel chips launched between 2017 and 2021 under the Pentium Silver/Gold and Celeron brands lack AVX (the first one, let alone AVX2).
- teo_zero 1y agoHow strange! I was about to add a comment that I would probably stick to SSE2 or something like that to be sure my code suits as large an audience as possible, including CPUs from more than 10 years ago, ARM, etc. Case in point: I've been very disappointed lately when I wanted to try Ghostty on my laptop and the binary compiled for Debian failed to run due to an invalid instruction. I don't want to force the same experience to others.
- lukaslalinsky 1y agoI really wish Zig decided to add SIMD intrisics. There are many SIMD algorithms that can be done, but you have to switch back to C for those, because they depend on operations outside of what LLVM provides for vectors.
- steeve 1y agoLike what? You can also directly call LLVM intrinsics in case this doesn’t work
- lukaslalinsky 1y agoJust a couple of days ago, I wanted to implement specialized StreamVByte decoding in Zig, but @shuffle() needs to mask to be compile time known, while _mm_shuffle_epi8() works just fine with a dynamic mask. I remember that some time ago, I couldn't find an alternative to _mm_alignr_epi8().
- aqrit 1y ago`_mm_alignr_epi8` is a compile-time known shuffle that gets optimized well by LLVM [1]. If you need the exact behavior of `pshufb` you can use asm or the llvm intrinsic [2]. iirc, I once got the compiler to emit a `pshufb` for a runtime shuffle... that always guaranteed indices in the 0..15 range? Ironically, I also wanted to try zig by doing a StreamVByte implementation, but got derailed by the lack of SSE/AVX intrinsics support. [1] https://github.com/aqrit/sse2zig/blob/444ed8d129625ab5deec345ac5fdb06f6f9d0c6c/sse.zig#L3501C15-L3501C30 https://github.com/aqrit/sse2zig/blob/444ed8d129625ab5deec34... [2] https://github.com/aqrit/sse2zig/blob/444ed8d129625ab5deec345ac5fdb06f6f9d0c6c/sse.zig#L3830 https://github.com/aqrit/sse2zig/blob/444ed8d129625ab5deec34...
- lukaslalinsky 1y agoOh, that's actually quite neat, it did not occur to me that you can use @shuffle with a compile time mask and it will optimize it to a specialized instruction.
- burntsushi 1y agoGood write-up. This is a very popular approach to substring search! It is still worst case `O(m*n)` though. Do you have a fallback implementation like the `memchr` crate has to guarantee `O(m+n)`? I'll also push back on some bits in the end: > But if it’s so much better, then why haven’t I made a pull request to > change std.mem.indexOf to use SIMD? Well, the reason is that > > std.mem.indexOf is generic over element size, and having a size > larger than u8 makes the algorithm much slower > > The algorithm used in stdmem.indexOf is cross-platform, while the > SIMD code wouldn’t be. (not all platforms have SIMD registers at all, > Arm has only 128-bit) Does Zig not have a way to specialize this for sequences of unsigned 8-bit integers? If not, and you're thereforce force to used a more generic algorithm, that seems pretty unfortunate. > Substring searching is rarely the bottleneck in programs, > especially ones written in a fast language like Zig. That’s why > I don’t personally think it would be worth it to add it to the > standard library. Oh I'm not sure I buy this at all! Substring search is a primitive operation and easily can be a bottleneck. There's a reason why widely used substring search implementations tend to be highly optimized.
- ozgrakkurt 1y agoIt is very easy to specialise a function in zig, you just put if(T == u8) or something like that inside the function and do w/e in there
- unwind 1y agoCan the compiler detect that and use the proper code so no test is needed at runtime? This is Zig so I guess the answer is "yeah, duh" but wanted to ask since it sounded like the solution is less, uh, "compiler-friendly" than I would expect.
- hansvm 1y agoYes, and if you're paranoid you can write if (comptime T == u8) { // code } to guarantee that if you're wrong about how the compiler behaves then you'll get a compiler error.
- garganzol 1y agoEvery language tries to implement the best in class memory set/search primitives. Maybe we should move them to something called libos, where they will be implemented by every host OS? Then, OS manufacturers can supply libos.so/libos.dll/libos.dylib as part of the official OS distributions. If the wheels get reinvented again and again, it means that they should be readily available.
- ozgrakkurt 1y agoIsn’t this what libc is? Like musl-libc or glibc
- garganzol 1y agolibc is de jure for C language in Unix-like systems. De facto, it is used for other things and by other languages creating a bit of an architectural mess. What I am talking about is creating a cross-vendor standard.
- vitalnodo 1y agoThere is an existing concept called an exokernel. Moreover, I think Zig could be very suitable for this, because it already encourages passing allocators to functions to make memory allocation explicit. Recently, I/O has been (and is still being) reworked in a similar way.
- deleted 1y ago[deleted]
- ww520 1y agoExcellent write up. I really appreciate the clear and detailed explanation of the algorithm. The nice thing about Zig’s SIMD operation is the register size support is transparent. You can just declare a 64-byte vector as the Block, and Zig would use an AVX256 or two AVX2 (32-byte) registers behind the scene. All other SIMD operations on the type are transparently done with regard to the registers when compiled to the targeted platform. Even using two AVX2 registers for 64 bytes of data is a win due to instruction pipelining. Most CPU have multiple SIMD registers and the two 32-byte data chunks are independent. CPU can run them in parallel. The next optimization is to line the data up at 64 byte alignment, to match the L1/L2 cache line size. Memory access is slow in general.
- burntsushi 1y agoAnother challenge may be as a result of using a portable SIMD API instead of specific ISA instructions. I'm specifically thinking about computing the mask, which on x86-64 is I assume implemented via movemask. But aarch64 lacks such an instruction, so you need to do other shenanigans for the best codegen: https://github.com/BurntSushi/memchr/blob/ceef3c921b5685847ea39647b6361033dfe1aa36/src/vector.rs#L358 https://github.com/BurntSushi/memchr/blob/ceef3c921b5685847e...
- ww520 1y agoA language level or std library level poly-fill for the missing SIMD operations for the platforms would be great.
- okr 1y agoI remember reading Daniel Lemire's Blog about SIMD a few days ago, in which he discussed substring search as well! https://lemire.me/blog/2025/08/09/why-do-we-even-need-simd-instructions/ https://lemire.me/blog/2025/08/09/why-do-we-even-need-simd-i...