Copying my feedback from reddit[1], where I discussed it in the context of the `memchr` crate.[2] I took a quick look at your library implementation and have s

Copying my feedback from reddit[1], where I discussed it in the context of the `memchr` crate.[2] I took a quick look at your library implementation and have some notes: * It doesn't appear to query CPUID, so I imagine the only way it uses AVX2 on x86-64 is if the user compiles with that feature enabled explicitly. (Or uses something like [`x86-64-v3`](https://en.wikipedia.org/wiki/X86-64#Microarchitecture_levels https://en.wikipedia.org/wiki/X86-64#Microarchitecture_level...).) The `memchr` crate doesn't need that. It will use AVX2 even if the program isn't compiled with AVX2 enabled so long as the current CPU supports it. * Your substring routines have multiplicative worst case (that is, `O(m * n)`) running time. The `memchr` crate only uses SIMD for substring search for smallish needles. Otherwise it flips over to Two-Way with a SIMD prefilter. You'll be fine for short needles, but things could go very very badly for longer needles. * It seems quite likely that your [confirmation step](https://github.com/ashvardanian/Stringzilla/blob/fab854dc4fd99a072fba1319810ad3ccf682238a/stringzilla/stringzilla.h#L353-L358 https://github.com/ashvardanian/Stringzilla/blob/fab854dc4fd...) is going to absolutely kill performance for even semi-frequently occurring candidates. The [`memchr` crate utilizes information from the vector step to limit where and when it calls `memcmp`](https://github.com/BurntSushi/memchr/blob/46620054ff25b16d22e6e7ab563a6bde6791ee60/src/arch/generic/packedpair.rs#L243-L254 https://github.com/BurntSushi/memchr/blob/46620054ff25b16d22...). Your code might do well in cases where matches are very rare. I took a quick peek at your benchmarks and don't see anything that obviously stresses this particular case. For substring search, the `memchr` crate uses a variant of the "[generic SIMD](http://0x80.pl/articles/simd-strfind.html#first-and-last http://0x80.pl/articles/simd-strfind.html#first-and-last)" algorithm. Basically, it takes two bytes from the needle, looks for positions where those occur and then attempts to check whether that position corresponds to a match. It looks like your technique uses the first 4 bytes. I suspect that might be overkill. (I did try using 3 bytes from the needle and found that it was a bit slower in some cases.) That is, two bytes is usually enough predictive power to lower the false positive rate enough. Of course, one can write pathological inputs that cause either one to do better than the other. (The `memchr` crat benchmark suite has a [collection of pathological inputs](https://github.com/BurntSushi/memchr/blob/46620054ff25b16d22e6e7ab563a6bde6791ee60/benchmarks/definitions/memmem/pathological.toml https://github.com/BurntSushi/memchr/blob/46620054ff25b16d22...).) It would actually be possible to hook Stringzilla up to `memchr`'s benchmark suite if you were interested. :-) [1]: https://old.reddit.com/r/rust/comments/163ph8r/memchr_26_now_uses_simd_for_substring_search_on/jy7zuwo/ https://old.reddit.com/r/rust/comments/163ph8r/memchr_26_now... [2]: https://github.com/BurntSushi/memchr https://github.com/BurntSushi/memchr