4 ms·
> and unicode support would also be a good idea. strings are not just ASCII anymore. GNU grep does have some Unicode support when using the UTF-8 locale. Thing
by burntsushi 6y ago
> and unicode support would also be a good idea. strings are not just ASCII anymore.
GNU grep does have some Unicode support when using the UTF-8 locale. Things like \b and \w are Unicode-aware, for example.
> it uses a very old and outdated substring search algorithm. even ripgrep does not use the state of the art EPSM substring search.
Boyer-Moore is still quite serviceable. So is Two-Way. And neither of them require platform specific vector instructions, so there will be a place for them for a while yet. And as for EPSM, I don't know of any place where that's used in practice. I don't even think Hyperscan implements it. It's hard to beat a well place memchr or a simpler prefilter SIMD approach[1].
> extended pattern matching is not jitted. in 2020 there's no excuse for that.
It's not the case that a JIT is always faster than a DFA (or a lazy DFA in GNU grep's case). So, umm, yeah, there are plenty of excuses for it.
> it cannot do unorm, thus fails to find some unicode patterns. debian shipped for a while with the unorm patches, but this caused huge performance regressions on UTF-8 locales. a lot of work is still ahead.
I don't know of any search tool that does Unicode normalization, probably exactly because of the performance overhead. Unicode normalization negates most if not all of the "clever" optimizations used by GNU grep. It would certainly render the choice of substring algorithm or JIT mostly useless, for example.
[1] - http://0x80.pl/articles/simd-strfind.html#algorithm-1-generic-simd http://0x80.pl/articles/simd-strfind.html#algorithm-1-generi...