12 ms·

> GNU grep really doesn't do all that well for multi-byte encodings. I might be wrong, but I have a recollection that this was massively improved somewhat rece

by philh 10y ago

> GNU grep really doesn't do all that well for multi-byte encodings. I might be wrong, but I have a recollection that this was massively improved somewhat recently, maybe a year or so back. Are you using a recent-ish version? This might also be to do with the needle - I think I was testing with a fixed ASCII string, which should work the same in either the UTF-8 or C locales (assuming a UTF-8 file). But \w I guess should behave differently depending on locale, e.g. I'd expect it to match ß in UTF-8 but not C.

burntsushi 10y ago
Yes, I'm on 2.25, which is the latest I believe. > I might be wrong, but I have a recollection that this was massively improved somewhat recently, maybe a year or so back. I seem to recall hearing that too. A brief skim of the commit log shows multiple possible improvements, but I don't have time to drill down further into that. It could just be that it used to be worse. :-) (If you look in the core DFA loop, which is in the `dfaexec_main` function inside `dfa.c`, then you can see that there is a completely different path through the DFA if it needs to use multibyte encodings.) > But \w I guess should behave differently depending on locale, e.g. I'd expect it to match ß in UTF-8 but not C. Exactly. But this doesn't have to have a measurable performance impact, assuming the number of matches doesn't radically change. (Well, this is a bit of a lie, because a Unicode aware \w is much bigger than an ASCII-only \w, which implies the DFA needs more memory, which could lead to a more general slowdown if the state cache needs to get emptied more frequently.)