3 ms·

I use another regex library that does that and do not like it at all Something like \w makes the regex complexity explode. It spends more time building the reg

I use another regex library that does that and do not like it at all Something like \w makes the regex complexity explode. It spends more time building the regex than actually matching it.

burntsushi 5y ago
You can't get around the Unicode problem for free. If your state machine operates on codepoints, then your state machine is almost certainly very slow. Not only because you now need to do UTF-8 decoding, but you also probably need a sparse representation (e.g., a hash map) to deal with the large space of Unicode codepoints since that's what your transitions are defined over. You're right that Unicode-aware \w is huge if you build a full byte-based DFA for it. But that's why Rust's regex crate doesn't build full DFAs. It only builds the part of the DFA it needs (at search time). So you get the best of both worlds: reasonably fast compilation combined with fast search. (A big enough state machine will still ultimately overwhelm it and result in slower searches or large memory growth, depending on which trade off you pick. But a simple \w is handled just fine.)