19 ms·
Taking a set of regexes and compiling them all into one automaton is exactly what RegexSet is: https://docs.rs/regex/latest/regex/struct.RegexSet.html https://d
by burntsushi 4y ago
Taking a set of regexes and compiling them all into one automaton is exactly what RegexSet is: https://docs.rs/regex/latest/regex/struct.RegexSet.html https://docs.rs/regex/latest/regex/struct.RegexSet.html
However, when you're talking about thousands of regexes, that's going to be a very large automaton. Probably impractically large. It's not NP-hard. It's "just" impractical.
Using heuristics to whittle down the set of possible regexes to match---likely using literals extracted from each regex---is exactly what you want to do for a problem like this. If all you have are literals, an Aho-Corasick automaton is feasible to build for 90,000 entries. Aho-Corasick, in my experience, doesn't really start to break down until you eclipse 1,000,000 entries.