Lexing is more-or-less regular expression searching. Most regex engines are a mix of CPU- and memory-bound when searching for complicated REs (as one would have

Lexing is more-or-less regular expression searching. Most regex engines are a mix of CPU- and memory-bound when searching for complicated REs (as one would have with a lexer; lex just combines patterns into a giant hardcoded DFA). So, there’s room on the CPU for multithreading to improve performance, but it must be done carefully to minimize both contention and cache evictions.