While it seems strange to me that a highly repetitive computation that lasts ~0.2 seconds would not kick the JITs (who needs them, then ;-) ?), I tried a simple

While it seems strange to me that a highly repetitive computation that lasts ~0.2 seconds would not kick the JITs (who needs them, then ;-) ?), I tried a simple blow-up of the input (by abusing "cat") - same results. For a 16MB input with > 2.6 million tokens, the handwritten lexer smokes the single-alternating-regex one: 1.1 sec vs. 2.9 sec.