3 ms·
Is that decision time-based? Because in terms of exercising the regexen and hand-written functions, they are surely run many thousands of times. I've concatena
by eliben 13y ago
Is that decision time-based? Because in terms of exercising the regexen and hand-written functions, they are surely run many thousands of times.
I've concatenated a number of Tablegen files together (the benchmark is described better here: http://eli.thegreenplace.net/2013/06/25/regex-based-lexical-analysis-in-python-and-javascript/ http://eli.thegreenplace.net/2013/06/25/regex-based-lexical-...) for a total of ~20KLOC, so we're talking about tens of thousands of tokens here (around a megabyte of data). That sounds like enough to kick the JITs, don't you think?
- azakai 13y agoI honestly do not know how regex JITs are tuned, but in JS JITs, even tens of thousands of iterations may not suffice. Just to be sure, I would concatenate some more stuff ;) so the test runs for say 10 seconds.
- eliben 13y agoWhile it seems strange to me that a highly repetitive computation that lasts ~0.2 seconds would not kick the JITs (who needs them, then ;-) ?), I tried a simple blow-up of the input (by abusing "cat") - same results. For a 16MB input with > 2.6 million tokens, the handwritten lexer smokes the single-alternating-regex one: 1.1 sec vs. 2.9 sec.
- azakai 13y agoNice, thanks for verifying that. Was a longshot, but might as well rule out everything possible.