3 ms·

Yes, you're right that PEG parsing is usually not the fastest option. As with many things, the operative question is: is it fast _enough_? > Note that this inc

Yes, you're right that PEG parsing is usually not the fastest option. As with many things, the operative question is: is it fast _enough_? > Note that this includes the time for tokenization, which is generally the bulk of the time spent in parsing Hmmm, I've never heard this before and I'm curious if you can point me to any benchmarks or other sources that demonstrate this. If it's true it's surprising to me!

kragen 2y ago
I can't! But if you have a conventional lex/yacc parser handy, it should be easy to test by measuring the time for just lexing a large input file and discarding the tokens. If you do the experiment, I'm interested to hear your results! I don't believe that software is ever fast enough, but trading off speed is often worthwhile, for example to get composability or readability.
- pdubroy 2y ago
  I found some benchmarks in the ANTLR project: https://github.com/antlr/grammars-v4/blob/master/java/java/Benchmarks.md https://github.com/antlr/grammars-v4/blob/master/java/java/B... Project Parsing/Lexing Ratio ---------------------------------------- jdk8 5.34x Spring Framework 3.81x Elasticsearch 5.76x RxJava 5.69x JUnit4 4.51x Guava 5.37x Log4j 2.92x Obviously this is just one toolkit, one language, and one set of examples. But in these benchmarks, the tokenization (lexing) time is a small fraction of the total parse time.
  - kragen 2y ago
    Thank you!
  - vanderZwan 2y ago
    Well, "small" is relative. It's certainly not a bottleneck, but it's still between 17.4% to 34.2% of the total time. That's definitely still in range where optimizing could have a measurable impact on performance difference (depending on how much room there's left for optimization). 100/2.92 = 34.2% 100/5.76 = 17.4%
    - kragen 2y ago
      Right, but it's clearly not the situation I was describing where cutting the parse time to zero for the stages after the tokenization stage would only slightly decrease total time.
jitl 2y ago
In my line of work (writing JavaScript that will run on Android devices) there is no such thing as fast enough. If you’re not going as fast as possible the experience is bad. After all this is a platform where getting data from the local disk is frequently slower than the network because there’s so little CPU & disk bandwidth…
- kragen 2y ago
  Also, if you're computing for 3 milliseconds for every touchmove event instead of 0.3 milliseconds, you'll run the battery down a lot faster. Though not actually ten times faster, because the screen keeps sucking power even when the CPU is asleep.

kragen 2y ago

I can't! But if you have a conventional lex/yacc parser handy, it should be easy to test by measuring the time for just lexing a large input file and discarding the tokens. If you do the experiment, I'm interested to hear your results! I don't believe that software is ever fast enough, but trading off speed is often worthwhile, for example to get composability or readability.

pdubroy 2y ago
I found some benchmarks in the ANTLR project: https://github.com/antlr/grammars-v4/blob/master/java/java/Benchmarks.md https://github.com/antlr/grammars-v4/blob/master/java/java/B... Project Parsing/Lexing Ratio ---------------------------------------- jdk8 5.34x Spring Framework 3.81x Elasticsearch 5.76x RxJava 5.69x JUnit4 4.51x Guava 5.37x Log4j 2.92x Obviously this is just one toolkit, one language, and one set of examples. But in these benchmarks, the tokenization (lexing) time is a small fraction of the total parse time.
- kragen 2y ago
  Thank you!
- vanderZwan 2y ago
  Well, "small" is relative. It's certainly not a bottleneck, but it's still between 17.4% to 34.2% of the total time. That's definitely still in range where optimizing could have a measurable impact on performance difference (depending on how much room there's left for optimization). 100/2.92 = 34.2% 100/5.76 = 17.4%
  - kragen 2y ago
    Right, but it's clearly not the situation I was describing where cutting the parse time to zero for the stages after the tokenization stage would only slightly decrease total time.

jitl 2y ago

In my line of work (writing JavaScript that will run on Android devices) there is no such thing as fast enough. If you’re not going as fast as possible the experience is bad. After all this is a platform where getting data from the local disk is frequently slower than the network because there’s so little CPU & disk bandwidth…

kragen 2y ago
Also, if you're computing for 3 milliseconds for every touchmove event instead of 0.3 milliseconds, you'll run the battery down a lot faster. Though not actually ten times faster, because the screen keeps sucking power even when the CPU is asleep.