3 ms·
For the year 1970 and after: never. This should always be part of the parser for reasons of being confidently correct. Thinking otherwise just leads to sufferi
by bmn__ 4y ago
For the year 1970 and after: never. This should always be part of the parser for reasons of being confidently correct.
Thinking otherwise just leads to suffering. The problem with tutorials like Tiny Compiler or Crafting Interpreters is that the authors do not run into problems with the code shown in their teaching materials, but as soon as a student wants to apply it to modestly complex grammars, it stops working. The authors traded conciseness for correctness which IMO is a bad trade-off, especially since a reasonably complete implementation of a parsing algorithm that has no shortcomings is perhaps only four to six times longer than a short and flawed one.
The point of this critique is to raise awareness; each student should not have to figure out that they got the bad end of said trade-off by trial and error, instead the teaching material should make this clear initially.
- deterministic 4y agoI am a professional compiler writer and nope you are wrong. I never tokenise everything up front and instead read tokens on demand. It is much faster and enables me to change how tokens are interpreted on the fly. Which enables me to parse different languages with different token definitions in the same file. Something that I need for the production work I do.
- bmn__ 4y ago>>> when is it useful, if ever, to tokenize the whole input beforehand >> never > nope you are wrong. I never tokenise everything up front Explain how you could possibly arrive at the complete opposite meaning of what I wrote when you even use the same word "never" as I did? Quick, get another downvote in, that'll teach me! Peak HN moment here.
- deterministic 4y agoSorry I clearly misunderstood your point. My bad. I upvoted your correction.