4 ms·
I am a professional compiler writer and nope you are wrong. I never tokenise everything up front and instead read tokens on demand. It is much faster and enable
by deterministic 4y ago
I am a professional compiler writer and nope you are wrong. I never tokenise everything up front and instead read tokens on demand. It is much faster and enables me to change how tokens are interpreted on the fly. Which enables me to parse different languages with different token definitions in the same file. Something that I need for the production work I do.
- bmn__ 4y ago>>> when is it useful, if ever, to tokenize the whole input beforehand >> never > nope you are wrong. I never tokenise everything up front Explain how you could possibly arrive at the complete opposite meaning of what I wrote when you even use the same word "never" as I did? Quick, get another downvote in, that'll teach me! Peak HN moment here.
- deterministic 4y agoSorry I clearly misunderstood your point. My bad. I upvoted your correction.