3 ms·

How do you handle good error messages or error recovery? I've played around with parser generators but I never figured out how to do either in a satisfactory fa

How do you handle good error messages or error recovery? I've played around with parser generators but I never figured out how to do either in a satisfactory fashion. Granted, my hand written parser doesn't do error recovery very well either.

ben509 6y ago
I'm not entirely settled on that. My working approach is if you're writing a compiler, it wants to have an unambiguous grammar and shouldn't even attempt recovery. If input doesn't parse, the compiler can recommend (or just run) the linter. That keeps the common case fast and simple. The linter/fixer can have a relaxed syntax specifically designed to handle messy code and suggest corrections. That comes from a philosophy of treating error correction and user assistance as a separate task. But that's an approach I'm taking because I want to get a reference implementation together as quickly as possible. It's definitely not how modern IDEs work. JetBrains GrammarKit uses a PEG[1] because they need strong support for error recovery[2] using hints. Another interesting library is Tree-sitter[3]; it does incremental parsing keeping an AST constantly up to date for you. Relevant to this whole discussion, GvR wrote a series on PEG parsers[4] in which he starts out by writing one by hand and then shows how to write one that accepts a grammar. [1]: https://github.com/JetBrains/Grammar-Kit https://github.com/JetBrains/Grammar-Kit [2]: https://github.com/JetBrains/Grammar-Kit#attributes-for-error-recovery-and-reporting https://github.com/JetBrains/Grammar-Kit#attributes-for-erro... [3]: https://tree-sitter.github.io/tree-sitter/ https://tree-sitter.github.io/tree-sitter/ [4]: https://medium.com/@gvanrossum_83706/peg-parsing-series-de5d41b2ed60 https://medium.com/@gvanrossum_83706/peg-parsing-series-de5d...
ltratt 6y ago
Alas, most parser generators don't have very good error recovery (and some have such terrible error recovery that I think it's worse than not having any!). It turns out that this isn't inevitable: there's been a long strand of research on decent error recovery for LR parsers, at least, but it needed a bit of a refresh to be practical. If you'll forgive the blatant self promotion, we tackled this problem in https://soft-dev.org/pubs/html/diekmann_tratt__dont_panic/ https://soft-dev.org/pubs/html/diekmann_tratt__dont_panic/ which is implemented in our Rust parsing system https://github.com/softdevteam/grmtools/ https://github.com/softdevteam/grmtools/. It won't beat the very best hand-written error recovery routines, but it's often not far behind.

ben509 6y ago

I'm not entirely settled on that. My working approach is if you're writing a compiler, it wants to have an unambiguous grammar and shouldn't even attempt recovery. If input doesn't parse, the compiler can recommend (or just run) the linter. That keeps the common case fast and simple. The linter/fixer can have a relaxed syntax specifically designed to handle messy code and suggest corrections. That comes from a philosophy of treating error correction and user assistance as a separate task. But that's an approach I'm taking because I want to get a reference implementation together as quickly as possible. It's definitely not how modern IDEs work. JetBrains GrammarKit uses a PEG[1] because they need strong support for error recovery[2] using hints. Another interesting library is Tree-sitter[3]; it does incremental parsing keeping an AST constantly up to date for you. Relevant to this whole discussion, GvR wrote a series on PEG parsers[4] in which he starts out by writing one by hand and then shows how to write one that accepts a grammar. [1]: https://github.com/JetBrains/Grammar-Kit https://github.com/JetBrains/Grammar-Kit [2]: https://github.com/JetBrains/Grammar-Kit#attributes-for-error-recovery-and-reporting https://github.com/JetBrains/Grammar-Kit#attributes-for-erro... [3]: https://tree-sitter.github.io/tree-sitter/ https://tree-sitter.github.io/tree-sitter/ [4]: https://medium.com/@gvanrossum_83706/peg-parsing-series-de5d41b2ed60 https://medium.com/@gvanrossum_83706/peg-parsing-series-de5d...

ltratt 6y ago

Alas, most parser generators don't have very good error recovery (and some have such terrible error recovery that I think it's worse than not having any!). It turns out that this isn't inevitable: there's been a long strand of research on decent error recovery for LR parsers, at least, but it needed a bit of a refresh to be practical. If you'll forgive the blatant self promotion, we tackled this problem in https://soft-dev.org/pubs/html/diekmann_tratt__dont_panic/ https://soft-dev.org/pubs/html/diekmann_tratt__dont_panic/ which is implemented in our Rust parsing system https://github.com/softdevteam/grmtools/ https://github.com/softdevteam/grmtools/. It won't beat the very best hand-written error recovery routines, but it's often not far behind.