4 ms·
I've been working on a parser that parses while typing and gives type and syntax errors as soon as possible but never any false positives. I didn't expect it to
by danybittel 3y ago
I've been working on a parser that parses while typing and gives type and syntax errors as soon as possible but never any false positives. I didn't expect it to be that difficult. I still haven't really grokked it and any bugfix usually ends in a game of whack a mole over integration tests. Any hints?
- ReleaseCandidat 3y agoIt depends on the language you're parsing, of course. But the most important thing is to know when your parser is on the right track again. The (most of the time) easiest (but not necessarily first possible) such point is the next definition statement. And make sure to _always_ consume (at least) one token (even if just adding it to the error expression in the AST), so there is no possibility of an endless loop.
- ivanjermakov 3y agoI can recommend this article: https://matklad.github.io/2023/05/21/resilient-ll-parsing-tutorial.html https://matklad.github.io/2023/05/21/resilient-ll-parsing-tu...
- orthoxerox 3y agoRoslyn (C# compiler) has been designed to quickly parse invalid code while still providing intellisense.
- ww520 3y agoOne way I found helpful in giving precise parsing errors without causing excessive false errors is to have the parser auto-completing the missing tokens with placeholder error nodes. E.g. if (cond1 And ) { ... } The And operator is missing its right operand, but everything else is valid. The parser knows the And operator needs two arms. It can complete the missing arm with an error node. fn parseCondition() { let leftNode = parseBoolExpr() if (lookaheadToken() is And) return parseAnd(leftNode) if (lookaheadToken() is Or) ... return leftNode } fn parseAnd(leftNode) { consumeToken() let rightNode = parseBoolExpr() if (rightNode != null) return AndNode(leftNode, rightNode) else return AndNode(leftNode, ErrorNode("missing operand", location)) } In this way, the conditional expression is parsed to completion without interruption. Multiple errors can be handled with completion and recorded. At the end it's a matter of traversing the AST to print out all the ErrorNodes.
- danybittel 3y agoI do have an error node, and it helps but it's still difficult. If I only typed if (cond & It creates error nodes for if and &.. but none of them should show yet. (squiggly lines) if (cond & * Now it should error on the multiply, except if multiply could also be a pointer.
- ww520 3y agoI would say both cases are errors in their current form. When more code is typed in, it can be parsed to validate again.