3 ms·
I've used PEG parsers for two projects, one written with peg/leg (which leafo mentioned), and one with a significantly modified fork of pyPEG [2]. My method for
by versteegen 10y ago
I've used PEG parsers for two projects, one written with peg/leg (which leafo mentioned), and one with a significantly modified fork of pyPEG [2]. My method for handling incorrect input is equivalent to what you described: I extended the pyPEG grammar to add 'checkpoints'. (Earlier, when using peg/leg I used alternative code blocks that threw errors.) Once a checkpoint is passed, the parser is not allowed to backtrack past it again. This avoids the problem of the parser backtracking all the way to the very beginning of input and producing an error equivalent to 'No'. As a bonus, adding checkpoints speeds up parsing and massively reduces memory usage, because as soon as you hit one you can delete state needed for backtracking. Yes, it does mean that you have to think about whether adding a checkpoint somewhere cuts off valid parses; the checkpoints really ought to be added automatically. Ideally you would organise your rules in such a way that all the alternative things that could appear in some context are in a rule called Foo, and put a checkpoint at the beginning, so if none of them match you get a "expected Foo" error. And if you see an opening quote or bracket or anything like that, immediately add a checkpoint because it has to be closed. This worked very well for me, but I imagine would not scale to something as nightmarish as perl.
Real example:
def expressionList(): return QUES, (expression, STAR, (PLUS, ",", CHECKPNT, expression))
(Note: pyPEG is funky, the ? * and + operators (QUES, STAR, PLUS) precede the thing they act on; I really should change that). Basically in this language if you see a comma in an expressionList, something must follow it. The error produced by this rule is:
On line 79 in rbtest.rbas:
Syntax error: while parsing expressionList, expected expression
dim x as integer = y(a, b,)
^
This is a parser for a DSL extension of FreeBASIC (which I called ReloadBasic). In addition to the DSL stuff it parses just enough of FB to determine the types of all the variables and arguments and the function/sub beginnings and ends. For the small fragment it handles, it often produces better errors than the handwritten recursive-descent FB compiler. (It's also about 1% the speed...) Source is here: [1]
[1] https://bitbucket.org/rbv/ohrrpgce/src/6ba83910c564aa474220e3e66a4f2ce1f7268c1b/reloadbasic/reloadbasic.py?at=master&fileviewer=file-view-default https://bitbucket.org/rbv/ohrrpgce/src/6ba83910c564aa474220e...
[2] https://bitbucket.org/rbv/ohrrpgce/src/6ba83910c564aa474220e3e66a4f2ce1f7268c1b/reloadbasic/pyPEG.py?at=master&fileviewer=file-view-default https://bitbucket.org/rbv/ohrrpgce/src/6ba83910c564aa474220e...