3 ms·

I'm not a Perl hacker, but there's one thing that I've wanted a bunch of times in other languages: Perl 6 does for PEGs (Parsing Expression Grammars) what Perl

I'm not a Perl hacker, but there's one thing that I've wanted a bunch of times in other languages: Perl 6 does for PEGs (Parsing Expression Grammars) what Perl 5 did for regexes: they're part of the core language, you can execute Perl expressions when a PEG rule is matched, that kind of thing. In Python when I have some data and it grows just a bit too irregular for regexes, I have to start manually writing state-machines or add a dependency on a third-party library and spend an afternoon explaining parsing theory to my co-workers.

nly 12y ago
I want to see Perl 6 push the boundaries but the world isn't quite like it was when Perl 5 made using regex seemless. Competition is much more stiff. Even in C you can use something like Ian Piumarta's PEG generator[0] that let's you embed semantic actions written in C. For regular grammars, scanners and lexers you have powerful tools like Ragel[1] (which targets 7 languages) I'm not a Python person, so if the Python libraries proved inadequate I'd probably resort to these tools and then push out a Python module using Cyphon[2]. Personally I'm not a fan of using semantic actions for AST construction. I generally prefer the 'attribute' approach taken by libraries like Spirit (C++) which make the process feel a lot more like deserialization. SAs are still useful however for rejecting or accepting rules based on complex requirements (like "is this IP address routable"?). [0] http://piumarta.com/software/peg/ http://piumarta.com/software/peg/ [1] http://www.colm.net/open-source/ragel/ http://www.colm.net/open-source/ragel/ [2] http://cython.org/ http://cython.org/
- rurban 12y ago
  With perl6 the peg backend is built-in, so you can compile and execute grammars on the fly. A bit like lua's lpeg. A perl6 Grammar is a first-class object.
  - nly 12y ago
    Sure, but what happens when someone needs something other than PEG/recursive descent, like expression parsing with a custom precedence table? Will Perl6 add that to the core language as well? Where do you draw the line? LPeg is a library, Spirit X3 (for C++, not yet complete) is a library. Both achieve pretty seamless integration with your code. I think in a library is where this stuff should be.
    - arnsholt 12y ago
      It's pretty easy (to the extent that implementing custom parsing logic is ever easy) to add custom parsing bits to a Perl 6 grammar. The grammar is just a special class with the rules as methods, so rules with non-standard parsing logic can be implemented using Perl 6 code. Reusing standard rules from custom rules is simply a matter of calling the corresponding method. The best example of this is probably the EXPR rule in the NQP compiler (which is in turn used by Rakudo's parser), which inserts an operator precedence parser into the recursive descent of the rest of the grammar.

nly 12y ago

I want to see Perl 6 push the boundaries but the world isn't quite like it was when Perl 5 made using regex seemless. Competition is much more stiff. Even in C you can use something like Ian Piumarta's PEG generator[0] that let's you embed semantic actions written in C. For regular grammars, scanners and lexers you have powerful tools like Ragel[1] (which targets 7 languages) I'm not a Python person, so if the Python libraries proved inadequate I'd probably resort to these tools and then push out a Python module using Cyphon[2]. Personally I'm not a fan of using semantic actions for AST construction. I generally prefer the 'attribute' approach taken by libraries like Spirit (C++) which make the process feel a lot more like deserialization. SAs are still useful however for rejecting or accepting rules based on complex requirements (like "is this IP address routable"?). [0] http://piumarta.com/software/peg/ http://piumarta.com/software/peg/ [1] http://www.colm.net/open-source/ragel/ http://www.colm.net/open-source/ragel/ [2] http://cython.org/ http://cython.org/

rurban 12y ago
With perl6 the peg backend is built-in, so you can compile and execute grammars on the fly. A bit like lua's lpeg. A perl6 Grammar is a first-class object.
- nly 12y ago
  Sure, but what happens when someone needs something other than PEG/recursive descent, like expression parsing with a custom precedence table? Will Perl6 add that to the core language as well? Where do you draw the line? LPeg is a library, Spirit X3 (for C++, not yet complete) is a library. Both achieve pretty seamless integration with your code. I think in a library is where this stuff should be.
  - arnsholt 12y ago
    It's pretty easy (to the extent that implementing custom parsing logic is ever easy) to add custom parsing bits to a Perl 6 grammar. The grammar is just a special class with the rules as methods, so rules with non-standard parsing logic can be implemented using Perl 6 code. Reusing standard rules from custom rules is simply a matter of calling the corresponding method. The best example of this is probably the EXPR rule in the NQP compiler (which is in turn used by Rakudo's parser), which inserts an operator precedence parser into the recursive descent of the rest of the grammar.