4 ms·

What's the state of parser generators in Python today? This "TaSu" looks very powerful (not talking about the language class but about tooling and feature set).

What's the state of parser generators in Python today? This "TaSu" looks very powerful (not talking about the language class but about tooling and feature set). Is this getting enough traction to becoming a go-to option? In Java (and maybe C++) ANTLR is a go-to option. Some languages have their own goto options, and others are lacking. When I did a quick research, Python seemed to be on the "lacking" side (multiple not-very-well-maintained-options but no clear winner). Is that changing (or was I missing something)?

frou_dh 4y ago
When I surveyed what was out there last year I wrote down these 3 as the most interesting: https://github.com/lark-parser/lark https://github.com/lark-parser/lark https://github.com/neogeny/TatSu https://github.com/neogeny/TatSu https://github.com/dabeaz/sly https://github.com/dabeaz/sly Lark seems to be the one with traction.
- ptmcg 4y ago
  The 3.0 release of pyparsing includes some nice extensions over its older 2.x code base, including LR parsing support and railroad diagram generation. Also, PEP-8 API naming has been added (but with pre-PEP-8 synonyms defined for legacy compatibility). https://github.com/pyparsing/pyparsing https://github.com/pyparsing/pyparsing (Disclosure: I am the author of pyparsing.)
UncleEntity 4y ago
I played around with the Earley parser that used to be used as part of the CPython build process (who’s name I can’t remember off the top of my head) a bunch and with a lot of hacking I got it into a pretty usable state. It isn’t really that easy to hack on because they do a lot of optimization which is probably one of the major reasons they kicked it out of the python repo. ANTLR does have python as one of the output languages IIRC. There’s also a flex/bison clone written in python whose name I also can’t remember. My current favorite parser generator library, Coco/R, has a python 2 implementation which I’ll maybe get around to porting to py3k one of these days, who knows? But to answer your question I don’t think there’s a go to library on the python side, I’ve played around with a bunch of them and usually just go for re2c/lemon or Coco/R from C(++) since I mainly just poke at these things probably more than I should. The next time I get a few days with nothing to do I plan to see if I can’t whip up a working lpeg (lua’s peg parser VM library thing) in python. Someone did a port a while ago but it is abandoned and, honestly, not all that good. No offense to the person who did it but you can do some pretty nifty things with the python C-api with some practice.
- mingodad 4y ago
  To add to your list I just found this project https://github.com/ChrisHixon/chpeg https://github.com/ChrisHixon/chpeg it has good performance and it's relatively simple compared to others.
- eesmith 4y ago
  The Earley parser was written by John Aycock. The flex/bison clone you're perhaps thinking of is PLY from David Beazley.
- mingodad 4y ago
  Also to add to your list is https://github.com/KuramitsuLab/pegpy https://github.com/KuramitsuLab/pegpy it's an intriguing peg parser with origin from a Japanese group https://github.com/nez-peg/nez https://github.com/nez-peg/nez .
jdowner 4y ago
Parsimonious is quite nice.
carapace 4y ago
I use SPARK (Scanning, Parsing, and Rewriting Kit) it makes Earley parsers. > The original version of this was written by John Aycock for his Ph.d thesis and was described in his 1998 paper: “Compiling Little Languages in Python” at the 7th International Python http://pages.cpsc.ucalgary.ca/~aycock/spark/ http://pages.cpsc.ucalgary.ca/~aycock/spark/ (Older) https://pypi.org/project/spark-parser/ https://pypi.org/project/spark-parser/ (More recent)

frou_dh 4y ago

When I surveyed what was out there last year I wrote down these 3 as the most interesting: https://github.com/lark-parser/lark https://github.com/lark-parser/lark https://github.com/neogeny/TatSu https://github.com/neogeny/TatSu https://github.com/dabeaz/sly https://github.com/dabeaz/sly Lark seems to be the one with traction.

ptmcg 4y ago
The 3.0 release of pyparsing includes some nice extensions over its older 2.x code base, including LR parsing support and railroad diagram generation. Also, PEP-8 API naming has been added (but with pre-PEP-8 synonyms defined for legacy compatibility). https://github.com/pyparsing/pyparsing https://github.com/pyparsing/pyparsing (Disclosure: I am the author of pyparsing.)

UncleEntity 4y ago

I played around with the Earley parser that used to be used as part of the CPython build process (who’s name I can’t remember off the top of my head) a bunch and with a lot of hacking I got it into a pretty usable state. It isn’t really that easy to hack on because they do a lot of optimization which is probably one of the major reasons they kicked it out of the python repo. ANTLR does have python as one of the output languages IIRC. There’s also a flex/bison clone written in python whose name I also can’t remember. My current favorite parser generator library, Coco/R, has a python 2 implementation which I’ll maybe get around to porting to py3k one of these days, who knows? But to answer your question I don’t think there’s a go to library on the python side, I’ve played around with a bunch of them and usually just go for re2c/lemon or Coco/R from C(++) since I mainly just poke at these things probably more than I should. The next time I get a few days with nothing to do I plan to see if I can’t whip up a working lpeg (lua’s peg parser VM library thing) in python. Someone did a port a while ago but it is abandoned and, honestly, not all that good. No offense to the person who did it but you can do some pretty nifty things with the python C-api with some practice.

mingodad 4y ago
To add to your list I just found this project https://github.com/ChrisHixon/chpeg https://github.com/ChrisHixon/chpeg it has good performance and it's relatively simple compared to others.
eesmith 4y ago
The Earley parser was written by John Aycock. The flex/bison clone you're perhaps thinking of is PLY from David Beazley.
mingodad 4y ago
Also to add to your list is https://github.com/KuramitsuLab/pegpy https://github.com/KuramitsuLab/pegpy it's an intriguing peg parser with origin from a Japanese group https://github.com/nez-peg/nez https://github.com/nez-peg/nez .

jdowner 4y ago

Parsimonious is quite nice.

carapace 4y ago

I use SPARK (Scanning, Parsing, and Rewriting Kit) it makes Earley parsers. > The original version of this was written by John Aycock for his Ph.d thesis and was described in his 1998 paper: “Compiling Little Languages in Python” at the 7th International Python http://pages.cpsc.ucalgary.ca/~aycock/spark/ http://pages.cpsc.ucalgary.ca/~aycock/spark/ (Older) https://pypi.org/project/spark-parser/ https://pypi.org/project/spark-parser/ (More recent)