5 ms·

It's a shame parsers are such a PITA to write. So many problems could be trivially solved if writing a grammar and generating a parser for it were in any way a

It's a shame parsers are such a PITA to write. So many problems could be trivially solved if writing a grammar and generating a parser for it were in any way a pleasant process.

justinpombrio 10y ago
This is the pain of text. The text format has won so handily that we rarely consider the possibility that files could be anything other than a sequence of characters. But if it were a tree instead, then "parsing" would be a simple, linear-time process with good error messages.
- megous 10y ago
  Editing would be annoying as hell though.
  - justinpombrio 10y ago
    You would use a different editor: a tree or structured editor. It would be different, but I doubt it would be worse.
    - megous 10y ago
      So, where are all the professional programmers doing blocks programming, or something like that? All visual programming is very niche. Last I got close to some was 18 years ago when working with LabView. It's not just different. It's suboptimal. It's just not compelling. You'd need visual everything. Like visual diff, visual git, make it easy to copy visual crap between various applications. Copy visual code to a web page? Copy it to chat to send to a friend? Instead of just writing, you'd need to remember UI shortcuts/icon to insert visual crap into your codebase... It's bleh. I clearly hate the idea. :D
  - e12e 10y ago
    Yeah, that's why no-one uses Maya to edit 3d scenes. It lacks the finesse of text, like povray ;-)
    - munificent 10y ago
      The difference there is that the thing you are authoring is itself visual. You can't be a skilled Maya user if you're not a good visual thinker/designer because that's the core competency of 3D design itself. Code is not inherently visual. If you made the editor visual, then it means people who are good at abstract reasoning and solving problems but don't have good aesthetic and spatial sense wouldn't be able to "write good code".
      - justinpombrio 10y ago
        > Code is not inherently visual. Nor is it text. It is a tree (hence the term "abstract syntax tree"). We just save it in an awful serialized format.
      - e12e 10y ago
        > The difference there is that the thing you are authoring is itself visual. Perhaps. Or it might be structural, an object to be 3d printed. It would have a physical appearance - but perhaps not be "inherently visual", any more than the algorithms for adjusting a quadcopter for autonomous hovering is "inherently aerial"? > people who are good at abstract reasoning and solving problems but don't have good aesthetic and spatial sense wouldn't be able to "write good code". Isn't that the case with textual representation of code too? Without a sense for aesthetics, one cannot create "beautiful code"? A chessboard/game can be represented in many ways in the form of code/syntax - even mathematical equations - and yet I would argue it is less accessible to learn and play chess going from just the standard textual representation, than from a board? How would you teach chess moves: by lifting a castle off the board, showing that it can move up-and-down and sideways, like so - or by saying that the piece-instances initialized to position a1,a8,h1 and h8 can move only be changing either of its two coordinates up and down?
avmich 10y ago
You can probably use parser generators for this. In this way it's possible to have a function accepting a grammar and an input and returning the parse tree, so all complications are hidden inside.
UK-AL 10y ago
There are tons of parser generators, but I find the code they generate unreadable and terrible for error reporting.
deleted 10y ago
[deleted]
melling 10y ago
There's a lot of information on compilers, parsers, etc on the Internet. I'm trying to gather some of the information h=on github: https://github.com/melling/ComputerLanguages/blob/master/compilers.org https://github.com/melling/ComputerLanguages/blob/master/com...
JustSomeNobody 10y ago
Recursive descent is pleasant. It's probably the least brutish and most elegant of methods.[0] [0] Of course, my opinion.
- grabcocque 10y ago
  I've used ANTLR, which is relatively easy to comprehend by being primarily a LL(k) recursive descent parser. Sadly, it also suffers from the usual problems of LL parser generators, like barfing on left-recursive grammars.
marcosdumay 10y ago
Try learning Prolog. Or, if you care about efficiency, Haskell.
girvo 10y ago
I've found Parsing Expression Grammars (PEGs) great for that sort of task. Way nicer than kludging something with regexes anyway, and way more powerful.
- thegeomaster 10y ago
  There's a Lua library called LPeg [1], which introduces a DSL of a kind to Lua that lets you write PEGs with minimal effort. The vis editor [2] uses LPeg files as lexers for syntax highlighting, and they look very clean IMO; for example, this is the C lexer: https://github.com/martanne/vis/blob/master/lua/lexers/ansi_c.lua https://github.com/martanne/vis/blob/master/lua/lexers/ansi_... [1]: http://www.inf.puc-rio.br/~roberto/lpeg/ http://www.inf.puc-rio.br/~roberto/lpeg/ [2]: https://github.com/martanne/vis https://github.com/martanne/vis
AgentME 10y ago
I've found parsers simple to make with a parser combinator library, like Bennu[0]. It matches how I thought parsing should work, and as a bonus the parsers made with it automatically support incremental parsing. Just for fun, I implemented a full JSON parser[1] with it. I assume a hand-written a recursive descent parser would run faster, but it definitely doesn't strike me as flexible or quick to write or modify, and anything extra like incremental support or good error messages is just a lot of one-off manual effort. [0] http://bennu-js.com/ http://bennu-js.com/ [1] https://github.com/AgentME/bennu-json https://github.com/AgentME/bennu-json
e12e 10y ago
Well, there's always the "Ometa" approach/extension to PEGs - the author works(?) at now y-combinator sponsored research into language design (not sure if he's active on hn though): http://www.tinlizzie.org/ometa-js/#Sample_Project http://www.tinlizzie.org/ometa-js/#Sample_Project Note, you need to select all before trying "do it"/"print it". See eg: the js compiler (top right, second drop down). I dare you to not find it so easy and fun that you don't try to implement a small DSL ;)
- asrp 10y ago
  There's the Ohm editor now. https://ohmlang.github.io/editor/ https://ohmlang.github.io/editor/ Ohm is the successor of Ometa. The editor shows which portion of the string are matched by each rule.

It's a shame parsers are such a PITA to write. So many problems could be trivially solved if writing a grammar and generating a parser for it were in any way a pleasant process.

justinpombrio 10y ago

This is the pain of text. The text format has won so handily that we rarely consider the possibility that files could be anything other than a sequence of characters. But if it were a tree instead, then "parsing" would be a simple, linear-time process with good error messages.

megous 10y ago
Editing would be annoying as hell though.
- justinpombrio 10y ago
  You would use a different editor: a tree or structured editor. It would be different, but I doubt it would be worse.
  - megous 10y ago
    So, where are all the professional programmers doing blocks programming, or something like that? All visual programming is very niche. Last I got close to some was 18 years ago when working with LabView. It's not just different. It's suboptimal. It's just not compelling. You'd need visual everything. Like visual diff, visual git, make it easy to copy visual crap between various applications. Copy visual code to a web page? Copy it to chat to send to a friend? Instead of just writing, you'd need to remember UI shortcuts/icon to insert visual crap into your codebase... It's bleh. I clearly hate the idea. :D
- e12e 10y ago
  Yeah, that's why no-one uses Maya to edit 3d scenes. It lacks the finesse of text, like povray ;-)
  - munificent 10y ago
    The difference there is that the thing you are authoring is itself visual. You can't be a skilled Maya user if you're not a good visual thinker/designer because that's the core competency of 3D design itself. Code is not inherently visual. If you made the editor visual, then it means people who are good at abstract reasoning and solving problems but don't have good aesthetic and spatial sense wouldn't be able to "write good code".
    - justinpombrio 10y ago
      > Code is not inherently visual. Nor is it text. It is a tree (hence the term "abstract syntax tree"). We just save it in an awful serialized format.
    - e12e 10y ago
      > The difference there is that the thing you are authoring is itself visual. Perhaps. Or it might be structural, an object to be 3d printed. It would have a physical appearance - but perhaps not be "inherently visual", any more than the algorithms for adjusting a quadcopter for autonomous hovering is "inherently aerial"? > people who are good at abstract reasoning and solving problems but don't have good aesthetic and spatial sense wouldn't be able to "write good code". Isn't that the case with textual representation of code too? Without a sense for aesthetics, one cannot create "beautiful code"? A chessboard/game can be represented in many ways in the form of code/syntax - even mathematical equations - and yet I would argue it is less accessible to learn and play chess going from just the standard textual representation, than from a board? How would you teach chess moves: by lifting a castle off the board, showing that it can move up-and-down and sideways, like so - or by saying that the piece-instances initialized to position a1,a8,h1 and h8 can move only be changing either of its two coordinates up and down?

avmich 10y ago

You can probably use parser generators for this. In this way it's possible to have a function accepting a grammar and an input and returning the parse tree, so all complications are hidden inside.

UK-AL 10y ago

There are tons of parser generators, but I find the code they generate unreadable and terrible for error reporting.

deleted 10y ago

[deleted]

melling 10y ago

There's a lot of information on compilers, parsers, etc on the Internet. I'm trying to gather some of the information h=on github: https://github.com/melling/ComputerLanguages/blob/master/compilers.org https://github.com/melling/ComputerLanguages/blob/master/com...

JustSomeNobody 10y ago

Recursive descent is pleasant. It's probably the least brutish and most elegant of methods.[0] [0] Of course, my opinion.

grabcocque 10y ago
I've used ANTLR, which is relatively easy to comprehend by being primarily a LL(k) recursive descent parser. Sadly, it also suffers from the usual problems of LL parser generators, like barfing on left-recursive grammars.

marcosdumay 10y ago

Try learning Prolog. Or, if you care about efficiency, Haskell.

girvo 10y ago

I've found Parsing Expression Grammars (PEGs) great for that sort of task. Way nicer than kludging something with regexes anyway, and way more powerful.

thegeomaster 10y ago
There's a Lua library called LPeg [1], which introduces a DSL of a kind to Lua that lets you write PEGs with minimal effort. The vis editor [2] uses LPeg files as lexers for syntax highlighting, and they look very clean IMO; for example, this is the C lexer: https://github.com/martanne/vis/blob/master/lua/lexers/ansi_c.lua https://github.com/martanne/vis/blob/master/lua/lexers/ansi_... [1]: http://www.inf.puc-rio.br/~roberto/lpeg/ http://www.inf.puc-rio.br/~roberto/lpeg/ [2]: https://github.com/martanne/vis https://github.com/martanne/vis

AgentME 10y ago

I've found parsers simple to make with a parser combinator library, like Bennu[0]. It matches how I thought parsing should work, and as a bonus the parsers made with it automatically support incremental parsing. Just for fun, I implemented a full JSON parser[1] with it. I assume a hand-written a recursive descent parser would run faster, but it definitely doesn't strike me as flexible or quick to write or modify, and anything extra like incremental support or good error messages is just a lot of one-off manual effort. [0] http://bennu-js.com/ http://bennu-js.com/ [1] https://github.com/AgentME/bennu-json https://github.com/AgentME/bennu-json

e12e 10y ago

Well, there's always the "Ometa" approach/extension to PEGs - the author works(?) at now y-combinator sponsored research into language design (not sure if he's active on hn though): http://www.tinlizzie.org/ometa-js/#Sample_Project http://www.tinlizzie.org/ometa-js/#Sample_Project Note, you need to select all before trying "do it"/"print it". See eg: the js compiler (top right, second drop down). I dare you to not find it so easy and fun that you don't try to implement a small DSL ;)

asrp 10y ago
There's the Ohm editor now. https://ohmlang.github.io/editor/ https://ohmlang.github.io/editor/ Ohm is the successor of Ometa. The editor shows which portion of the string are matched by each rule.