4 ms·

The auto-generated AST data structures are really the selling point here. Having to manually write an AST for your language and update it if you update your lan

The auto-generated AST data structures are really the selling point here. Having to manually write an AST for your language and update it if you update your language is mechanical and tedious. I suggest the developer also look into auto-generated visitor / traversal functions as well, as those are also very mechanical and tedious. Another suggestion is to make sure the grammar preserves source information (file, line, column span) and implement re-printing which preserves comments and whitespace in unmodified sections of the AST. I get those are a lot of work though, as-is this is very impressive.

mhh__ 4y ago
Auto generating stuff also forces you to not bolt tons of shit into all the last nodes
jzimmerman64 4y ago
> I suggest the developer also look into auto-generated visitor / traversal functions as well In fact, langcc already has visitor/traversal functions, though they are not so well documented yet. I have just added an example (expr_simpl in https://github.com/jzimmerman/langcc/blob/main/examples/calc/calc_main.cpp https://github.com/jzimmerman/langcc/blob/main/examples/calc...). > Another suggestion is to make sure the grammar preserves source information (file, line, column span) This is already present as well (the calc example illustrates this functionality). > implement re-printing which preserves comments and whitespace I considered doing this, but it's surprisingly difficult to get the specification right, especially in whitespace-sensitive languages like Python.
layer8 4y ago
I don’t find this to be very impressive. I used SableCC [0] 15 years ago, which generates Java source code (classes) for the AST along with the parser, including Visitor traversal. It was missing line number information and character data between tokens, but that was straightforward to add by modifying the built-in templates used to generate the AST classes. I was also able to add generically-typed return values to the Visitor mechanism just by editing the built-in templates. [0] https://sablecc.org/ https://sablecc.org/
HarHarVeryFunny 4y ago
> The auto-generated AST data structures are really the selling point here But sometimes you want something custom ... A fun project I did a while back was implementing a C subset (no structs, pointers or goto) using a directly-executable AST! Kind of a hybrid approach between a compiler and interpreter. This was built as a C++ library to load and execute C-like macros (including a domain-specific library of built-in functions). What I did (using C++) was have two base classes, Statement and Expression, with respective virtual functions Execute() and Value(). Each type of C statement and expression then derived a corresponding "AST" class from these. e.g. class IfStatement derived from Statement and implemented the Execute() method. The IfStatement constructor (following the grammar) took an Expression and two Statement base-class pointers as arguments which it stored. The implementation of IfStatement::Execute() was then just: if (expression->Value() if_statement->Execute(); else else_statement->Execute(); Similar for ForStatement, etc, etc. The root of the AST was then a CompoundStatement representing the entire program as a list of Statements, and you would then execute the program just by calling the Execute() method on this statement. In this case I used a recursive descent parser (part of the library I built) to build this "AST", but of course you could do the same for a table driven parser, and it would make a nice demo of how little code you need to write to not only parse but also execute/evaluate whatever you are parsing.

mhh__ 4y ago

Auto generating stuff also forces you to not bolt tons of shit into all the last nodes

jzimmerman64 4y ago

> I suggest the developer also look into auto-generated visitor / traversal functions as well In fact, langcc already has visitor/traversal functions, though they are not so well documented yet. I have just added an example (expr_simpl in https://github.com/jzimmerman/langcc/blob/main/examples/calc/calc_main.cpp https://github.com/jzimmerman/langcc/blob/main/examples/calc...). > Another suggestion is to make sure the grammar preserves source information (file, line, column span) This is already present as well (the calc example illustrates this functionality). > implement re-printing which preserves comments and whitespace I considered doing this, but it's surprisingly difficult to get the specification right, especially in whitespace-sensitive languages like Python.

layer8 4y ago

I don’t find this to be very impressive. I used SableCC [0] 15 years ago, which generates Java source code (classes) for the AST along with the parser, including Visitor traversal. It was missing line number information and character data between tokens, but that was straightforward to add by modifying the built-in templates used to generate the AST classes. I was also able to add generically-typed return values to the Visitor mechanism just by editing the built-in templates. [0] https://sablecc.org/ https://sablecc.org/

HarHarVeryFunny 4y ago

> The auto-generated AST data structures are really the selling point here But sometimes you want something custom ... A fun project I did a while back was implementing a C subset (no structs, pointers or goto) using a directly-executable AST! Kind of a hybrid approach between a compiler and interpreter. This was built as a C++ library to load and execute C-like macros (including a domain-specific library of built-in functions). What I did (using C++) was have two base classes, Statement and Expression, with respective virtual functions Execute() and Value(). Each type of C statement and expression then derived a corresponding "AST" class from these. e.g. class IfStatement derived from Statement and implemented the Execute() method. The IfStatement constructor (following the grammar) took an Expression and two Statement base-class pointers as arguments which it stored. The implementation of IfStatement::Execute() was then just: if (expression->Value() if_statement->Execute(); else else_statement->Execute(); Similar for ForStatement, etc, etc. The root of the AST was then a CompoundStatement representing the entire program as a list of Statements, and you would then execute the program just by calling the Execute() method on this statement. In this case I used a recursive descent parser (part of the library I built) to build this "AST", but of course you could do the same for a table driven parser, and it would make a nice demo of how little code you need to write to not only parse but also execute/evaluate whatever you are parsing.