3 ms·
There is something deeply attractive about writing an ast-free single pass compiler. It's a great exercise to write a compiler that targets a scripting language
by norir 3y ago
There is something deeply attractive about writing an ast-free single pass compiler. It's a great exercise to write a compiler that targets a scripting language like lua or js in a single pass. I think though that there are significant advantages to having an ast and multiple compilation passes, particularly on present day computers that have effectively unlimited memory for the purpose of compiling a single source file.
Multiple passes allows you to break down the problem into separate orthogonal components. To support the same functionality in a single pass, you have to jam an enormous amount of functionality into one place which can make it very difficult to see the forest of the trees as well as introduce unintended coupling. It also is difficult to implement certain recursive features in a single pass (though this could be considered an advantage depending on how powerful you intend the language to be).
Ideally, a compiler can also provide apis that will expose the ast at different phases. It is somewhat sad to me that tree-sitter has emerged as an LCD solution for syntax highlighting and semantic analysis. If a compiler could export e.g. a json representation of the ast after parsing, that could be used for syntax highlighting rather than tree-sitter.
- disconcision 3y agotree sitter seems to do pretty well in its limited role; why is this unfortunate in your opinion?
- NateEag 3y agoNot OP, but I think their point was clear - they seem to wish that compilers offered their own ASTs / tools for working with ASTs, so that tools like tree-sitter wouldn't exist at all. It makes sense - tree-sitter is in large part a reimplementation of many, many parsers, and supporting a new language means duplicating yet another parser. The tree-sitter grammars are doomed by nature to forever chase the latest features in any evolving language, and reaching full compatibility turns out to be a ton of effort in many cases. That said, language design being the freewheeling "if you can dream it, you can make it" thing it is, though, I doubt it's realistic to expect every language to have a fully-reusable, standardized parser - people just won't ever be that consistent, IMO. Those who want consistency will need to provide it themselves, in the form of tools like tree-sitter.
- funcDropShadow 3y ago> Those who want consistency will need to provide it themselves, in the form of tools like tree-sitter. Or in the form of radical simplicity. For the purpose of the parents discussion Lisp is the ultimate minimal language. Since, the syntax is its own AST. Other aspects of most Lisp are certainly not so simple.