2 ms·
Are there any parsers (or parser generators) that allow to generate a "Lossless Syntax Tree" for an arbitrary grammar? Specifically, a parser that tags every pa
by goerz 10y ago
Are there any parsers (or parser generators) that allow to generate a "Lossless Syntax Tree" for an arbitrary grammar? Specifically, a parser that tags every parsed token with a tuple (line_id, col, length) that tells me where it was in the original parsed file. I've been thinking for ages about writing some refactoring tools for Fortran that would need something like that.
- nly 10y agoPEGTL for C++ provides such a tuple to its semantic actions. It'd be up to you of course to store that information in to your parse tree.
- maxbrunsfeld 10y agoThis is a parser generator that I'm working on: https://github.com/tree-sitter/tree-sitter https://github.com/tree-sitter/tree-sitter It produces concrete syntax trees that can be queried by line or character index. The library is specifically focused on incremental parsing, for use in a text editor, but it also works fine for normal 'batch' parsing workloads. It has a simple C API that you should be able to use from any language. Currently, there are bindings to JavaScript and Haskell. Here are some example grammars: * C - https://github.com/tree-sitter/tree-sitter-c https://github.com/tree-sitter/tree-sitter-c * JavaScript - https://github.com/tree-sitter/tree-sitter-javascript https://github.com/tree-sitter/tree-sitter-javascript * Go - https://github.com/tree-sitter/tree-sitter-go https://github.com/tree-sitter/tree-sitter-go * Ruby - https://github.com/tree-sitter/tree-sitter-ruby https://github.com/tree-sitter/tree-sitter-ruby * Python - https://github.com/tree-sitter/tree-sitter-python https://github.com/tree-sitter/tree-sitter-python
- goerz 10y agoThat looks extremely interesting!