2 ms·
This is great! I also found I was able to become productive with tree-sitter very quickly, even for tasks it’s not directly designed for. I wrote a (naive) type
by eyelidlessness 5y ago
This is great! I also found I was able to become productive with tree-sitter very quickly, even for tasks it’s not directly designed for. I wrote a (naive) type-stripping TypeScript “compiler” over a weekend using tree-sitter to identify the type annotation nodes to strip. Day 1 it performed on par with ESBuild (which admittedly is doing a lot more) for small modules. Day 2 it was close to ESBuild for a real world 10k LOC module.
This was all a spike to see how I could leverage tree-sitter for real use cases (I really do not want to build yet another TypeScript compiler, I just know the problem space well enough that it was a good starting point).
In the past week I’ve been writing a grammar for XPath. I have a bunch of compelling work related needs which could benefit from that, and maybe surprisingly no grammar for XPath already exists. It’s mostly going smoothly for a lot of my test cases (damn do I have a lot of test cases to reuse, gratefully), but I will say this has been the most challenging part of using tree-sitter so far. Not enough to deter, but worth mentioning if anyone is approaching the library from a similar perspective.
I’m mostly translating a bunch of BNF-ish grammar definitions into tree-sitter’s grammar definition DSL. It’s a good programmatic API for this, but conflict resolution is surprisingly hard to get an instinct for. And the s-expression tree representation + runtime API doesn’t give much insight into nodes which just appear as `(ERROR …stuff)` with no access to details about what caused the error (which doesn’t throw, it just silently builds an invalid tree).
None of this is meant to dissuade anyone from using tree-sitter! It’s an incredible library with a zillion applicable use cases if you’re looking for something with its capabilities. Just cautioning that developing a grammar has been one of the rough edges for me.
- siraben 5y agoWriting grammars presents an interesting set of challenges, especially dealing with ambiguity and precedence as you mention. I find that the `tree-sitter playground` command is convenient for interactively trying out different possibilities. There's also a debug flag "-D" for `tree-sitter parse` which gives detailed information about the shift/reduce points in the parser state and different branches it parsed simultaneously (because it's a GLR parser).
- eyelidlessness 5y agoThanks! I found out about the debug flag yesterday, but I was oh my phone after I called it quits so I haven’t tried it yet. Gonna give it a try today.