3 ms·
Tree-Sitter
- Sphax 3y agoI use neovim, the tree-sitter support is great especially with nvim-treesitter which means I can just run `TSInstall <parser name>` and I'm done. Writing a tree-sitter parser is not that easy though. I wrote tree-sitter-templ to parse templ files and I still have a hard time identifying some parsing errors, especially since I had to implement a custom external scanner which in my experience was the source of a lot of bugs. [0]: https://github.com/vrischmann/tree-sitter-templ https://github.com/vrischmann/tree-sitter-templ
- hiAndrewQuinn 3y agoIt's not easy, but after a while it does become somewhat meditative. I think the TypeScript nature of it helps a lot. I picked up Neovim at my last job and got a decent start on a Tree-sitter parser off hours for the WinSCP batch programming language of all things. But then I couldn't get it to actually install correctly into Neovim, and I switched jobs before I figured it out. :( EDIT: actually, hey, your instructions on installing are really clear! I wonder if I can use them here to help guide me again. It was really cool, though! Love parsers. https://github.com/hiAndrewQuinn/tree-sitter-winscp https://github.com/hiAndrewQuinn/tree-sitter-winscp
- sergiogdr 3y agoHow does TS compare to using semantic highlighting in LSP?
- semiquaver 3y agoA large number of LSP servers and related software use tree-sitter as a backend.
- ReleaseCandidat 3y agoThat's not possible, as Treesitter doesn't do for example type-checking (unification). It can be used as the parser to generate the AST with which you're actually doing the work.
- spinningslate 3y agoAgree, though not sure that's what the parent meant. I think they meant: 1. Some LSP implementations use Tree Sitter as the parser in their implementation 2. But it's only part of the implementation. It just generates the parse tree, on which various other LSP features are built, e.g. enhanced syntax highlighting, go to definition, ... Tree Sitter is pretty good for (1) because it was built specifically to cope with code that is (a) changing rapidly and (b) is, by default, not valid. So it has good error reporting and recovery.
- cybrexalpha 3y agoHaving tried both with Neovim, I ended up going with just LSPs instead of using tree-sitter.
- ReleaseCandidat 3y agoWith Treesitter you have another parser which is redundant at best and inconsistent with the LSP at worst.
- pedrovhb 3y agoNot sure I understand your point. LSP is a protocol and tree-sitter is a parser generator. They're kind of orthogonal concepts; a tree-sitter parser couldn't ever be used directly in place of an LSP server, but an LSP server may well make use of tree-sitter as a first step for extracting information from the code and keeping it in sync. If it doesn't it'll have to come up with some other way of parsing the code in any case, so I don't see how it could be said to be redundant or inconsistent. Of course, tree-sitter's thing is how universal it is. There's parsers for tons of languages, and you can work with them all using the same API, though you're on your own for attributing semantic meaning. Most popular languages have language-specific tools (e.g. `libcst`) which are usually more powerful for that specific language, so they'd probably be better starting points for building a language-specific LSP server which I imagine is the common case.
- ReleaseCandidat 3y ago> Not sure I understand your point. The problem is using Treesitter (for syntax highlighting and "semantic movements") and an LSP at the same time. So if your language has a LSP, using Treesitter additionally is redundant at best and introduces inconcistency at worst. I'm not talking about using Treesitter as the parser for the LSP. > Most popular languages have language-specific tools I'd say even less popular langauges like Coq^H^H^HRocq, Lean 4, Koka, Idris, Unison, ... have their "own" tools, I do not know of a language that uses a Treesitter parser in its LSP, but I do know about tools like https://semgrep.dev/ https://semgrep.dev/ (written in OCaml) and Github's code search which use Treesitter.
- 3836293648 3y agoYou're forgetting that treesitter is much, much faster than LSPs.
- thworp 3y agoIt's slightly worse but a lot faster and better at dealing with spelling/syntax errors. Also, the LSP can render inline error messages and line markers on top of TS highlighting so not much information is lost by just using TS highlighting everywhere.
- nilsherzig 3y agoNeovim supports treesitter based selections and "jumps", which I think are awesome https://www.reddit.com/r/neovim/comments/r10llx/the_most_amazing_builtin_feature_nobody_ever/ https://www.reddit.com/r/neovim/comments/r10llx/the_most_ama... Or https://github.com/RRethy/nvim-treesitter-textsubjects https://github.com/RRethy/nvim-treesitter-textsubjects Helix also supports them by default
- vsnf 3y agoI've tried to write a parser using tree-sitter multiples times, but it's complicated enough that I've found it to be simpler to write my own parsers. Of course as a mere amateur, my parsers don't have certain nice to have features like good error messages, or good incomplete syntax recovery. I wish tree-sitter was easier to use.
- a1o 3y agoI didn't have much trouble writing tree-sitter parsers after taking a look to examples but I also resigned to writing my own parsers instead of using it. There were two concerns I couldn't shake from using it. 1. In C-like languages you usually can do forward declarations, so you can opaquely define something, use it and have it properly defined later, usually due to how things need to use other things in their declaration. You can't cleanly do this in tree-sitter, you will need to handwrite some C code for this on top. 2. Even though the resulting parser is a single C-file, it still depends on some tree-sitter library and building it requires nodejs and npm packages. This means that anyone updating my C/C++ codebase has to also understand JS tooling. I would rather not have such dependency. If tree-sitter could have standalone tools that could be built only using either a C or C++ compiler, I would not have such a problem.
- bfrog 3y agoI tried to write a vhdl treesitter grammar and I too found it hard. I got something working enough for decent highlights but not a complete grammar by any means.
- dang 3y agoRelated. Others? How to Get Started with Tree-Sitter - https://news.ycombinator.com/item?id=36106421 https://news.ycombinator.com/item?id=36106421 - May 2023 (67 comments) Topiary: A code formatting engine leveraging Tree-sitter - https://news.ycombinator.com/item?id=36008139 https://news.ycombinator.com/item?id=36008139 - May 2023 (33 comments) Rust-sitter: Define your entire tree-sitter grammar in Rust code - https://news.ycombinator.com/item?id=34892916 https://news.ycombinator.com/item?id=34892916 - Feb 2023 (14 comments) Combobulate: Structured Movement and Editing with Tree-Sitter - https://news.ycombinator.com/item?id=34601089 https://news.ycombinator.com/item?id=34601089 - Jan 2023 (15 comments) Writing a TLA⁺ tree-sitter grammar: my foray into free software - https://news.ycombinator.com/item?id=34356429 https://news.ycombinator.com/item?id=34356429 - Jan 2023 (3 comments) Emacs: Feature/tree-sitter merged into master - https://news.ycombinator.com/item?id=33715321 https://news.ycombinator.com/item?id=33715321 - Nov 2022 (89 comments) Tree-sitter grammar for org-mode - https://news.ycombinator.com/item?id=30945926 https://news.ycombinator.com/item?id=30945926 - April 2022 (31 comments) How to write a linter using tree-sitter in an hour - https://news.ycombinator.com/item?id=30822544 https://news.ycombinator.com/item?id=30822544 - March 2022 (11 comments) How to write a tree-sitter grammar in an afternoon - https://news.ycombinator.com/item?id=30661127 https://news.ycombinator.com/item?id=30661127 - March 2022 (9 comments) Tree Sitter and the Complications of Parsing Languages - https://news.ycombinator.com/item?id=29327424 https://news.ycombinator.com/item?id=29327424 - Nov 2021 (134 comments) Diffsitter: A tree-sitter based AST difftool to get meaningful semantic diffs - https://news.ycombinator.com/item?id=27875333 https://news.ycombinator.com/item?id=27875333 - July 2021 (53 comments) Tree-sitter: an incremental parsing system for programming tools - https://news.ycombinator.com/item?id=26225298 https://news.ycombinator.com/item?id=26225298 - Feb 2021 (134 comments) Tree-sitter: new incremental parsing system for programming tools (2018) [video] - https://news.ycombinator.com/item?id=21675113 https://news.ycombinator.com/item?id=21675113 - Dec 2019 (28 comments) Tree-sitter – a new parsing system for programming tools [video] - https://news.ycombinator.com/item?id=18213022 https://news.ycombinator.com/item?id=18213022 - Oct 2018 (25 comments)