4 ms·

One idea for speedups that this article doesn't mention, that I've used very successfully, is splitting an extension into two parts: The Javascript extension th

One idea for speedups that this article doesn't mention, that I've used very successfully, is splitting an extension into two parts: The Javascript extension that runs in VS Code, and a separate language server[1] written in whatever fast language you desire (without having to compile to WASM). I do this with an extension I develop, and it works very well. Microsoft even provides a library to make this easy, and sample code[2]. In my case the language server is written in Rust, and uses tree-sitter. This combination feels like a super power. (The first version of my extension was written in clojurescript, using the instaparse parser. It quickly became apparent that it was way too slow.) A note on my experience with tree-sitter: It's awesome in every way except one. It's so fast that so far I haven't even bothered with incremental parsing: I can do full parsing on every keystroke; and I know that if I ever need a speed boost, I can add incremental parsing. The API is sane and easy to use. The query API is powerful. But the main weakness of tree-sitter is that the error messages are nearly content-free. The most common error looks like this: "(error)". In my case I can deal with that, but I can imagine that for many purposes that's not sufficient. There's been an issue open for 3 years about improving the error messages[3] but I haven't seen a ton of progress (unless I missed it?). 1. https://microsoft.github.io/language-server-protocol/specification https://microsoft.github.io/language-server-protocol/specifi... 2. https://github.com/microsoft/vscode-extension-samples/tree/main/lsp-sample https://github.com/microsoft/vscode-extension-samples/tree/m... 3. https://github.com/tree-sitter/tree-sitter/issues/255 https://github.com/tree-sitter/tree-sitter/issues/255

peakaboo 5y ago
I use treesitter from neovim so it's pretty crazy fast. :)
capableweb 5y ago
It seems to be mentioned, unless I misunderstand what you mean. The second-last section in the article begins with this: > If you really need to do some CPU intensive work, it’s now possible to offload some of the workload to a language server. This allows you to implement the bulk of your extension in another language (for instance, writing Rust code and compiling it down to WASM). The section above also mentions tree-sitter.
- jjwiseman 5y ago
  I think you're right. I got confused because he talks about compiling to WASM; I'm not knowledgeable about WASM but a language server is an entirely separate process, which means you can compile your Rust to native x86 or whatever you want. I assume Rust compiled to native machine code is faster than WASM? I guess the one advantage of WASM in that case is that you don't have to compile the language server for every platform the extension runs on. (In my case, I compile the language server for x86 Linux and Windows.)
patrec 5y ago
> A note on my experience with tree-sitter: It's awesome in every way except one. One more: the Rube-Goldberg build setup. You need rust, npm, node and a C compiler as well as Docker or Emscripten. Tree-sitter is mostly written in Rust and there are Rust bindings, but you can't use them to for the wasm compilation target because it requires linking a C generated grammar which breaks things. I hope they'll move on to something that's possible to build and use with pure Rust.
jjwiseman 5y ago
I forgot to mention one other thing about tree-sitter: the error recovery/tolerance, which is a requirement for practical parsing applications. It does an excellent job of returning a parse tree even if the text contains tons of errors. Instaparse, besides being much slower, just gave up if a single character was out of place. Super rarely there will be an error that causes tree-sitter to fail more comprehensively, but it is one of the few parser frameworks that even tries.

peakaboo 5y ago

I use treesitter from neovim so it's pretty crazy fast. :)

capableweb 5y ago

It seems to be mentioned, unless I misunderstand what you mean. The second-last section in the article begins with this: > If you really need to do some CPU intensive work, it’s now possible to offload some of the workload to a language server. This allows you to implement the bulk of your extension in another language (for instance, writing Rust code and compiling it down to WASM). The section above also mentions tree-sitter.

jjwiseman 5y ago
I think you're right. I got confused because he talks about compiling to WASM; I'm not knowledgeable about WASM but a language server is an entirely separate process, which means you can compile your Rust to native x86 or whatever you want. I assume Rust compiled to native machine code is faster than WASM? I guess the one advantage of WASM in that case is that you don't have to compile the language server for every platform the extension runs on. (In my case, I compile the language server for x86 Linux and Windows.)

patrec 5y ago

> A note on my experience with tree-sitter: It's awesome in every way except one. One more: the Rube-Goldberg build setup. You need rust, npm, node and a C compiler as well as Docker or Emscripten. Tree-sitter is mostly written in Rust and there are Rust bindings, but you can't use them to for the wasm compilation target because it requires linking a C generated grammar which breaks things. I hope they'll move on to something that's possible to build and use with pure Rust.

jjwiseman 5y ago

I forgot to mention one other thing about tree-sitter: the error recovery/tolerance, which is a requirement for practical parsing applications. It does an excellent job of returning a parse tree even if the text contains tons of errors. Instaparse, besides being much slower, just gave up if a single character was out of place. Super rarely there will be an error that causes tree-sitter to fail more comprehensively, but it is one of the few parser frameworks that even tries.