7 ms·

Rust Analyzer in 2018 and 2019

181 points by jamesmunns 8y ago

jamesmunns 8y ago
Hey! This is a post by matklad [1], one of the members of the Ferrous Systems [2] team. The Rust Analyzer project [3] is an experimental compiler front-end, designed to power things like code completion, syntax error reporting, and other things that are necessary when building IDE support. Let us know if you have any questions! [1]: https://github.com/matklad https://github.com/matklad [2]: https://ferrous-systems.com https://ferrous-systems.com [3]: https://github.com/rust-analyzer/rust-analyzer https://github.com/rust-analyzer/rust-analyzer
- jmarinez 8y ago
  Sure. Great work! Would rust-analyzer provide enough of a foundation to build something like this?: https://gtoolkit.com https://gtoolkit.com Mind you, in the IDE above reflection builtin the language allows for the incredible amount of power. In Rust's case, the only way I can I see it replicated is at a layer before compile time, hence my interest in your analyzer.
  - matklad 8y ago
    Unlikely: rust-analyzer focuses on static analysis. So it aims to power something akin to IntelliJ IDEA, but probably would be a bad fit for something like lightable/elisp/smalltalk ide.
techvella 8y ago
https://techvella.com/top-five-best-fps-gaming-mouse/ https://techvella.com/top-five-best-fps-gaming-mouse/
zellyn 8y ago
It would be interesting to develop Xi and rust-analyzer in tandem, and see how much of an IDE Xi could become while leaving it still basically a text editor.
- cmyr 8y ago
  Language features are definitely in the near-term plans for xi, and depending on what stage Rust Analyzer is at it could definitely be a good initial target for some of that work.
  - kzrdude 8y ago
    Does xi have any Vim/modal like features?
    - raphlinus 8y ago
      We're still trying to figure out the best way to implement those. There's a prototype, but it hasn't landed in master. It's definitely on the roadmap though.
      - zellyn 8y ago
        I thought Kakoune was an interesting experiment: an attempt to embrace vi-like modal editing, but keep it consistent, and go all-in on it.
        zbraniecki 8y ago
        I really want Kakoune to succeed. I love the UX behind it and it feels like a vim-done-right.
        cmyr 8y ago
        The major innovation of kakoune, imo, is to combine modal editing with multiple selections. These are two very powerful concepts that haven't been combined well before, to the best of my knowledge. In any case yes, I think kak is a great editor and a definite source of inspiration.
      - kzrdude 8y ago
        Sounds cool! I don't know what to say about Vim-like. At this point I think it's mostly a dream that one day, something can replace Vim. Not saying it's perfect, it's just so entrenched, and full of features. That said, I don't think I'd be after a perfect Vim copy. I think I would like an editor that uses the same approach, doesn't have to be exactly the same keybindings.
n-gatedotcom 8y ago
As an end user, one thing I care a lot about is if the dev env would be truly integrated. I want a one-click/command install of everything that I need for my vim to be a rust IDE. I tried in vscode and had to install two different plugins (one of which had a similar name to another and I had to go look up which one to install) python and then lldb using admin permissions which failed on my school computers . Side: Has emacs been losing traction slowly? It's a third of vim!
- theredbox 8y ago
  AFAIK it was always behind VIM. People learned vim because of its ergonomics and install base. Also many people did not know TRAMP is a thing so yeah...
  - n-gatedotcom 8y ago
    Wasn't emacs farther ahead for full-time programmers though? I'd love to see if there were some historical stats.
- pjmlp 8y ago
  For me XEmacs was the go-to environment while I didn't had access to Borland like IDEs on UNIX environments. Nowadays XEmacs seems to have died, I have plenty of IDEs to choose from for my favourite languages, even on GNU/Linux and Emacs seems to still lack some of the nice features of XEmacs. As for VIM, I learned VI on Xenix, but never could bother to actually use it more than for profile files, or as the go-to editor over telnet/ssh sessions when nothing else was installed by default. I assume many on the IDE camp might share a similar experience.
sqs 8y ago
We (Sourcegraph) have also found the “dump all the data from the compiler” approach limiting because it doesn’t work for the live editing use case (as the article says), and then that means you have 2 analyzers: one used with editors and one used for offline tools like Sourcegraph, DXR, etc. That bifurcation means each one gets less community involvement. The ideal IMO is to always target the live editing use case and add caching and optimizations that use language-specific knowledge, so that in both cases you’re querying a live server but it can compute the answers quickly. I haven’t worked with Rust specifically, but it’s interesting to see that they came to a similar conclusion. BTW Sourcegraph is interested in funding this effort and building better Rust support for code browsing and search into our product using https://docs.sourcegraph.com/extensions https://docs.sourcegraph.com/extensions. I just reached out to the author.
- n-gatedotcom 8y ago
  You guys created language server protocol, right? I think langserver.org is owned by sg?
  - sqs 8y ago
    No, Microsoft created LSP, but we’ve built a lot of language servers and added LSP for code intelligence in Sourcegraph (and on code hosts like GitHub and GitLab with https://docs.sourcegraph.com/integration/browser_extension https://docs.sourcegraph.com/integration/browser_extension). (Aside: are you from n-gate.com, as your username suggests?)
    - n-gatedotcom 8y ago
      Oh right. I just googled. Why add two canonical sources when the original github does a fine job and is also publicly editable (by pull requests)?
      - sqs 8y ago
        Microsoft told us they definitely did not plan to make any kind of language server registry, so we saw a community need for an informational site to exist with more details than the wiki page they had. We’ve heard it’s useful, but we don’t want to create duplication. Want to file an issue on the langserver.org repo for discussion?
ceronman 8y ago
To better understand why this project is trying to re-implement the compiler frontend, I really recommend this video from Anders Hejlsberg (creator of C# and TypeScript) explaining how modern compiler construction oriented towards IDE works: https://www.youtube.com/watch?v=wSdV1M7n4gQ https://www.youtube.com/watch?v=wSdV1M7n4gQ
thorel 8y ago
I am not sure I understand the criticism of RLS. It feels to me that the Language Server Protocol does allow for incremental parsing etc. In particular, the approach taken by rust-analyzer > Analyzer maintains a "database" of input facts (source code + information about project structure) and derived facts (syntax trees, name resolution information, types). The client then can change input facts, and query derived facts at the current state of the world. seems that it could also be adopted by a language sever for Rust.
- matklad 8y ago
  The criticism are not about the protocol itself, but about a particular implementation. rust-analyzer also uses LSP as an interface.
- a_humean 8y ago
  The criticism is not directed at the LSP. RLS just implements the specification, but as you said LSP doesn't create these constraints. One outcome of all this might be that RLS internally gets replaced by something like rust-analyzer, which is something that the maintainer of RLS has mused in a recent blog post as one of many possible options for the future of RLS: https://www.ncameron.org/blog/more-on-rls-version-numbering/ https://www.ncameron.org/blog/more-on-rls-version-numbering/
coldtea 8y ago
>The only fundamental problem with IntelliJ Rust is that it is not written in Rust, and I would really love to have a pure-rust solution for IDEs. Why? The main reason is that Rust is the perfect language for building these sorts of tools Citation needed.
- matklad 8y ago
  I assume you are talking about the last bit? This is of course a personal opinion anyone is free to disagree with. However in practice many popular programming languages are implemented in C or C++ (clang, gcc, swift, CPython, V8, SpiderMonkey, HotSpot) and in theory the best language for compilers is [Oca]ml. Rust is a nice synthesis of theory and practice.
  - pjmlp 8y ago
    That is mostly by convenience than anything else, as many devs don't want to bother with the whole process of bootstrapping a programming language. Hotspot long term roadmap is to be rewritten in Java for example, aka Project Metropolis. And not all JVMs are implemented in C or C++.
- saghm 8y ago
  I think it's pretty clear from context that this is the author's opinion.
phlm 8y ago
I find building solid foundation and tooling really important, and it seems where are right there at this point with Rust. But IDEs have always been controversial beasts. Some might say Visual Studio at one point was one of the best representations of good set of trade-offs that made great code-writing environment. Now it's different story. But what I am far more interested in in terms of Rust and it's IDE ergonomics is what can be achieved differently? Rust by itself is different beast. Unique language that pushed different way of thinking and structuring code. So can that be used as an inspiration for something different in terms of IDEs and developer UIs?
georgewfraser 8y ago
There is another approach to bootstrap an incremental compiler frontend on top of a batch compiler, which I used to create a Java language server [1]. Batch compilers usually support incremental compilation at the file level, so you are already close to a solution: re-compile the active file every time the user edits. This works but is too slow. You can dramatically improve the speed by simply erasing all the code that isn’t relevant to the current edits, which usually means erasing all the method bodies. This simple trick allows you to build a fast incremental compiler on an existing batch compiler. [1] https://github.com/georgewfraser/java-language-server https://github.com/georgewfraser/java-language-server
- jamesmunns 8y ago
  This sounds like a use for the everybody-loops compiler flag: https://github.com/rust-lang/rust/pull/19964 https://github.com/rust-lang/rust/pull/19964

181 points by jamesmunns 8y ago

Hey! This is a post by matklad [1], one of the members of the Ferrous Systems [2] team. The Rust Analyzer project [3] is an experimental compiler front-end, designed to power things like code completion, syntax error reporting, and other things that are necessary when building IDE support. Let us know if you have any questions! [1]: https://github.com/matklad https://github.com/matklad [2]: https://ferrous-systems.com https://ferrous-systems.com [3]: https://github.com/rust-analyzer/rust-analyzer https://github.com/rust-analyzer/rust-analyzer

jmarinez 8y ago
Sure. Great work! Would rust-analyzer provide enough of a foundation to build something like this?: https://gtoolkit.com https://gtoolkit.com Mind you, in the IDE above reflection builtin the language allows for the incredible amount of power. In Rust's case, the only way I can I see it replicated is at a layer before compile time, hence my interest in your analyzer.
- matklad 8y ago
  Unlikely: rust-analyzer focuses on static analysis. So it aims to power something akin to IntelliJ IDEA, but probably would be a bad fit for something like lightable/elisp/smalltalk ide.

techvella 8y ago

https://techvella.com/top-five-best-fps-gaming-mouse/ https://techvella.com/top-five-best-fps-gaming-mouse/

zellyn 8y ago

It would be interesting to develop Xi and rust-analyzer in tandem, and see how much of an IDE Xi could become while leaving it still basically a text editor.

cmyr 8y ago
Language features are definitely in the near-term plans for xi, and depending on what stage Rust Analyzer is at it could definitely be a good initial target for some of that work.
- kzrdude 8y ago
  Does xi have any Vim/modal like features?
  - raphlinus 8y ago
    We're still trying to figure out the best way to implement those. There's a prototype, but it hasn't landed in master. It's definitely on the roadmap though.
    - zellyn 8y ago
      I thought Kakoune was an interesting experiment: an attempt to embrace vi-like modal editing, but keep it consistent, and go all-in on it.
      - zbraniecki 8y ago
        I really want Kakoune to succeed. I love the UX behind it and it feels like a vim-done-right.
      - cmyr 8y ago
        The major innovation of kakoune, imo, is to combine modal editing with multiple selections. These are two very powerful concepts that haven't been combined well before, to the best of my knowledge. In any case yes, I think kak is a great editor and a definite source of inspiration.
    - kzrdude 8y ago
      Sounds cool! I don't know what to say about Vim-like. At this point I think it's mostly a dream that one day, something can replace Vim. Not saying it's perfect, it's just so entrenched, and full of features. That said, I don't think I'd be after a perfect Vim copy. I think I would like an editor that uses the same approach, doesn't have to be exactly the same keybindings.

n-gatedotcom 8y ago

As an end user, one thing I care a lot about is if the dev env would be truly integrated. I want a one-click/command install of everything that I need for my vim to be a rust IDE. I tried in vscode and had to install two different plugins (one of which had a similar name to another and I had to go look up which one to install) python and then lldb using admin permissions which failed on my school computers . Side: Has emacs been losing traction slowly? It's a third of vim!

theredbox 8y ago
AFAIK it was always behind VIM. People learned vim because of its ergonomics and install base. Also many people did not know TRAMP is a thing so yeah...
- n-gatedotcom 8y ago
  Wasn't emacs farther ahead for full-time programmers though? I'd love to see if there were some historical stats.
pjmlp 8y ago
For me XEmacs was the go-to environment while I didn't had access to Borland like IDEs on UNIX environments. Nowadays XEmacs seems to have died, I have plenty of IDEs to choose from for my favourite languages, even on GNU/Linux and Emacs seems to still lack some of the nice features of XEmacs. As for VIM, I learned VI on Xenix, but never could bother to actually use it more than for profile files, or as the go-to editor over telnet/ssh sessions when nothing else was installed by default. I assume many on the IDE camp might share a similar experience.

sqs 8y ago

We (Sourcegraph) have also found the “dump all the data from the compiler” approach limiting because it doesn’t work for the live editing use case (as the article says), and then that means you have 2 analyzers: one used with editors and one used for offline tools like Sourcegraph, DXR, etc. That bifurcation means each one gets less community involvement. The ideal IMO is to always target the live editing use case and add caching and optimizations that use language-specific knowledge, so that in both cases you’re querying a live server but it can compute the answers quickly. I haven’t worked with Rust specifically, but it’s interesting to see that they came to a similar conclusion. BTW Sourcegraph is interested in funding this effort and building better Rust support for code browsing and search into our product using https://docs.sourcegraph.com/extensions https://docs.sourcegraph.com/extensions. I just reached out to the author.

n-gatedotcom 8y ago
You guys created language server protocol, right? I think langserver.org is owned by sg?
- sqs 8y ago
  No, Microsoft created LSP, but we’ve built a lot of language servers and added LSP for code intelligence in Sourcegraph (and on code hosts like GitHub and GitLab with https://docs.sourcegraph.com/integration/browser_extension https://docs.sourcegraph.com/integration/browser_extension). (Aside: are you from n-gate.com, as your username suggests?)
  - n-gatedotcom 8y ago
    Oh right. I just googled. Why add two canonical sources when the original github does a fine job and is also publicly editable (by pull requests)?
    - sqs 8y ago
      Microsoft told us they definitely did not plan to make any kind of language server registry, so we saw a community need for an informational site to exist with more details than the wiki page they had. We’ve heard it’s useful, but we don’t want to create duplication. Want to file an issue on the langserver.org repo for discussion?

ceronman 8y ago

To better understand why this project is trying to re-implement the compiler frontend, I really recommend this video from Anders Hejlsberg (creator of C# and TypeScript) explaining how modern compiler construction oriented towards IDE works: https://www.youtube.com/watch?v=wSdV1M7n4gQ https://www.youtube.com/watch?v=wSdV1M7n4gQ

thorel 8y ago

I am not sure I understand the criticism of RLS. It feels to me that the Language Server Protocol does allow for incremental parsing etc. In particular, the approach taken by rust-analyzer > Analyzer maintains a "database" of input facts (source code + information about project structure) and derived facts (syntax trees, name resolution information, types). The client then can change input facts, and query derived facts at the current state of the world. seems that it could also be adopted by a language sever for Rust.

matklad 8y ago
The criticism are not about the protocol itself, but about a particular implementation. rust-analyzer also uses LSP as an interface.
a_humean 8y ago
The criticism is not directed at the LSP. RLS just implements the specification, but as you said LSP doesn't create these constraints. One outcome of all this might be that RLS internally gets replaced by something like rust-analyzer, which is something that the maintainer of RLS has mused in a recent blog post as one of many possible options for the future of RLS: https://www.ncameron.org/blog/more-on-rls-version-numbering/ https://www.ncameron.org/blog/more-on-rls-version-numbering/

coldtea 8y ago

>The only fundamental problem with IntelliJ Rust is that it is not written in Rust, and I would really love to have a pure-rust solution for IDEs. Why? The main reason is that Rust is the perfect language for building these sorts of tools Citation needed.

matklad 8y ago
I assume you are talking about the last bit? This is of course a personal opinion anyone is free to disagree with. However in practice many popular programming languages are implemented in C or C++ (clang, gcc, swift, CPython, V8, SpiderMonkey, HotSpot) and in theory the best language for compilers is [Oca]ml. Rust is a nice synthesis of theory and practice.
- pjmlp 8y ago
  That is mostly by convenience than anything else, as many devs don't want to bother with the whole process of bootstrapping a programming language. Hotspot long term roadmap is to be rewritten in Java for example, aka Project Metropolis. And not all JVMs are implemented in C or C++.
saghm 8y ago
I think it's pretty clear from context that this is the author's opinion.

phlm 8y ago

I find building solid foundation and tooling really important, and it seems where are right there at this point with Rust. But IDEs have always been controversial beasts. Some might say Visual Studio at one point was one of the best representations of good set of trade-offs that made great code-writing environment. Now it's different story. But what I am far more interested in in terms of Rust and it's IDE ergonomics is what can be achieved differently? Rust by itself is different beast. Unique language that pushed different way of thinking and structuring code. So can that be used as an inspiration for something different in terms of IDEs and developer UIs?

georgewfraser 8y ago

There is another approach to bootstrap an incremental compiler frontend on top of a batch compiler, which I used to create a Java language server [1]. Batch compilers usually support incremental compilation at the file level, so you are already close to a solution: re-compile the active file every time the user edits. This works but is too slow. You can dramatically improve the speed by simply erasing all the code that isn’t relevant to the current edits, which usually means erasing all the method bodies. This simple trick allows you to build a fast incremental compiler on an existing batch compiler. [1] https://github.com/georgewfraser/java-language-server https://github.com/georgewfraser/java-language-server

jamesmunns 8y ago
This sounds like a use for the everybody-loops compiler flag: https://github.com/rust-lang/rust/pull/19964 https://github.com/rust-lang/rust/pull/19964