4 ms·
Hey! Author here. Happy to answer any questions.
by popzxc 1mo ago
Hey! Author here.
Happy to answer any questions.
- bip-bop-robot 1mo agoRust does not have a specification. How do you know your LSP is providing right information?
- popzxc 1mo agoWell, most of stuff is not really ambiguous: if you have a struct and found its inherent impl for it, then methods from this impl block are related to this structure. If `a` has type `Foo` and then you have `let b = a;`, then `b` has type `Foo` too. With things like trait solving I am not reinventing the wheel, and use official tooling (Chalk). Even though now the new solver is recommended, Chalk still does its job and lets me not to worry about potentially the most complex part of the machinery. In places that seem to be underdocumented, it's always possible to: 1) look into sysroot implementation for clues 2) look into compiler sources 3) hijack stuff from rust-analyzer I am lucky to not be the first guy who does a Rust LSP, so it's not that fundamental of a research, and much more of just an implementation :)
- potamic 1mo agoCould you elaborate a bit on why RA's incremental approach takes more memory? Intuitively it feels that it should take less, because you're only processing what you need? Whereas you seem to indicate that you save a full analysis snapshot to disk and load it all up when needed? Shouldn't that consume the max memory for a workspace? Fantastic project btw, and it couldn't come at a better time. With the way prices are going I really hope people start paying attention to memory again.
- popzxc 1mo agoIt's explained in the blog post, but in short: rust-analyzer stores the data it needs in memory all the time, while Rust Glancer might consume more memory during indexing (because it's not lazy and does more indexing), but after that it only loads _necessary_ information for the duration of the query. Several things here: 1. We don't need all the information (project can have 1000+ dependencies, while query might only care about the current open file), so the amount of information we load is smaller. 2. Most of the time IDE does not actually do any queries, so if you switch to browser/Slack, you don't pay the tax. 3. Since data is loaded to the disk, after initial indexing restarting no longer consumes that much ram, and you get reindexing for free. 4. Besides offloading, I implement quite a bit of memory optimizations (some of which are covered in docs: https://rust-glancer.github.io/docs/development/MEMORY.html https://rust-glancer.github.io/docs/development/MEMORY.html ), so it's a combination of factors.
- rtpg 1mo agoLet's say I type "SomeStructure" and haven't imported it, how does rust glancer _not_ end up poking at all the dependencies to figure out where it is, and to import it? In some sense I feel like that example is the canonical "you need a full picture of the world" case, to the point that it's probably worth special casing _just_ that, and using local analysis for everything... but it seems hard to avoid that pain on any typo for example Inversely... do you have a ballpark for what the size of the on-disk structures end up being? Not that it matters _that_ much but Rust projects already have a tendancy to be disk hogs during development
- popzxc 1mo agoOn the first question -- auto imports is an example of a "heavy" functionality since we indeed need to scan more than is imported in the project, which is why I'm still working on this to optimize properly (it's decently fast, but I want it to be faster before I'll enable it, since it's also something we want to see in completions, and completions must be _very_ fast). Still, we don't need "everything": we only need _reachable_ items in _direct_ dependencies; typically project has much less direct dependencies than transitive ones (e.g. 20 direct dependencies can fan out to 1000 transitive deps), and each dependency usually has less exported items than total amount of items. Finally, for that you only need semantic data (e.g. items) rather than bodies. So while it's a big task (though finding references is still significantly bigger and tougher, because that's where it gets to "it can be anywhere in reverse dependencies bodies), it's not "we need the whole world". As for the size -- the Rust Glancer artifacts for Rust Glancer itself currently take 225mb. And that's proportial to the project size only, e.g. it doesn't grow over time on its own.
- afdbcreid 1mo agoUnfortunately that does not work. Consider: // Crate indirect_dep: pub struct Foo; // Crate direct_dep: extern crate indirect_dep; pub use indirect_dep::*; // Crate my_crate: use direct_dep::$0; ($0 denotes the cursor). You cannot know what to complete without analyzing `indirect_dep`. Add macros to the mix and you're going to get a nightmare (yes, name resolution in Rust is a nightmare). In fact, you sometimes need to do type inference inside bodies to infer other bodies, because auto traits propagate across opaques. You might say this does not matter because it only matters for diagnostics... But it does not. It can matter for method resolution. In general, I'm fairly sure that it is just impossible to build a 100% correct analyzer for Rust code without a query system like rustc's (it can be stored on disk, that's a different story). You can go pretty far without - and this will be enough for some people for an IDE! - but not all the way.
- meerita 1mo agoSuper nice! Are you planning support for Zed editor?
- positron26 1mo agoThe point of language servers is to support all editors that speak LSP.
- legobmw99 1mo agoZed is an editor that requires you to still write a tiny plugin basically telling it how to download and start the server, unlike something like emacs where this is just user config
- positron26 1mo agoInternet is dead.
- legobmw99 1mo agoHuh, first time getting called a bot for me. I assume it’s the “Zed is an editor”, I was originally going to say something like “Zed is one of those editors like VScode that requires plugins for LSPs” and then shortened it
- positron26 1mo agoNo, even the humans are dead. Both the "user configuration" and the library are configuration masquerading as code. One is dynamically evaluated, the other compiled. I just figure users of Zed know this... so what are we pretending to talk about?
- tclancy 1mo agoHow come I can’t get no Tang around here?
- medzernik 1mo agoplease support the zed editor (pleading face emoji)
- dbdr 1mo agoSuper cool! In the comparison table, you indicate indexing times. Could you also measure memory usage, since that's the stated goal of the project?
- popzxc 1mo agoI will work on creating a more or less fair benchmark soon-ish, but right now the initial indexing typically consumes more RAM than rust analyzer does, but not awfully so. The difference, however, is that with Rust Glancer you don’t need full reindexing often, so it probably compensates for that to a degree.
- dbdr 1mo agoOh, if peak RAM usage is higher, in which scenario do you find that's a significant gain? Maybe I misunderstood, what I got from the blog post is that there will be indexing on save (vs on each keystroke with RA). That would be often enough that lower RAM enough in between would not be of much gain. Is that only an incremental indexing with normally low RAM usage? So you would essentially fully index only once per project, + whenever you upgrade dependencies or upgrade rustc?
- avianlyric 1mo agoIt looks like it only partially reindexes on save, but it completely reindexes the single file that’s saved, rather than doing partial reindex of the file on every keystroke like rust-analyser. But importantly it’s not reindexing the entire project, which is the expensive operation. So you have higher peak RAM when opening a brand new project, but much lower RAM usage while working on the project.
- popzxc 1mo agoExactly. You pay higher RAM usage price once, for 5-30 seconds at the very beginning of the project (or if you make changes that invalidate the dependency graph, which is rather rare). In 95% of cases and 99.999% of idle time using the editor, you enjoy lower RAM usage. And note that on dirty buffers Rust Glancer doesn't do reindexing at all: it reuses the last available analysis, plus it does syntax-based shallow overlay that is sufficient to be useful but doesn't necessarily detect semantic changes. It's a tradeoff, but this tradeoff makes Rust Glancer competitive in terms of latency with rust-analyzer without compromising RAM and while keeping your CPU cool. > it completely reindexes the single file that’s saved This is true, though I have to mention that change in the file might invalidate its reverse dependencies, which can make the partial analysis bigger than just one file/crate, but it's still very fast in practice.
- dkersten 1mo agoPretty cool! rust-analyzer takes such a huge amount of memory. Usually it’s not a problem but occasionally I’ve run into issues. Having an alternative, even with tradeoffs, is great.
- cjg007 1mo ago[flagged]
- rdescartes 1mo agoCould you mind to tell me what is your plan on proc-macros ? You said you have an idea to “will not require actual code execution”. On the other hand, RA current method of handling in Proc-Macros are very fragile.
- popzxc 1mo agoThe exact shape is TBD (I target proc macros support for 0.3.0, whereas 0.2.0 will be about completeness/editors support), but in short -- I'm thinking about "plugin"/DSL architecture where proc macro _effects_ can be described beforehand and applied by the LSP without actual code execution. Executing proc macros is a lot of effort, but LSP mostly cares about the externally observable effects, which is a much smaller subset.
- rdescartes 1mo agoIf it’s possible, that would be awesome. Do you mean that the author of the proc macro provides that information through attributes? (Disclosure: I originally implemented the proc-macro part of RA.)