6 ms·
We (Sourcegraph) have also found the “dump all the data from the compiler” approach limiting because it doesn’t work for the live editing use case (as the artic
by sqs 8y ago
We (Sourcegraph) have also found the “dump all the data from the compiler” approach limiting because it doesn’t work for the live editing use case (as the article says), and then that means you have 2 analyzers: one used with editors and one used for offline tools like Sourcegraph, DXR, etc. That bifurcation means each one gets less community involvement. The ideal IMO is to always target the live editing use case and add caching and optimizations that use language-specific knowledge, so that in both cases you’re querying a live server but it can compute the answers quickly. I haven’t worked with Rust specifically, but it’s interesting to see that they came to a similar conclusion.
BTW Sourcegraph is interested in funding this effort and building better Rust support for code browsing and search into our product using https://docs.sourcegraph.com/extensions https://docs.sourcegraph.com/extensions. I just reached out to the author.
- n-gatedotcom 8y agoYou guys created language server protocol, right? I think langserver.org is owned by sg?
- sqs 8y agoNo, Microsoft created LSP, but we’ve built a lot of language servers and added LSP for code intelligence in Sourcegraph (and on code hosts like GitHub and GitLab with https://docs.sourcegraph.com/integration/browser_extension https://docs.sourcegraph.com/integration/browser_extension). (Aside: are you from n-gate.com, as your username suggests?)
- n-gatedotcom 8y agoOh right. I just googled. Why add two canonical sources when the original github does a fine job and is also publicly editable (by pull requests)?
- sqs 8y agoMicrosoft told us they definitely did not plan to make any kind of language server registry, so we saw a community need for an informational site to exist with more details than the wiki page they had. We’ve heard it’s useful, but we don’t want to create duplication. Want to file an issue on the langserver.org repo for discussion?
- nerpderp82 8y agoLove your about page!
- AlexCoventry 8y agoYou are violating The Prime Directive.
- n-gatedotcom 8y agoWhat would that be?
- AlexCoventry 8y agoIf you can't figure it out, perhaps you are truly a Hackernews. :-)
- n-gatedotcom 8y agoI don't see why one should be well versed in stupid hollywood-isms.:)And hackernews is a derogatory term that I will have you know I am very offended by. Better only than web-boy
- matklad 8y agoThe thing is, if you target only live editing case, you can become slower for "batch compilation" case. For source browsers this probably isn't too important, b/c one can add caching using something like LSIF. However for command line compilers a similar caching won't work: in the majority of cases, you compile new code. But adding fast "batch mode" on top of "live" editing seems easier than going in the opposite direction.
- nicoburns 8y agoHow often does one do non-incremental compiles though?
- losvedir 8y agoI think it's pretty common for CI, but I guess that use case can handle a little lower performance.
- smaddox 8y agoAnd for each CI compile, how many times have devs done an incremental compile? Probably a lot.
- adrianN 8y agoI've worked on projects that took over an hour to compile. I would not like lower performance for CI. Many workflows require successful CI runs before one can merge for example.
- nicoburns 8y agoIt's really only Rust and C++ that ever take that long, right? IMO, that's simply the compiler being unacceptably slow. I also can't see a slightly less efficient parse tree being a big factor in compile times this kind of length...
- woolvalley 8y agoWell with a good build cache, CI can do things like incremental compiles too. You can even put the cache on the network and have most of your CI use incremental compiles.
- the8472 8y agoEclipse does make this work for java with ECJ. It gains total and correct knowledge of the code base by instrumenting a compiler designed for incremental compilation. But it's also usable for batch compilation.
- hinkley 8y agoThere's a whole class of problems around this that I'd like to see get some more attention. It shows up for all kinds of static analysis in code editors, including things as simple as autocomplete. For instance, if I have a file that compiles and I start typing in the space between two function bodies, I'm probably implementing a new function. In fact that probability is so high that you should just assume that to be the case, even if the code I've written so far doesn't have balanced nesting yet. All decisions and advice should be based on the speculation that the code will eventually represent a new set of blocks, not a reorganization of the remaining ones. You can't make the same assumption about if statements inside of a function, or about incomplete multiline comments, but you can assume that anywhere from zero to all of the sibling blocks are about to have their scope changed (but not ancestors). In short, the gap between how I'd describe a commit and how the diff tool represents it also prevents live code analysis from being able to work as well as it might.
- skybrian 8y agoIt seems like parsing both forwards in the normal way and backwards from the end of the file would be helpful? That is, take the longest suffix that parses correctly and assume any syntax errors are before that for error-reporting purposes. Most programming languages aren't designed for parsing backwards, though!
- hinkley 8y agoBackward in time, not a linear file scan.
- matklad 8y agoThat's not a particularly hard problem, all tools which try to solve it (for example, IntelliJ), solve it easily. For example, here's the syntax tree that rust-analyzer builds for this example: fn foo() { } fn fn bar() { } Syntax tree (whitespace nodes elided for clarity): SOURCE_FILE@[0; 31) FN_DEF@[0; 12) FN_KW@[0; 2) NAME@[3; 6) IDENT@[3; 6) "foo" ... FN_DEF@[14; 16) FN_KW@[14; 16) err: `expected a name` FN_DEF@[18; 30) FN_KW@[18; 20) NAME@[21; 24) IDENT@[21; 24) "bar" ... The problem is, a lot of tools are build with the assumption that the code is correct&complete, and don't try to tackle this problem at all.