6 ms·

LSP is pretty ok. Better than the before times I suppose. Although C++ has long had good IDE support so it hasn't affected me too much. I have a maybe wrong an

LSP is pretty ok. Better than the before times I suppose. Although C++ has long had good IDE support so it hasn't affected me too much. I have a maybe wrong and bad opinion that LSP is actually at the wrong level. Right now every language needs to implement a from scratch implementation of their LSP server. These implementations are HUGE and take YEARS to develop. rust-analyzer is over 365,000 lines of code. And every language has their own massive, independent implementation. When it comes to debugging all native language support common debug symbol formats. PDB for Windows and DWARF for Nixy things. Any compiled language that uses LLVM gets debug symbols and rich debugging "for free". I think there should be a common Intellisense Database file format for providing LSP or LSP-like capabilities. Ok sure there will still be per-language work to be done to implement the IDB format. But you'd get like 95% of the implementation for free for any LLVM language. And generating a common IDB format should be a lot simpler than implementing a kajillion LSP protocols. My dream world has a support file that contains: full debug symbols, full source code, and full intellisense data. It should be trivial to debug old binaries with full debugging, source, and intellisense. This world could exist and is within reach!

PoignardAzur 2y ago
> rust-analyzer is over 365,000 lines of code. That has nothing to do with LSP. Rust Analyzer is similar in scope to a second implementation of the Rust compiler.
- forrestthewoods 2y ago
  > similar in scope to a second implementation of the Rust compiler. I know. That’s really bad!
  - nobodywasishere 2y ago
    I disagree. A compiler for batch building programs and a compiler for providing as much semantic information about incomplete/incorrect/constantly changing programs are completely different tasks that require completely different architectures and design considerations.
    - forrestthewoods 2y ago
      I don’t think that’s true at all. First of all, a compiler for a 100% correct program definitely has all the necessary information for robust intellisense. They don’t currently save all the data, but it should exist. So the only real question is whether they can support the 0.01% of files that incomplete and changing? I’ll readily admit I am not a compiler expert. So I’m open to being wrong. But I certainly don’t see why not. Compilers already need to support incorrect code so they can print helpful error messages. Including different errors spread through out a single file. It may be that current compilers are badly architected for incremental intellisense generation. But I don’t think that’s an intrinsic difference. I see no reason that the tasks require “completely different architectures”.
      - troupo 2y ago
        > First of all, a compiler for a 100% correct program definitely has all the necessary information for robust intellisense. It doesn't. Intellisense is supposed to work on 100% incorrect and incomplete programs. To the point that it should work in syntactically invalid code.
        forrestthewoods 2y ago
        > Intellisense is supposed to work on 100% incorrect and incomplete programs. Correct. I literally discussed this scenario in my comment! If the program compiles successfully then the compiler has all the information it needs for intellisense. If the program does NOT fully compile then the compiler may or may not be able to emit sufficient intellisense information. I assert that compilers should be able to support this common scenario. It is not particularly different from needing to support good, clear error messages in the face of syntactically invalid code.
        troupo 2y ago
        > I assert that compilers should be able to support this common scenario Not necessarily. These are two very different tasks quite at odds with each other
- czei002 2y ago
  I would blame Rust though. For example, Rust has macros which are way too powerful and make it very hard to write a LSP (https://rust-analyzer.github.io/blog/2021/11/21/ides-and-macros.html https://rust-analyzer.github.io/blog/2021/11/21/ides-and-mac...) Very interesting is how Roslyn/Typescript does it: https://www.youtube.com/watch?v=qnyOHY7AiZk https://www.youtube.com/watch?v=qnyOHY7AiZk
nobodywasishere 2y ago
That sounds similar to LSIF https://microsoft.github.io/language-server-protocol/specifications/lsif/0.6.0/specification/ https://microsoft.github.io/language-server-protocol/specifi...
Zababa 2y ago
"any LLVM language" is a lot but also not that much. You're missing Python, JS, Go, Ruby, etc.
aidenn0 2y ago
1. I use zero languages that use either PDB or DWARF that are not named "C". 2. You are either overestimating the level of detail available in PDB/DWARF or underestimating the massive amount of language-specific work needed for even basic features (e.g. methods, which lack any cross-language ABI) given just what PDB/DWARF give you. 3. What LSP provides and what PDB/DWARF offer are only very loosely related. Consider the case of writing function1, then (without compiling) writing function2 that calls function1. It is typical for an LSP to offer completion and argument information when writing out the call for function1. That's not something you get "for free" with PDB/DWARF.
- forrestthewoods 2y ago
  > You are either overestimating the level of detail available in PDB/DWARF Uhhh. I didn’t say PDB/DWARF already have the necessary information. In fact I even proposed a new file format! I suggest you re-read what I said. > Consider the case of writing function1, then (without compiling) writing function2 that calls function1. It is typical for an LSP to offer completion and argument information when writing out the call for function1. That's not something you get "for free" with PDB/DWARF. What do you think LSP servers do in the background? They’re effectively compilers that are CONSTANTLY compiling the code. Amusingly rust-analyzer takes longer to bootstrap than a full and complete clean and build. Maybe it’s not as parallel, I’m not sure.
ReleaseCandidat 2y ago
Rust-analýzer is an example of what not to do, which is reimplementing a compiler frontend. Ideálly it should be the samé as the "real" compiler is using. Of course this has it's own problems, which the Haskell LSP this post is about, shows. As compilers not written for being used "interactively". > Any compiled language that uses LLVM gets debug symbols and rich debugging "for free". That doesn't hold for C++ and much less for any language even "less C" than C++. Like languages using a GC, e.g. Roc https://www.roc-lang.org/ https://www.roc-lang.org/
- forrestthewoods 2y ago
  > That doesn't hold for C++ What do you mean? Why not? Clang PDBs for C++ work great. A GC isn’t particular disruptive to debug symbols afaik.
  - ReleaseCandidat 2y ago
    > What do you mean? Why not? You need support for each language in the debugger, as symbols do not contain semantics. As users we for example know that `foo::bar` and `foo::baz` are methods of the same class `foo`, the debugger doesn't. The problem with GCs is that pointers must contain some additional information (like an additional bit for mark and sweep), they are not "just" pointing to some memory. Without "knowing" that, the debugger cannot follow the pointer to its target. Or tricks with unboxed ints like making them 1 bit smaller and using the first 1 as a tag for "this is not a pointer, but an integer".
thramp 2y ago
I've responded on reddit before (https://www.reddit.com/r/rust/comments/1eqqwa7/comment/lhwwns2/ https://www.reddit.com/r/rust/comments/1eqqwa7/comment/lhwwn...), but I'll restate and cover some other things here. > I have a maybe wrong and bad opinion that LSP is actually at the wrong level. Right now every language needs to implement a from scratch implementation of their LSP server. These implementations are HUGE and take YEARS to develop. rust-analyzer is over 365,000 lines of code. And every language has their own massive, independent implementation. rust-analyzer a big codebase, but it's also less problematic than the raw numbers would make you think. rust-analyzer has a bunch of advanced functionality (term search https://github.com/rust-lang/rust-analyzer/pull/16092 https://github.com/rust-lang/rust-analyzer/pull/16092 and refactors), assists (nearly 20% of rust-analyzer!) and tests. > I think there should be a common Intellisense Database file format for providing LSP or LSP-like capabilities. Ok sure there will still be per-language work to be done to implement the IDB format. I think you might be describing formats like (https://code.visualstudio.com/blogs/2019/02/19/lsif https://code.visualstudio.com/blogs/2019/02/19/lsif) and SCIP (https://github.com/sourcegraph/scip https://github.com/sourcegraph/scip). I personally like SCIP a bit more LSIF because SCIP's design makes it substantially easier to incrementally update a large index. We use SCIP with Glean (https://glean.software/ https://glean.software/) at work; it's pretty nice. > But you'd get like 95% of the implementation for free for any LLVM language. And generating a common IDB format should be a lot simpler than implementing a kajillion LSP protocols. I wouldn't say 95%. SCIP/LSIF can do the job for navigation, but that's only a subset of what you want from an IDE. For example: - Intellisense/autocomplete is extremely latency sensitive where milliseconds count. If you have features like Rust/Haskell's traits/typeclasses that allow writing blanket implementations like `impl<T> SomeTrait for T`, it's often faster to try to solve that trait bound on-the-fly than storing/persisting that data. - It'd be nice to handle features like refactors/assists/lightbulbs. That's going to result in a bunch of de novo code needs to exist outside of a standard compiler, not counting all the supporting infrastructure. > My dream world has a support file that contains: full debug symbols, full source code, and full intellisense data. Rust tried something similar in 2017 with the Rust Language Server (RLS, https://github.com/rust-lang/rls https://github.com/rust-lang/rls). It worked, but most people found it too slow because it was invoking a batch compiler on every keystroke.

PoignardAzur 2y ago

> rust-analyzer is over 365,000 lines of code. That has nothing to do with LSP. Rust Analyzer is similar in scope to a second implementation of the Rust compiler.

forrestthewoods 2y ago
> similar in scope to a second implementation of the Rust compiler. I know. That’s really bad!
- nobodywasishere 2y ago
  I disagree. A compiler for batch building programs and a compiler for providing as much semantic information about incomplete/incorrect/constantly changing programs are completely different tasks that require completely different architectures and design considerations.
  - forrestthewoods 2y ago
    I don’t think that’s true at all. First of all, a compiler for a 100% correct program definitely has all the necessary information for robust intellisense. They don’t currently save all the data, but it should exist. So the only real question is whether they can support the 0.01% of files that incomplete and changing? I’ll readily admit I am not a compiler expert. So I’m open to being wrong. But I certainly don’t see why not. Compilers already need to support incorrect code so they can print helpful error messages. Including different errors spread through out a single file. It may be that current compilers are badly architected for incremental intellisense generation. But I don’t think that’s an intrinsic difference. I see no reason that the tasks require “completely different architectures”.
    - troupo 2y ago
      > First of all, a compiler for a 100% correct program definitely has all the necessary information for robust intellisense. It doesn't. Intellisense is supposed to work on 100% incorrect and incomplete programs. To the point that it should work in syntactically invalid code.
      - forrestthewoods 2y ago
        > Intellisense is supposed to work on 100% incorrect and incomplete programs. Correct. I literally discussed this scenario in my comment! If the program compiles successfully then the compiler has all the information it needs for intellisense. If the program does NOT fully compile then the compiler may or may not be able to emit sufficient intellisense information. I assert that compilers should be able to support this common scenario. It is not particularly different from needing to support good, clear error messages in the face of syntactically invalid code.
        troupo 2y ago
        > I assert that compilers should be able to support this common scenario Not necessarily. These are two very different tasks quite at odds with each other
czei002 2y ago
I would blame Rust though. For example, Rust has macros which are way too powerful and make it very hard to write a LSP (https://rust-analyzer.github.io/blog/2021/11/21/ides-and-macros.html https://rust-analyzer.github.io/blog/2021/11/21/ides-and-mac...) Very interesting is how Roslyn/Typescript does it: https://www.youtube.com/watch?v=qnyOHY7AiZk https://www.youtube.com/watch?v=qnyOHY7AiZk

nobodywasishere 2y ago

That sounds similar to LSIF https://microsoft.github.io/language-server-protocol/specifications/lsif/0.6.0/specification/ https://microsoft.github.io/language-server-protocol/specifi...

Zababa 2y ago

"any LLVM language" is a lot but also not that much. You're missing Python, JS, Go, Ruby, etc.

aidenn0 2y ago

1. I use zero languages that use either PDB or DWARF that are not named "C". 2. You are either overestimating the level of detail available in PDB/DWARF or underestimating the massive amount of language-specific work needed for even basic features (e.g. methods, which lack any cross-language ABI) given just what PDB/DWARF give you. 3. What LSP provides and what PDB/DWARF offer are only very loosely related. Consider the case of writing function1, then (without compiling) writing function2 that calls function1. It is typical for an LSP to offer completion and argument information when writing out the call for function1. That's not something you get "for free" with PDB/DWARF.

forrestthewoods 2y ago
> You are either overestimating the level of detail available in PDB/DWARF Uhhh. I didn’t say PDB/DWARF already have the necessary information. In fact I even proposed a new file format! I suggest you re-read what I said. > Consider the case of writing function1, then (without compiling) writing function2 that calls function1. It is typical for an LSP to offer completion and argument information when writing out the call for function1. That's not something you get "for free" with PDB/DWARF. What do you think LSP servers do in the background? They’re effectively compilers that are CONSTANTLY compiling the code. Amusingly rust-analyzer takes longer to bootstrap than a full and complete clean and build. Maybe it’s not as parallel, I’m not sure.

ReleaseCandidat 2y ago

Rust-analýzer is an example of what not to do, which is reimplementing a compiler frontend. Ideálly it should be the samé as the "real" compiler is using. Of course this has it's own problems, which the Haskell LSP this post is about, shows. As compilers not written for being used "interactively". > Any compiled language that uses LLVM gets debug symbols and rich debugging "for free". That doesn't hold for C++ and much less for any language even "less C" than C++. Like languages using a GC, e.g. Roc https://www.roc-lang.org/ https://www.roc-lang.org/

forrestthewoods 2y ago
> That doesn't hold for C++ What do you mean? Why not? Clang PDBs for C++ work great. A GC isn’t particular disruptive to debug symbols afaik.
- ReleaseCandidat 2y ago
  > What do you mean? Why not? You need support for each language in the debugger, as symbols do not contain semantics. As users we for example know that `foo::bar` and `foo::baz` are methods of the same class `foo`, the debugger doesn't. The problem with GCs is that pointers must contain some additional information (like an additional bit for mark and sweep), they are not "just" pointing to some memory. Without "knowing" that, the debugger cannot follow the pointer to its target. Or tricks with unboxed ints like making them 1 bit smaller and using the first 1 as a tag for "this is not a pointer, but an integer".

thramp 2y ago

I've responded on reddit before (https://www.reddit.com/r/rust/comments/1eqqwa7/comment/lhwwns2/ https://www.reddit.com/r/rust/comments/1eqqwa7/comment/lhwwn...), but I'll restate and cover some other things here. > I have a maybe wrong and bad opinion that LSP is actually at the wrong level. Right now every language needs to implement a from scratch implementation of their LSP server. These implementations are HUGE and take YEARS to develop. rust-analyzer is over 365,000 lines of code. And every language has their own massive, independent implementation. rust-analyzer a big codebase, but it's also less problematic than the raw numbers would make you think. rust-analyzer has a bunch of advanced functionality (term search https://github.com/rust-lang/rust-analyzer/pull/16092 https://github.com/rust-lang/rust-analyzer/pull/16092 and refactors), assists (nearly 20% of rust-analyzer!) and tests. > I think there should be a common Intellisense Database file format for providing LSP or LSP-like capabilities. Ok sure there will still be per-language work to be done to implement the IDB format. I think you might be describing formats like (https://code.visualstudio.com/blogs/2019/02/19/lsif https://code.visualstudio.com/blogs/2019/02/19/lsif) and SCIP (https://github.com/sourcegraph/scip https://github.com/sourcegraph/scip). I personally like SCIP a bit more LSIF because SCIP's design makes it substantially easier to incrementally update a large index. We use SCIP with Glean (https://glean.software/ https://glean.software/) at work; it's pretty nice. > But you'd get like 95% of the implementation for free for any LLVM language. And generating a common IDB format should be a lot simpler than implementing a kajillion LSP protocols. I wouldn't say 95%. SCIP/LSIF can do the job for navigation, but that's only a subset of what you want from an IDE. For example: - Intellisense/autocomplete is extremely latency sensitive where milliseconds count. If you have features like Rust/Haskell's traits/typeclasses that allow writing blanket implementations like `impl<T> SomeTrait for T`, it's often faster to try to solve that trait bound on-the-fly than storing/persisting that data. - It'd be nice to handle features like refactors/assists/lightbulbs. That's going to result in a bunch of de novo code needs to exist outside of a standard compiler, not counting all the supporting infrastructure. > My dream world has a support file that contains: full debug symbols, full source code, and full intellisense data. Rust tried something similar in 2017 with the Rust Language Server (RLS, https://github.com/rust-lang/rls https://github.com/rust-lang/rls). It worked, but most people found it too slow because it was invoking a batch compiler on every keystroke.