3 ms·
Hmm. I have the opposite impression - separate processes using IPC seems way overrated to me, especially for language servers. When you compare the IntelliJ arc
by mike_hearn 5y ago
Hmm. I have the opposite impression - separate processes using IPC seems way overrated to me, especially for language servers. When you compare the IntelliJ architecture to VSC it's hard to escape the feeling that LSP exists only because JavaScript isn't type or thread safe. In such a language you have no choice but to create a 'backend' of some sort that exchanges messages with the frontend. Yet that's not a great way to implement an IDE. The task is just fundamentally well suited to fast, memory safe, type safe, thread safe languages and very poorly suited to languages like JS and Rust.
Why is that?
1. An in-process type safe API can be evolved and iterated rapidly. There's no need to write informal English specs for everything because the API itself provides so much guidance. Actually, IntelliJ is rather notorious for its absence of JavaDocs though this reflects cultural decisions inside JetBrains itself more than anything intrinsic (they take "code should be self documenting" perhaps a bit too seriously). In contrast, the LSP spec is enormous and the document itself doesn't provide much assistance implementing it (the article discusses this somewhat).
2. A protocol spec doesn't give you any re-usable code. If you write an IntelliJ plugin there's tons of re-usable helper logic and you can easily customize other plugins. The whole thing is designed for on-the-fly presentation compilers. With LSP you may or may not have anything to base your work on.
3. An IDE is fundamentally a highly parallel mutable database, in which the data can be in arbitrarily invalid forms at any point and the user is allowed to change anything whilst many different analyses are running in parallel. Shared data structures are everything in this problem, yet the LSP model effectively forbids all shared state between backend and frontend. This feels like a constraint inherited from browsers - it arguable doesn't make things easier, it just replaces relatively simple and statically/dynamically analyzable locking protocols with complicated sync and rec protocols. Both sides have to maintain their own copies of each other's data structures instead of re-using a single source of truth. This can get really messy, really fast. You're also constantly paying the costs and overheads of multi-process like context switching (especially expensive on Windows), IPC copies, process overhead, serialization etc.
4. A presentation compiler doesn't seem to benefit much from being written in Rust or C++. Developer workstations are fast and have plenty of RAM, so GCd/JITd languages don't seem like a particular problem here. Developer productivity on the other hand is at a premium. People want features. One reason JetBrains can add new languages and features so fast is that they're writing in relatively forgiving languages like Java and Kotlin, where you don't have to think too hard about ownership or memory management. Some thought is still needed - see the disposable model inside their IDEs - but you can more or less just go for it with a high rate of feature throughput.
You raise the question of how to kill hung operations.
In practice that doesn't seem to be a common problem. I've been using IntelliJ with various languages for years and there have been many bugs, but I can't remember any time that a plugin hard-hung whilst analyzing. There are cases where plugin bugs happen but exceptions and GC mean the IDE can simply catch the errors and disable it. These days it can do that temporarily and simply reload the plugin on the fly. Other times e.g. if a pure static analysis crashes it just logs the error and uploads it for analysis. That particular analysis might not appear as a consequence but there are so many different optional quality-of-life analyses being done you often hardly notice.
- zozbot234 5y ago> Shared data structures are everything in this problem, yet the LSP model effectively forbids all shared state between backend and frontend. The "shared state" in this context is IDE program text, and analyzer diagnostics. Copying those across address spaces and keeping them in sync is super fast, the overhead is too low to matter.
- mike_hearn 5y agoAnd auto-completion suggestions (potentially means switching context back and forth on every keystroke), and inline editor hints / styling / folding regions, whatever is being changed by refactorings and so on. Recall that most analysis is done async, so results arrive at different times. Even if you're not doing anything then, your IDE/backend will be context switching back and forth as stuff arrives to be rendered. In the limit this degrades to the backend being the entire IDE and the frontend being just an translator between the OS IO system a stream of JSON rendering commands.
- carlmr 5y ago>The task is just fundamentally well suited to fast, memory safe, type safe, thread safe languages and very poorly suited to languages like JS and Rust. Huh? The first part of the sentence makes me think of Rust. It's fast, memory safe, type safe, and thread safe.
- mike_hearn 5y agoBut JS isn't, so people look for other languages to fill the gaps. They seem to often settle on Rust, but it pays a relatively high productivity price to get those properties without garbage collection. Is GC a problem in language analysis systems? Doesn't feel that way these days, and anyway JVMs have pauseless GCs anyway now.
- carlmr 5y agoI don't think GC is an issue, pauses would usually not even be noticed by the user. In terms of speed, as long as you don't choose Python I think you'll be fine. I do think in terms of productivity, something like F#, that has GC, ML style syntax with piping and partial application, and a strict enough type system with Hindley-Milner type inference, would be the most productive. My hold up was mostly with the sentence I quoted which seems to contradict itself.