9 ms·

Language Servers and IDEs

139 points by lkurusa 9y ago

thorel 9y ago
Worth mentioning the Google Kythe project [0] which is designed around the idea of providing a graph of entities appearing in the code to build tools which work with code. I always thought that it would be good if the Google Kythe and the Language server protocol projects could merge, i.e. if the Language server protocol could expose a Kythe-like graph to editors. This seems to be along the lines of what OP suggests. [0] https://kythe.io/ https://kythe.io/
sqs 9y ago
Interesting post. I have worked a lot with LSP personally and professionally. I'm CEO of Sourcegraph, and we use LSP everywhere. (Go to https://sourcegraph.com/github.com/mholt/caddy/-/blob/caddyhttp/httpserver/https.go#L22:23$references https://sourcegraph.com/github.com/mholt/caddy/-/blob/caddyh... and open your browser's network inspector Web Socket list to see it live, for example.) I think LSP's success is a demonstration of the "less is more" principle. It is simple and designed only to satisfy an editor's needs. But unlike many similar attempts, LSP has a lot of traction; see https://github.com/Microsoft/language-server-protocol/wiki/Protocol-Implementations https://github.com/Microsoft/language-server-protocol/wiki/P... for a list of implementations (http://langserver.org/ http://langserver.org/ for more details). One of the reasons why this simplicity is necessary is that if a protocol even tries to represent structure or relationships across many different languages, it becomes harder and harder to add support for more languages. You can try to make it work for N languages, but the (N+1)th language always has some new language or syntax element that requires you to either change the specification (and update the N previous implementations) or deal with hacks (which goes against the whole reason to design the "one schema to rule them all"). A second reason is that even if you can make it work for all the languages you care about, it's very difficult to find enough language experts across all of those languages to build and maintain the implementations. It takes a lot of time to understand and communicate a complex schema and how the edge cases are handled across the various languages. And in the time it takes you to find and educate that team, you've already fallen behind new languages features being added or new popular languages coming on the scene. (Unless you are Google. Their Kythe project is an exception. :) LSP has explicitly rejected the goal of modeling complex language features, at least so far. Perhaps with the impressive initial traction they could add it on and succeed where many other attempts haven't yet. But I doubt it. (If you want to read more background about LSP, in addition to the Microsoft repo, you can check out some articles my cofounder wrote: https://text.sourcegraph.com/how-sourcegraph-scales-with-the-language-server-protocol-a4e8fd3fbae5 https://text.sourcegraph.com/how-sourcegraph-scales-with-the..., https://text.sourcegraph.com/part-2-how-sourcegraph-scales-with-the-language-server-protocol-73b86175f38b https://text.sourcegraph.com/part-2-how-sourcegraph-scales-w..., and https://text.sourcegraph.com/sourcegraph-code-intelligence-and-the-language-server-protocol-f2479fdd92ef. https://text.sourcegraph.com/sourcegraph-code-intelligence-a...)
- xg15 9y ago
  > It is simple and designed only to satisfy an editor's needs. But isn't that exactly the point of the post? The approach is nice for editors but unsuitable for IDEs. > You can try to make it work for N languages, but the (N+1)th language always has some new language or syntax element that requires you to either change the specification (and update the N previous implementations) or deal with hacks (which goes against the whole reason to design the "one schema to rule them all"). Seems to me LSP has the exact same problem with use-cases. By the description of the post, the goal seems to be to solve a fixed set of use-cases (highlighting, autocompletion, etc) for a flexible set of languages. If you want to add use-case n+1, you're faced with the same rewrite/hack conundrum. (Disclaimer: I don't have much expert with LSP)
  - seren 9y ago
    Isn't LSP an application of Pareto Principle ? 80% of your needs are covered by small set of features.
  - sqs 9y ago
    > But isn't that exactly the point of the post? The approach is nice for editors but unsuitable for IDEs. Yep. I'm just giving my thoughts on what it'd take to extend LSP so it addresses the issues raised in the blog post. I think it's much harder than it might seem. > Seems to me LSP has the exact same problem with use-cases. By the description of the post, the goal seems to be to solve a fixed set of use-cases (highlighting, autocompletion, etc) for a flexible set of languages. If you want to add use-case n+1, you're faced with the same rewrite/hack conundrum. The difference is between "What does an editor need?" and "What is a faithful representation of an element of code in a programming language?" The former is easy (and it's what LSP does) even to do for 50 languages; the latter is way harder than the casual reader would expect.
- Analemma_ 9y ago
  I agree with everything you've said, especially how LSP's simplicity has been key to its success. I do also agree with the author and would like a standard extensibility point, so that LSP could be the One True Way for every language to have full support in every IDE. I just want that to be done slowly and carefully, because it would be very easy to get wrong. If I had to guess, I'd say this might be coming in the future. Possibly Microsoft wanted to break out of its habit of overengineering things to encourage uptake and avoid scaring people away. They might be willing to carefully add more to "LSP 2.0" once people are comfortable with it.
icholy 9y ago
> LSP does not specify how to do general code generation. Don't quick fixes cover this?
- josteink 9y ago
  It covers specific known cases recognized by the particular language server implementations. It doesn't do general code-generation, which would imply that pretty much anything could be done. But I guess that's really nitpicking. Code fixes are mostly fine for me.
deleted 9y ago
[deleted]
skybrian 9y ago
As an alternate design, check out Dart's analysis server [1]. It's Dart-specific but has successfully been used with multiple editor and IDE plugins. [1] https://htmlpreview.github.io/?https://github.com/dart-lang/sdk/blob/master/pkg/analysis_server/doc/api.html https://htmlpreview.github.io/?https://github.com/dart-lang/...
squizza 9y ago
Are you sure you don't want to pool your efforts with intellij rust? Having one great ide beats having several kind of ok ones. They're releasing weekly improvements and it's the nicest IDE I have found to code rust in.
sddfd 9y ago
If I understand correctly, there are missing features in the RLS? Why not improve the RLS?
- sanxiyn 9y ago
  It's not only that there are missing features in RLS (specific implementation), but also that there are missing features in LSP (protocol). As long as RLS follows LSP, RLS can't be improved in certain ways.
cardiffspaceman 9y ago
Why is a server specification better than a DSO (library) specification that is shared between editors? At http://langserver.org http://langserver.org it is noted that plugins are created for N languages X M editors, and the USP of the language server idea is to reduce M to 1 through a shared server. I just don't see the force that drives us towards a server instead of a library.
- Matthias247 9y ago
  Because depending on your programming language it can be anywhere from annoying up to impossible to load and utilize a shared library. IO (and especially HTTP) however are easily available in most environments. Also languages servers might often be written in the language-that-they-care-for themselves (e.g. the C# language server in C#), because that's the environment where most of the tooling for that language is available. If you need everything in C then you need to rewrite the tooling. Or you just build a C library which then internally does RPC to the non-C process, but in this case you made the language-service an implementation detail for one language.
- coldtea 9y ago
  >I just don't see the force that drives us towards a server instead of a library. Obviously because the server can just be written ONCE, in some language (and preferably the targeting language it covers) and be done with it, whereas a library would always need bindings to be used by different editors that are themselves written in different languages (C, C++, Java, Javascript, etc).
- MaulingMonkey 9y ago
  Visual Studio crashing or hanging because of a bug in parsers or plugins is always annoying.
OJFord 9y ago
> Now, whether I would personally use [the language server protocol] in Vim is a whole different story. Anyone writing rust in vim would presumably use rust-lang/rust.vim - which I don't think uses RLS - is that all that's meant here, or is there a more general reason not to use LSP in vim, regardless of language?
- steveklabnik 9y ago
  It does not currently use RLS, but it also doesn't have any comparable features, really.
coldtea 9y ago
>LSP is incomplete The protocol currently only specifies very basic functionality: autocompletions, quick fixes, semantic highlighting, finding uses, formatting and that just about covers it. This is nothing compared to things a good IDE can do. Specifically when it comes to code generation, I think that's where IDEs truly shine, but at the moment LSP does not specify how to do general code generation. Yeah, get your thing to the point that it does even these, and we can talk about enhancing LSP later...
- SolarNet 9y ago
  I think the point is that the OP has been contracted by Google for 3 months to provide those things. The IDE in question is fully featured, hence why in three days the OP was able to fully implement the features RLS provided. The OP is now at the point where they have to do their how thing. Why so hostile?
pjmlp 9y ago
As reference, the Energize Programming System for C++ from Lucid as they pivoted from Lisp Machines, did make use of a language repository known as Cadillac. https://www.semanticscholar.org/paper/Foundation-for-a-C-Programming-Environment-Gabriel-Bourbaki/50048b7bbf93b7980a6947e48efda629c405e296 https://www.semanticscholar.org/paper/Foundation-for-a-C-Pro...
- lispm 9y ago
  Lucid never developed for Lisp Machines. Lucid's business was Common Lisp for Unix.
  - pjmlp 9y ago
    Thanks for the correction, you are of course right.
saghm 9y ago
If I had read this a week ago, I probably would have dismissed this due to having already had a better experience using the RLS with VS Code than any other Rust editing experience so far. However, quite recently I started trying out using Intellij with the Rust plugin, and I've noticed all sorts of little things that make the editing experience better that weren't present when using the RLS; for instance, if I originally importing something like `use foo::Bar;` and I want to import another item from `foo`, I can add an open brace before the "B" and start getting autocomplete when I being typing even before putting the comma delimiter and closing brace. I'm not familiar enough with the LSP and general implementation of editor plugins, but it's fairly easy for me to believe that implementing the above feature in Intellij using a custom plugin more fully integrated with their existing completion engine than eschewing the tools the IDE already had and building something independent of them.
sly010 9y ago
What's people's thoughts on learning languages with basic editors first before switching to IDEs? Especially with scala/sbt I feel like no mortal is able to contribute to a project unless they use the same version of the same IDE (which then effectively becomes a hard dependency of the entire project). Should I give in and learn to bundle the editor with the projects? This feels wrong. Programming is not CAD. Or is it?
- epage 9y ago
  > What's people's thoughts on learning languages with basic editors first before switching to IDEs? When I started learning Rust 6 months ago, I asked myself this question. I chose to use rustfmt so I could spend more time worrying about what my code was doing as I modified it and less time over what any Rust style guide might say. I chose to not use anything else though. My focus was on learning the language and not getting a project done. With that in mind, I wanted the learning process of digging through the documentation to become familiar with what is available rather than just doing "auto complete programming".

139 points by lkurusa 9y ago

Worth mentioning the Google Kythe project [0] which is designed around the idea of providing a graph of entities appearing in the code to build tools which work with code. I always thought that it would be good if the Google Kythe and the Language server protocol projects could merge, i.e. if the Language server protocol could expose a Kythe-like graph to editors. This seems to be along the lines of what OP suggests. [0] https://kythe.io/ https://kythe.io/

sqs 9y ago

Interesting post. I have worked a lot with LSP personally and professionally. I'm CEO of Sourcegraph, and we use LSP everywhere. (Go to https://sourcegraph.com/github.com/mholt/caddy/-/blob/caddyhttp/httpserver/https.go#L22:23$references https://sourcegraph.com/github.com/mholt/caddy/-/blob/caddyh... and open your browser's network inspector Web Socket list to see it live, for example.) I think LSP's success is a demonstration of the "less is more" principle. It is simple and designed only to satisfy an editor's needs. But unlike many similar attempts, LSP has a lot of traction; see https://github.com/Microsoft/language-server-protocol/wiki/Protocol-Implementations https://github.com/Microsoft/language-server-protocol/wiki/P... for a list of implementations (http://langserver.org/ http://langserver.org/ for more details). One of the reasons why this simplicity is necessary is that if a protocol even tries to represent structure or relationships across many different languages, it becomes harder and harder to add support for more languages. You can try to make it work for N languages, but the (N+1)th language always has some new language or syntax element that requires you to either change the specification (and update the N previous implementations) or deal with hacks (which goes against the whole reason to design the "one schema to rule them all"). A second reason is that even if you can make it work for all the languages you care about, it's very difficult to find enough language experts across all of those languages to build and maintain the implementations. It takes a lot of time to understand and communicate a complex schema and how the edge cases are handled across the various languages. And in the time it takes you to find and educate that team, you've already fallen behind new languages features being added or new popular languages coming on the scene. (Unless you are Google. Their Kythe project is an exception. :) LSP has explicitly rejected the goal of modeling complex language features, at least so far. Perhaps with the impressive initial traction they could add it on and succeed where many other attempts haven't yet. But I doubt it. (If you want to read more background about LSP, in addition to the Microsoft repo, you can check out some articles my cofounder wrote: https://text.sourcegraph.com/how-sourcegraph-scales-with-the-language-server-protocol-a4e8fd3fbae5 https://text.sourcegraph.com/how-sourcegraph-scales-with-the..., https://text.sourcegraph.com/part-2-how-sourcegraph-scales-with-the-language-server-protocol-73b86175f38b https://text.sourcegraph.com/part-2-how-sourcegraph-scales-w..., and https://text.sourcegraph.com/sourcegraph-code-intelligence-and-the-language-server-protocol-f2479fdd92ef. https://text.sourcegraph.com/sourcegraph-code-intelligence-a...)

xg15 9y ago
> It is simple and designed only to satisfy an editor's needs. But isn't that exactly the point of the post? The approach is nice for editors but unsuitable for IDEs. > You can try to make it work for N languages, but the (N+1)th language always has some new language or syntax element that requires you to either change the specification (and update the N previous implementations) or deal with hacks (which goes against the whole reason to design the "one schema to rule them all"). Seems to me LSP has the exact same problem with use-cases. By the description of the post, the goal seems to be to solve a fixed set of use-cases (highlighting, autocompletion, etc) for a flexible set of languages. If you want to add use-case n+1, you're faced with the same rewrite/hack conundrum. (Disclaimer: I don't have much expert with LSP)
- seren 9y ago
  Isn't LSP an application of Pareto Principle ? 80% of your needs are covered by small set of features.
- sqs 9y ago
  > But isn't that exactly the point of the post? The approach is nice for editors but unsuitable for IDEs. Yep. I'm just giving my thoughts on what it'd take to extend LSP so it addresses the issues raised in the blog post. I think it's much harder than it might seem. > Seems to me LSP has the exact same problem with use-cases. By the description of the post, the goal seems to be to solve a fixed set of use-cases (highlighting, autocompletion, etc) for a flexible set of languages. If you want to add use-case n+1, you're faced with the same rewrite/hack conundrum. The difference is between "What does an editor need?" and "What is a faithful representation of an element of code in a programming language?" The former is easy (and it's what LSP does) even to do for 50 languages; the latter is way harder than the casual reader would expect.
Analemma_ 9y ago
I agree with everything you've said, especially how LSP's simplicity has been key to its success. I do also agree with the author and would like a standard extensibility point, so that LSP could be the One True Way for every language to have full support in every IDE. I just want that to be done slowly and carefully, because it would be very easy to get wrong. If I had to guess, I'd say this might be coming in the future. Possibly Microsoft wanted to break out of its habit of overengineering things to encourage uptake and avoid scaring people away. They might be willing to carefully add more to "LSP 2.0" once people are comfortable with it.

icholy 9y ago

> LSP does not specify how to do general code generation. Don't quick fixes cover this?

josteink 9y ago
It covers specific known cases recognized by the particular language server implementations. It doesn't do general code-generation, which would imply that pretty much anything could be done. But I guess that's really nitpicking. Code fixes are mostly fine for me.

deleted 9y ago

[deleted]

skybrian 9y ago

As an alternate design, check out Dart's analysis server [1]. It's Dart-specific but has successfully been used with multiple editor and IDE plugins. [1] https://htmlpreview.github.io/?https://github.com/dart-lang/sdk/blob/master/pkg/analysis_server/doc/api.html https://htmlpreview.github.io/?https://github.com/dart-lang/...

squizza 9y ago

Are you sure you don't want to pool your efforts with intellij rust? Having one great ide beats having several kind of ok ones. They're releasing weekly improvements and it's the nicest IDE I have found to code rust in.

sddfd 9y ago

If I understand correctly, there are missing features in the RLS? Why not improve the RLS?

sanxiyn 9y ago
It's not only that there are missing features in RLS (specific implementation), but also that there are missing features in LSP (protocol). As long as RLS follows LSP, RLS can't be improved in certain ways.

cardiffspaceman 9y ago

Why is a server specification better than a DSO (library) specification that is shared between editors? At http://langserver.org http://langserver.org it is noted that plugins are created for N languages X M editors, and the USP of the language server idea is to reduce M to 1 through a shared server. I just don't see the force that drives us towards a server instead of a library.

Matthias247 9y ago
Because depending on your programming language it can be anywhere from annoying up to impossible to load and utilize a shared library. IO (and especially HTTP) however are easily available in most environments. Also languages servers might often be written in the language-that-they-care-for themselves (e.g. the C# language server in C#), because that's the environment where most of the tooling for that language is available. If you need everything in C then you need to rewrite the tooling. Or you just build a C library which then internally does RPC to the non-C process, but in this case you made the language-service an implementation detail for one language.
coldtea 9y ago
>I just don't see the force that drives us towards a server instead of a library. Obviously because the server can just be written ONCE, in some language (and preferably the targeting language it covers) and be done with it, whereas a library would always need bindings to be used by different editors that are themselves written in different languages (C, C++, Java, Javascript, etc).
MaulingMonkey 9y ago
Visual Studio crashing or hanging because of a bug in parsers or plugins is always annoying.

OJFord 9y ago

> Now, whether I would personally use [the language server protocol] in Vim is a whole different story. Anyone writing rust in vim would presumably use rust-lang/rust.vim - which I don't think uses RLS - is that all that's meant here, or is there a more general reason not to use LSP in vim, regardless of language?

steveklabnik 9y ago
It does not currently use RLS, but it also doesn't have any comparable features, really.

coldtea 9y ago

>LSP is incomplete The protocol currently only specifies very basic functionality: autocompletions, quick fixes, semantic highlighting, finding uses, formatting and that just about covers it. This is nothing compared to things a good IDE can do. Specifically when it comes to code generation, I think that's where IDEs truly shine, but at the moment LSP does not specify how to do general code generation. Yeah, get your thing to the point that it does even these, and we can talk about enhancing LSP later...

SolarNet 9y ago
I think the point is that the OP has been contracted by Google for 3 months to provide those things. The IDE in question is fully featured, hence why in three days the OP was able to fully implement the features RLS provided. The OP is now at the point where they have to do their how thing. Why so hostile?

pjmlp 9y ago

As reference, the Energize Programming System for C++ from Lucid as they pivoted from Lisp Machines, did make use of a language repository known as Cadillac. https://www.semanticscholar.org/paper/Foundation-for-a-C-Programming-Environment-Gabriel-Bourbaki/50048b7bbf93b7980a6947e48efda629c405e296 https://www.semanticscholar.org/paper/Foundation-for-a-C-Pro...

lispm 9y ago
Lucid never developed for Lisp Machines. Lucid's business was Common Lisp for Unix.
- pjmlp 9y ago
  Thanks for the correction, you are of course right.

saghm 9y ago

If I had read this a week ago, I probably would have dismissed this due to having already had a better experience using the RLS with VS Code than any other Rust editing experience so far. However, quite recently I started trying out using Intellij with the Rust plugin, and I've noticed all sorts of little things that make the editing experience better that weren't present when using the RLS; for instance, if I originally importing something like `use foo::Bar;` and I want to import another item from `foo`, I can add an open brace before the "B" and start getting autocomplete when I being typing even before putting the comma delimiter and closing brace. I'm not familiar enough with the LSP and general implementation of editor plugins, but it's fairly easy for me to believe that implementing the above feature in Intellij using a custom plugin more fully integrated with their existing completion engine than eschewing the tools the IDE already had and building something independent of them.

sly010 9y ago

What's people's thoughts on learning languages with basic editors first before switching to IDEs? Especially with scala/sbt I feel like no mortal is able to contribute to a project unless they use the same version of the same IDE (which then effectively becomes a hard dependency of the entire project). Should I give in and learn to bundle the editor with the projects? This feels wrong. Programming is not CAD. Or is it?

epage 9y ago
> What's people's thoughts on learning languages with basic editors first before switching to IDEs? When I started learning Rust 6 months ago, I asked myself this question. I chose to use rustfmt so I could spend more time worrying about what my code was doing as I modified it and less time over what any Rust style guide might say. I chose to not use anything else though. My focus was on learning the language and not getting a project done. With that in mind, I wanted the learning process of digging through the documentation to become familiar with what is available rather than just doing "auto complete programming".