17 ms·

Why LSP?

373 points by afdbcreid 5y ago

abdellah123 5y ago
It's worth noting that typescript/js support in VSCode doesn't come from an LSP. It's actually tangled with VSCode itself
salmo 5y ago
Ugh. This whole thing is exhausting because everything is so oversimplified (as is any conversation on the topic) and there isn’t a right answer. You can’t compare the architecture of VSCode, vim, emacs, VS, IntelliJ, etc. and come to any conclusion. They are completely different tools, built at different times, with different goals, by different (dis)organizations. Emacs was a bunch of primitives that were wrapped in Lisp, and you could do anything. It was a day 1 goal. Ignoring the things that evolved to become normal in the Lisp on top is weird. It was extensible by design and completely open ended, with packages evolving to being part of the expected toolchain. Vim took vi and bolted on some scripting interface that kept being pushed, adding extensibility by demand. Now we have neovim embedding lua which is more emacs-like, and driving vim features as well. But it got to add higher level “primitives” because it’s a new implementation. IDEs like VS, Borland X, Sun Studio, etc. were purpose built tools for a specific toolchain package. Only after the fall of traditional IDE/RAD environments did VS start trying to be an everything IDE. Eclipse and Netbeans started as Java IDEs in Java, were all focused on architecture, and enabled plugins. People started making plugins to support all the languages (and then totally break the IDE by installing the wrong combo). As eclipse, well eclipsed, NetBeans you started seeing Eclipse “distros” of canned plug-in combos to not implode. I know folks with an eclipse install per codebase they work on. IntelliJ got the architecture part from Eclipse, but stayed very hand curated like the old RADs and IDEs. You can install all the things in IntelliJ Ultimate or buy the more focused language specific ones (PyCharm, CLion, etc). The plugins are less flaky because they’re predominantly 1st party. The language specific ones make each toolchain ergonomic like a traditional IDE. VSCode got the benefit of being the newest. Strip down below an “IDE” but above a text editor exposing primitives. Use web tech so the kids can make stuff because Java isn’t the lingua franca it once was. It could look back for a better sweet spot. So whatever. Use what you find productive and comfortable. I wander between probably 10 languages. I tend to have an editor and terminal side by side. It’s comfortable for me. I prefer the editor+ experience more than the integrated IDE one. I also am generally mouse-adverse. But that’s me and I’m not “right.” I like my neovim acting like VSCode more than VSCode acting like vim. But it’s because it matches what my hands do already. When I do !!fmt or whatever and it doesn’t work, it takes me out of my flow. And when editing remotely, I still get vi/vim just without bells and whistles so my hands still work. Many just never have that use case. That comes at a cost of maintaining a crazy config for me vs clicking for plugins. It used to be 10-15 vim plugins. Now it’s more LSP + a few. But I revision control it and revisit every 6 months or so. That trade off’s worth it to me, maybe not to you. Some folks still love their full IDEs with tighter integration with a toolchain and language-specific features ergonomically placed. They likely use Java/C#/C++ and have to dabble in html+js at times. Use and make what you want. Make an LSP or don’t. Integrate them into something or don’t. All of the experiences described in the article still exist. No one has to use any of them. Follow your heart (or I guess your hands). Saying one is best is like saying some kind of shoes are best for everyone. They’re tools, but ones we live in. Wear what’s comfortable. In the words of a song by Sheryl Crow that I don’t particularly like “If it makes you happy, then why the hell are you so sad?”
jez 5y ago
I think this post hits the nail on the head. Before LSP, you could google for “$language Vim support” and find some results but more often than not, you were going to get nothing. Once LSP gained popularity, the question shifted to “why doesn’t $language have a language server yet?” which set an easy goal post for languages and compilers to go from 0 to 1 in terms of IDE support.
- wodenokoto 5y ago
  But isn’t that exactly what he starts of arguing against? The first chart shows that before lsp you needed special language support for each language for each editor, which is unfeasable. After lsp you just need lsp support for each language. The author then goes on to say “this is wrong”
  - aaron_m04 5y ago
    Right. My takeaway after reading the whole post was "so it really _is_ M * N after all".
  - ubertaco 5y ago
    I don't think the author is quite saying "this is wrong". It seems like he's more saying "this is an oversimplification, and the reality is a more-nuanced version of that statement" -- specifically, that rather than "now languages just build a language server, and editors just accept LSP", it's that "now languages just build a language server, and editors just accept LSP, plus a bit of LSP-specific configuration to point the editor at the particular LSP for a context." Breaking things down a bit further, there are three scenarios discussed in the article: 1. M × N -- every language needs a from-scratch, bespoke plugin for every editor 2. M + N -- the "standard explanation" for LSP, where you build one LSP client implementation per editor, one LSP server implementation per language, and you're done 3. The reality of what exists today: one LSP server implementation per language, one LSP client implementation per editor, and a tiny bit of (usually-end-user-supplied) configuration in your editor to glue those two sides together. The key difference between scenario 1 and scenario 3 is that scenario 1 requires someone with _very deep_ knowledge of a language's structure and semantics to implement a fully-fledged plugin for _every_ editor, usually requiring deep knowledge of that editor (or vice-versa), where scenario 3 allows both ends to wrap that deep knowledge up behind a standardized interface so that any shmuck can write like <100 lines of lua to glue the two ends together (speaking from my own experience as an "any shmuck" using neovim)
kqr 5y ago
> Notably, the two-sided market problem was solved by Microsoft, who were a vendor of both languages (C# and TypeScript) and editors (VS Code and Visual Studio), and who were generally loosing in the IDE space to a competitor (JetBrains). Talk about commoditising your complement! ---- Unrelated: how are LSP implementations and Emacs support generally these days? When I last tried 2–3 years ago the Emacs-specific plugins for various languages were still more reliable and predictable than their LSP counterparts, but has this changed a lot? If so, I might be inclined to try again.
- vim-guru 5y ago
  It's catching up. You can use them in tandem though and ignore the features that overlap by configuration. I use Eglot with a bunch of different languages. Clojure is the only one where I still use other packages for dealing with similar things.
  - ashton314 5y ago
    Eh, gonna be really hard to be Emacs' prowess in the land of Lisps. This rings pretty true to me: they all work well together.
  - yewenjie 5y ago
    Can you please comment on your experience with Eglot? Is it better/ faster?
- ashton314 5y ago
  LSP support with lsp-mode is fantastic with at least Rust. I use Elixir at my day job and while it does a good job of doing 98% of what I want (go to definition, find references (iffy; Elixir is difficult to analyze like this), code lenses, autocomplete, etc.) there are a few rough patches. This is likely due to shortcomings with the lang server implementation, not the lsp-mode library itself. Like I said, editing Rust is awesome in Emacs; saw someone working on Rust in VS Code and didn't notice anything that Emacs couldn't match. See also lsp-ui; I don't turn this on personally, but it does give you some significant bling.
vim-guru 5y ago
I have feelings about the last paragraph there. Most protocols are limiting by nature, so although LSP has made wonders to a lot of languages for a lot of editors out there; I'm stuck thinking that it might have stifled progress for some as well. I don't have a proposed solution or anything, just pondering if we could have done better.
- OtomotO 5y ago
  We could always have done better, but the nice thing is, that most of the time, we can. Will it take time and energy? Sure! Such is life. At least it's not as bad as e.g. JavaScript.
- rob74 5y ago
  Well (Ok, I'm partly contradicting the article here), most of the things LSP supports (autocomplete, jump to definition, ...) have been pretty standard features of more advanced IDEs (the original Visual Studio, the various JetBrains IDEs, even good ol' Delphi) for decades now. So the "problem domain" is pretty well defined already, LSP is just standardizing the interfaces.
  - dmitriid 5y ago
    Advanced IDEs are advanced because they provide so much more beyond basic jump to definition and autocomplete (which, again, can be very advanced and be context-aware, provide code shortcuts etc.). It's good that tooling has improved across the board for many languages with LSP, but there's so much more still.
- chii 5y ago
  > it might have stifled progress for some as well which language got stifled as a result of the existence of LSP? Would that language still have stifled anyway without LSP?
  - vim-guru 5y ago
    I haven't looked much into the implementation details of LSP, so I was not aware of custom-commands upon writing my comment. Maybe it hasn't stifled at all, but efforts have definitely been split, as prior art is still maintained years after LSP's launch
p00f 5y ago
> Contrast this with Emacs and Vim. They just don’t have proper completion as an editor’s extension point. Rather, they expose low-level cursor and screen manipulation API, and then people implement competing completion frameworks on top of that! This is just wrong, https://vimhelp.org/insert.txt.html#compl-omni https://vimhelp.org/insert.txt.html#compl-omni The first occurrence of "omni" I can find in the git log is this commit from 2012: https://github.com/vim/vim/commit/5d3a8038b6a59e6f1b219f27ec77d6067e643ac1 https://github.com/vim/vim/commit/5d3a8038b6a59e6f1b219f27ec... which means it's even older
- the-smug-one 5y ago
  I want to further bang about on this: This is a plain false statement. Emacs's completion-at-point was (C-h C-f) introduced at 23.2, in May of 2010. company-mode has commits going back to 2009.
  - monsieurbanana 5y ago
    That's his point, people use company mode because Emacs doesn't have built-in "modern" completion UI. Completion at point uses the minibuffer, right? If so I wouldn't call it a real alternative, specially without minibuffers plug-ins like ivy and co. I don't think the Emacs situation is bad at all, but I agree 100% that it needs third party plug-ins to be anything close to what people expect of an editor in 2022.
    - the-smug-one 5y ago
      His point is poorly expressed in that case. Let's look at the quote again. >Contrast this with Emacs and Vim. They just don’t have proper completion as an editor’s extension point. Rather, they expose low-level cursor and screen manipulation API, and then people implement competing completion frameworks on top of that! People don't implement competing completion frameworks on top of "low-level cursor and screen manipulation API". There are more high-level APIs included in Emacs, such as completion-at-point. To quote company-mode's website: >The CAPF back-end provides a bridge to the standard completion-at-point-functions facility, and thus works with any major mode that defines a proper completion function. To me the author misrepresents what Emacs has built-in and what package writers use. Technically company-mode isn't third party, it's part of GNU Emacs. It needs to be downloaded, yes, but it's not third party. Emacs needs a package recommendation engine (I know VS Code has this, awesome!).
v-yadli 5y ago
One particular problem of M+N is that it requires LSP to cover all the wanted features provided by server and supported by client. This is not always the case for existing servers. Semantic highlighting wasn't in the protocol until 3.0 (iirc), and many servers have extra non-standard endpoints. So the real world situation is that there's a core feature set allowing M+N, and additionally, custom endpoints and features requiring MN. Nevertheless the non-standard MN endpoints at least follow the "shape" of LSP -- same communication channel and message format, and that makes MN much easier than before already! As the core protocol evolves, more MN features will be decoupled to M+N.
- jez 5y ago
  This is definitely true but like you mention because there’s a common communication channel and message format, these language-specific RPC endpoints exist whereas before LSP, language servers didn’t have “custom” features because there weren’t IDE-first language servers at all. There are already a more than a couple LSP features I wish existed, but in the mean time it’s good enough to have the (typed! and using rich data structures!) RPC endpoints exist for enterprising users to wire up to their IDE’s presentation layer.
- orlp 5y ago
  I also would like to note that this could provide a chilling effect for programming language design in the future. If LSP's become ultra-mainstream it might carve out a path of least resistance that prevent future innovation and adoption of language features incompatible with the protocol. Although I must admit I have no idea what such features could look like.
  - oblio 5y ago
    Standardization ossifies some things but also allows building many things on top of a stable base. Innovation could just happen at a higher level.
    - TuringTest 5y ago
      > Innovation could just happen at a higher level. That, or retro-backing into the protocol novel features required by those experimental languages. In my experience, industry-wide ossification and chilling effects appear not because the environment makes it difficult to build new designs, but because it makes it difficult to think outside the current paradigm and imagine novel features. Once those innovative ideas are out of the box and many people understand their need, the frameworks needed to support them are created. I know because right now we're experiencing such a paradigm shift in terms of note-taking and knowledge building (abandoning WYSIWYG word processors in favor of networked-thinking bidirectional-linked graphs of notes), which IMHO sooner or later will extend to programming tools as well.
skybrian 5y ago
The problem with this argument is that it's based on only a few of the most popular IDE's that the author happens to remember, but there were a bunch of others. For Java, you might remember NetBeans or VisualAge. There's a long tail. Some of these IDE's supported more than one language. For example, VisualAge also supported C++ and SmallTalk. But each IDE supported a different set of languages with implementations of varying quality, they each had their own complicated plugin framework, and that's the N:M problem. Even if it were just IntelliJ and Eclipse, that's still implementing support for a new language twice from the ground up, and you'll be doing it in Java both times, which is a lot of work for a team working on a new programming language. You can look at the Wikipedia page to get a sense of how many different IDE's and languages there are out there: https://en.wikipedia.org/wiki/Comparison_of_integrated_development_environments https://en.wikipedia.org/wiki/Comparison_of_integrated_devel...
- zmmmmm 5y ago
  > you'll be doing it in Java both times I honestly think this is highly underrated as a reason. Teams working on a language want to program in that language. It doesn't really matter if it's complex, or they have to repeat the work for multiple IDEs. They love doing that. They will actually compete to do that (look how many posts on HN these days are just "$MUNDANE_TOOL written in Rust"). The main thing LSP did is it moved the boundary of the implementation to the other side of the language barrier.
hsn915 5y ago
I tried a few language plugins for VSCode that were all based on LSP, and they were all broken. So for me, the question is wrong. It's not "Why is LSP so great?". The question should be "Why are people so hyped about tech that's based on questionable assumptions and that hasn't proved itself?" The reason VSCode invented LSP is that different programming languages don't have a reliable way to intercommunicate (other than as separate processes communicating over a socket).
- adwn 5y ago
  > The reason VSCode invented LSP is that different programming languages don't have a reliable way to intercommunicate (other than as separate processes communicating over a socket). The processes-communicating-via-IPC architecture is way underrated for plugins. It's much easier to separate the plugin from the host, and to cleanly stop or restart the plugin. For example, sometimes (rarely) rust-analyzer hits a bug and starts consuming 100% CPU. When that happens, I just kill that process and VS Code transparently restarts rust-analyzer; I don't have to restart the IDE, and all other plugins are unaffected as well. That would be practically impossible if rust-analyzer were a library instead of an independent process.
  - zozbot234 5y ago
    That's because separate processes are not "plugins" at all, but part of an outside toolset that may or may not be supported by a thin "plugin"-like IDE component. The patterns involved are completely different; the separate-processes pattern was popularized by *nix, whereas plugins were always more common in Windows and Mac.
    - adwn 5y ago
      The concept of a "plugin" isn't restricted to dynamically linking shared libraries, but rather defined by the concept of having an independent host application, which can load additional functionality at runtime. It doesn't matter whether this is achieved through shared libraries, interpreted code, or separate processes.
  - 5y ago
IshKebab 5y ago
> If the standard theory were correct, then, before LSP, we would have lived in a world where some languages has superb IDE support in some editors. For example, IntelliJ would have been great at Java, Emacs at C++, Vim at C#, etc. My recollection of that time is quite different Interesting theory but my recollection of that time is pretty much exactly that. Lots of IDEs with great Java support (IntelliJ, Netbeans, Eclipse), a very small number with good CJJ support (Visual Studio, Qt Creator, maybe KDevelop or Code::Blocks if you're generous with "good"), and not much else. How many IDEs had great Delphi support? As far as I know exactly one - the Delphi IDE. Ok there were a couple of exceptions - Eclipse had decent C++ support IIRC and IntelliJ sort of had support for loads of languages, but that was a business whose main product is IDEs. I think the alternative suggestion is also true. They're both good reasons for the success of LSP.
junon 5y ago
I recently switched my neovim config over to LSP from ncm. My experience is now much buggier, slower, features work some days and not others, syntax highlighting just stops working randomly, trash files get spewn all over my repositories that I have to .git/info/exclude, etc. Not sure what the Right Direction is when it comes to text editing but this isn't it.
- davidatbu 5y ago
  Assuming you're talking about this ncm[0], are you aware that ncm is a " completion framework for neovim", which is different from an LSP? ncm is either an LSP client itself, or, it is talking to neovim's internal LSP client, to get completions from LSPs. It's also important to note that LSPs almost always provide you more than auto completion (for example, go to definition, go to implementation, find references). Do let me know if you have more questions, neovim and LSPs are my "daily driver" as a dev, so to speak. [0] https://github.com/ncm2/ncm2 https://github.com/ncm2/ncm2
  - junon 5y ago
    Yep I'm aware of what it is, I used it for years and years, thanks. NCM ancillary plugins that interacted with e.g. libclang were more what I was referring to, but I'd hoped the average reader could deduce that. Those features of LSP I do not use, namely because LSP is a buggy mess. Some files have wild syntax errors reported for them that don't exist when compiling, but not others. Half the time, the auto-completion simply doesn't do anything. Nothing pops up, nothing auto-completes, etc. And now, just a few weeks after switching over, without changing a thing about my config, the arrow keys for selection have completely stopped working inexplicably.
    - davidatbu 4y ago
      So, synthesizing your reply here with your reply to @mjlbach, it seems to me like you are attributing to the whole LSP ecosystem (by saying things like "LSP is a buggy mess") (1) problems with treesitter, and (2) problems with clangd, one LSP server implementation. I feel like if your comments' utility to the conversation would be greatly improved by prefacing them with something like "in my experience with clangd, treesitter and neovim". I say this because my experience with LSPs is the polar opposite of yours[0]. While I believe there's value to sharing our personal experiences, I think we have to be careful not to present our personal experience (especially with (neo)vim setups, which tend to vary wildly from one to the next) as representative of the whole ecosystem. [0] I also have nothing but praise for most of the LSPs I've used (clangd, typescript-language-server, rust-analyzer, clangd, pyright, css-language-server, sumneko/lua-language-server, of the top of my head).
pjmlp 5y ago
> To get a decent IDE support, you either used a language supported by JetBrains (IntelliJ or ReSharper) or. VS, Eclipse, Netbeans, Delphi, C++ Builder, KDevelop, QtCreator,... > Notably, the two-sided market problem was solved by Microsoft, who were a vendor of both languages (C# and TypeScript) and editors (VS Code and Visual Studio), and who were generally loosing in the IDE space to a competitor (JetBrains) Nope, the IDE space on Windows has always been owned by Microsoft since Borland dropped the ball with their management crisis where to go next. VSCode and LSP were created out of Monaco and Erich Gamma's stewardship, yep one of the four on the patterns book, and original Eclipse team. "VS Code an Overnight Success… 10 years in the making" https://www.youtube.com/watch?v=hilznKQij7A https://www.youtube.com/watch?v=hilznKQij7A
- Ygg2 5y ago
  > VS, Eclipse, Netbeans, Delphi, C++ Builder, KDevelop, QtCreator,... Speak for yourself. Editing C# is much better in Rider than VStudio. Eclipse was a competitior, but I dread having to use it for anything nowadays. CLion just doesn't work correctly, yet. For Rust VS Code is better, but only because of awesome rust-analyzer plugin.
  - pjmlp 5y ago
    Not keeping up with the times with VS? Eclipse is still my get to go, InteliJ wants me to buy Clion + InteliJ licenses for features Eclipse does out of the box. VSCode isn't an IDE, plus Rust is the new kid in town.
    - Ygg2 5y ago
      > VSCode isn't an IDE Add enough plugins and it will be. Rust being new is one part, other part is that Jetbrains really insists on their plugins vs LSP. I get Rust errors in Clion on programs that run normally. Bugs that rust-analyzer/cargo don't report.
      - kuschku 5y ago
        Bolting 200 things to a bicycle doesn't make it a truck, and no amount of addons can turn VS Code into an IDE. The I means Integrated, and VS Code with plugins and LSPs is anything but.
Shadonototra 5y ago
LSP is a great thing, one of the very few things i appreciate from microsoft
lf-non 5y ago
One thing I have difficulty understanding is that given MS is (was?) the primary proponent of LSP, and the spec was originally written with TS in mind, and VSCode has first class support for LSP, why do community projects like typescript-language-server (along with a bunch of others like the one by sourcegraph) need to exist ? Why can't arbitrary LSP clients connect directly to tsserver (maintained by MS) ? [1] https://github.com/typescript-language-server/typescript-language-server https://github.com/typescript-language-server/typescript-lan...
- yanis_t 5y ago
  I think the problem is that the tsserver for historical reasons even though is quite similar, still does NOT implement the LSP protocol. That's why you need another level here.
- richdougherty 5y ago
  > Why can't arbitrary LSP clients connect directly to tsserver (maintained by MS) ? Can you explain a bit more about why they can't connect?
  - norman784 5y ago
    tsserver isn't LSP compliant, iirc they started with tsserver way before LSP was out, but since then didn't migrated to LSP. I think they didn't migrated because: - It's a big selling point for vscode. - It might hard to migrate what they have to LSP, they might implement a lot of non LSP standards. - Migrate to LSP could possibly delay new features because of the rewrite in tsserver.
  - lf-non 5y ago
    Because tsserver (maintained by MS) is not an LSP server. There is an open issue [1] that has been open since 2020. [1] https://github.com/microsoft/TypeScript/issues/39459 https://github.com/microsoft/TypeScript/issues/39459
- paskozdilar 5y ago
  > Why can't arbitrary LSP clients connect directly to tsserver (maintained by MS) ? As far as I'm informed, they can. For example, NeoVim language client has a pretty wide range of officially supported servers [0], including tsserver. [0] https://github.com/neovim/nvim-lspconfig/blob/master/doc/server_configurations.md https://github.com/neovim/nvim-lspconfig/blob/master/doc/ser...
DrBazza 5y ago
It's not IDEs in 2022 that use LSP, it's editors. As a grey-beard, M x N wasn't "small". From memory, notepad, notepad++, scite, vim, emacs, netbeans, eclipse, VS studio, IDEA, sublime text, bbedit, plus a number of linux only ones. Languages, VB, C++, Pascal/Delphi, Fortran, Java, C# (from 1998/9?). The problem was tight coupling of languages to IDEs, in particular VB, and C++ to VS, and Java to Eclipse.
davidatbu 5y ago
Tangentially related: If a talented folks out there could please help out with (to the best of my knowledge) the only LSP for Kotlin out there[0], that would be amazing! I'm really attracted to Kotlin as a language[1], but, last I checked, IntelliJ is the only way to get great tooling for Kotlin, which is a show stopper for me since I've invested a lot into my neovim config. [0] https://github.com/fwcd/kotlin-language-server/projects/1 https://github.com/fwcd/kotlin-language-server/projects/1 [1] What attracts me to Kotlin is that it is actually statically typed (ruling out Python/JS/TS), with a relatively expressive type system (ruling out Golang), great type inference, not verbose (ruling out Java), not too involved with low level details (ruling out Rust), not too steep of a learning curve (ruling out Rust again), and a great cross-platform community (ruling out Swift).
- ZeroGravitas 5y ago
  I guess you're aware but the Jetbrains folks (who do Kotlin and the IDE range) have been a bit cold towards LSP since it's basically commodifying their USP for their IDEs (rich, language specific integration) which is why kotlin LSP is a bit of a no man's land.
  - Tmpod 5y ago
    Yeah, that's true. I can see their reasoning, but I'd hope they wouldn't be so cold.
  - zozbot234 5y ago
    Their USP is native, efficient tooling for something that's usually implemented in managed platforms like Electron/JS or the JVM. They would in fact benefit from a rich LSP ecosystem, since their tooling would be more easily applicable across languages.
    - ZeroGravitas 5y ago
      I don't use them, but aren't their IDEs still JVM?
codeflo 5y ago
This is a “correction” of a strawman only: The left diagram is exactly what would have been necessary to achieve the same result. Since that’s clearly infeasible, most of the arrows didn’t actually exist, and never would have. That was always the point. It’s precisely because the left picture is impractical that something like the LSP was necessary. Somehow, the author misunderstood the original argument, dismissed it, independently discovered the actual argument, and now claims they’re the one who found the true reason for LSP, when in reality, it’s just the standard argument.
- afdbcreid 5y ago
  No. The point is that while we need analyzer for each language, the connections between the editors & the analyzers have no problems to be N*M because they're very small. So we don't really need a protocol like LSP. But the rise of this protocol caused people to write independent, non-per-editor analyzers, and this is indeed necessary. I completely agree with that.
  - codeflo 5y ago
    Hmm, maybe I misunderstood the author’s point slightly, but I find what you suggest the argument was even less convincing. Here’s the history: Every editor has a language abstraction already. And some language analyzers had an “editor abstraction”, perhaps notably Roslyn for C# with its programmable refactoring rules that worked in multiple editors. None of that was new. But that’s still N*M if you do the math, so that graph was never completely filled — for the obvious reason that this would have been practically impossible. No mystery about it.
    - mgdlbp 5y ago
      And considering how VB.NET was incorporated first-class besides C#, Roslyn seems rather ahead of its time, with both language abstraction at the compiler level and language-agnostic analysers based on semantic info. I still can't get over the awesomeness of being able to just throw together a fully supported custom analyser from a template—unique to Roslyn to this day, AFAIK.
numToStr 5y ago
> And try to write more about how to implement language servers — I feel like there’s still not enough knowledge about this out there. Yes, please! There is next to nothing about "How to write a language server using LSP". All I can find is "how to write a language server for VSCode". - Written by a neovim user
- ZeroGravitas 5y ago
  I think the idea is that those should be the same though. Like the move to standardise web extensions so a 'chrome' extension works in obscure browsers too.
- matklad 5y ago
  Well, I think it’s a bit more than that already :-) https://m.youtube.com/playlist?list=PLhb66M_x9UmrqXhQuIpWC5VgTdrGxMx3y https://m.youtube.com/playlist?list=PLhb66M_x9UmrqXhQuIpWC5V... https://rust-analyzer.github.io/blog/2020/07/20/three-architectures-for-responsive-ide.html https://rust-analyzer.github.io/blog/2020/07/20/three-archit...
yanis_t 5y ago
LSP is what made me make a full switch to Vim[0]. Neovim even made it part of its API out-of-the-box, and that's a perfectly reasonable move given how much benefit it brings. [0] https://www.vimfromscratch.com/articles/vim-and-language-server-protocol https://www.vimfromscratch.com/articles/vim-and-language-ser...
nvarsj 5y ago
Great article! LSP brought me back to Emacs. Real IDE support with best in class editing abilities is bliss. Microsoft really nailed it with LSP (albeit, I agree with the caveats the implementation has some warts). Jetbrains must be sweating - more and more of my colleagues and coworkers are using VSCode these days. I think Jetbrains should seriously consider LSPifying their backends. If they don't, they risk someone else building a better JVM LSP and then they are going to lose a lot of market share. Plus it gives them the opportunity to shape the LSP spec. I'd happily pay Jetbrains right now a sub just for the backend if I can use it with my editor of choice.
- kstrauser 5y ago
  Same for me. I used Emacs for many years, but other editors (especially VSCode) were getting better language support and it was starting to feel long in the tooth. With LSP support, it’s amazing again. I’m happy to be back in my favorite environment.
- zmmmmm 5y ago
  Eclipse already did (integrate JVM LSP support). It kinda sucks still in lots of ways, but at the same time, it is pretty surprising how well it works and is totally usable, eg: for working on React. If more work is done on it then it's definitely going to start change things a bit (in part also because Eclipse itself has improved dramatically in the last few years).
- Shadonototra 5y ago
  intellij isn't popular just for code completion people use intellij because of the various plugin/features/debugging for all the jvm ecosystem/frameworks https://www.jetbrains.com/idea/features/#jvm-frameworks https://www.jetbrains.com/idea/features/#jvm-frameworks similar to how Rider became immensely popular (1st class Unity Engine support) and now Rider C++ support for .sln solution and 1st class Unreal Engine support intellij's problem is their plugin API and development cycle, for a while you needed to restart the whole IDE whenever you wanted to recompile your plugin, as a result, the 3rd-party plugin ecosystem is very poor hopefully they fix all the annoying things with Fleet https://www.jetbrains.com/fleet/ https://www.jetbrains.com/fleet/ they went with JVM so it gives them a poor start already (no comparatif benefit over VSCode and electron, same bloatshit, but they can eat some market share by providing a better out of the box experience)
kqr 5y ago
I might be a bit contrarian here, but > In particular, add assist/code action/(bulb) as a first-class UX concept already. It’s the single most important UX innovation of IDEs, which is very old at this point. Its outright ridiculous that this isn’t a standard interface across all editors. In the grand scheme of things, this particular super-local tactical optimisation seems like entirely the wrong problem to focus on with all the inefficiencies that come with badly designed software. Whether I write public double Hours { get { return _seconds / 3600; } } or public double Hours => _seconds / 3600; is really not going to make a big difference to the quality of my software. Is this where we want to focus our attention? The day I get a little pop-up that says - "are you sure you need to invent your own time abstraction here?", or - "this is a derived value that should only be used in presentation and not business logic or persistance", or - "the customer has not been using the feature that this code is relevant for in six months, are you sure you want to spend time on this?" then we can start to rave about the single most important UX innovations.
- turtleyacht 5y ago
  Interesting! So the problem of LSPs (or code tools like IDEs) approaches automating the evaluation of "one-level up" abstractions similar to the feedback during code reviews.
oxff 5y ago
Rust-Analyzer on VS-Code is probably the best development experience I've ever had. Makes me want to gouge my eyes out whenever I have to use something else.
zabil 5y ago
I think LSP is great. I've used it in one my side projects to quickly add support for VS Code. We took a more direct approach. Instead of having a separate LSP server our main program runs on LSP server mode. This made a lot of things like distribution and interaction with the core functionality simpler. LSP also solves a big pain by making it easy to build tooling when teams choose to implement an external DSL's.
jmull 5y ago
I don’t know why the author spends so much time “debunking” the M * N argument for LSP. It seems a minor side point. It’s a bit incoherent anyway. The author’s argument seems to be that if you have language-provided pluggable language servers and editors that can host these language server plugins appropriately, then the M * N shrinks down to a small problem, without the help of LSP. Well, OK, but that’s actually the original LSP M * N argument, except with LSP replaced with something just like LSP.
- matklad 5y ago
  The point is that you don’t need common protocol. A bunch of language-specific protocols would have worked! It’s interesting to ask why that didn’t happen.
  - codeflo 5y ago
    But in fact that did happen. Several languages started to implement “editor neutral” code analysis and refactoring. That worked, for the most popular editor/language combinations, but not as a general solution because of N*M.
    - pas 5y ago
      Can you please give a few examples?
      - nerdponx 5y ago
        OCaml (Merlin), Common Lisp (SWANK), Clojure (nREPL), Python (Jedi), even Idris 2 (its IDE Mode).
  - bzxcvbn 5y ago
    Is it really interesting? Every editor would have had to support a bunch of different protocols, instead of just one. That's back to square one.
  - nerdponx 5y ago
mekster 5y ago
I've just dug through the Neovim ecosystem trying to upgrade my 5 years old .vimrc and I feel it has come a long way since LSP became a thing. It can now actually act like real IDE as the editor knows about the code pretty well. https://github.com/folke/trouble.nvim https://github.com/folke/trouble.nvim https://github.com/tanvirtin/vgit.nvim https://github.com/tanvirtin/vgit.nvim https://github.com/kosayoda/nvim-lightbulb https://github.com/kosayoda/nvim-lightbulb Some features look pretty close to what you get in JetBrains. I can see some parts are kind of rough until you Google enough to fix/customize to your needs but if I take enough time, I might as well feel like its a decent replacement for a real IDE. For a starter, AstroNvim helped me figure out what the juicy plugins are these days. It seems NvChad and LunarVim are preconfigured competitors to it. I wonder if JetBrains one day gets left behind for not using LSP ecosystem but perhaps Fleet might implement it.
- KrishnaShripad 5y ago
  Can't recommend AstroNvim enough as a starting point to Nvim. It is the perfect bootstrap needed (though I would disable their integration with the scroll plugin — Neoscroll. Prefer the native scroll instead).
  - mekster 5y ago
    Exactly what I did. Not sure if people like that slow scrolling.
david_draco 5y ago
There are N programming languages and M libraries (e.g., dates, file-formats, databases, data analysis method X). Currently, each language (Go, Rust, Python, Julia), needs to reimplement each library from scratch, which is a huge effort and delays language readiness for users. Maybe building another programming language can be accelerated by deriving its M libraries from another, using some kind of abstract libraries or translation methods, through a service similar to LSP.
- matklad 5y ago
  That’s what deno is doing IIRC! They rely heavily on design work that went into web standards and Go standard library, and harness open-source community to deliver implementation.
  - oblio 5y ago
    That's in no way what he's suggesting and it's something every language in creation has done: getting inspiration from existing languages. He's saying about actual working code, something akin to universal libraries you could use from any language. Or maybe some auto-translation/hooking up of libraries from another language, through some sort of "library protocol".
  - bartlomieju 5y ago
    Bartek from Deno here, this is mostly true. The runtime heavily relies on the Web standards, while Go was a heavy inspiration for Deno's standard library (deno_std [0]) in the early days. Since then, we've been relying on Web standards even more and the standard library started seeing more modules added that were based on popular npm modules. On another note: huge thanks for rust-analyzer, not only is it my main daily driver, but we also used it as a reference for implementation of "deno lsp" - the language server that ships with Deno. [0] https://github.com/denoland/deno_std https://github.com/denoland/deno_std
    - matklad 4y ago
      <3 I can't say I am a user of deno today, but my feelings about deno remind me my feelings about Rust circa 2015: it's not clear if the thing succeeds, but, if it does, that would be a huge boon for the whole industry, as finally "better is better" would take over "worse is better". Deno has all the potential to become the first really dependable scripting environment! So, I wish you luck and hope one day I'll migrate my jekyll+asciidoctor blog to deno :-)
chrismorgan 5y ago
> But don’t make LSP itself a first class concept. Why not? I’m interested in any concrete reasons why doing so would be a bad idea, why you should maintain independent extension points and bind LSP to them, rather than providing LSP-shaped extension points, as it were.
- matklad 5y ago
  Several things: _First_, a general clean architecture considerations -- protocol is "view", it shouldn't infect the model. _Second_, LSP is a way to consume semantic info, but there might be others. A different protocol might come along, or you might want to embed the server as a library somewhere. _Third_, LSP is language agnostic, so, to implement LSP, you'd have to dumb-down internal rich language specific representation to a language agnostic one. The system will be more evolvable if such dumbing down happens near the edge of the system. _Fourth_, LSP really just stretches the surface of whats possible. Language servers should push the boundaries of whats possible to both deliver a better UX for a specific language as well as to pressure the protocol itself to implement more advanced features. The design guideline we use in rust-analyzer is that we are building not for LSP/VS Code, but for a hypothetical, rust-specific perfect IDE whose frontend is capable of doing anything we want.
wheelerof4te 5y ago
I see that the days when people just sat down with their favorite code editor and wrote code are long gone.
- pas 5y ago
  my favorite editor was the turbo pascal one! (then Delphi, then PHP, then PHP with unit tests and something like TDD/BDD, then entirely too much Bash, and then python, and then TS, Scala, Rust) I never learned to code without a superfast feedback loop. (maybe I've never learned to code!!!?) oh, okay, there was a C++ project (a music recognizer - shazam clone) and I've mostly hacked that together in vim until the compiler stopped complaining about move semantics and right-references.
- h2odragon 5y ago
  I've argued with others over the utility of "Do What You Think I Meant" editors. I feel the difference is significant; the kiddies are trying to draw using pantographs and exoskeleton rigs, in terror of the possibilities they might explore had they not the artificial constraints on their tooling.
qwerty456127 5y ago
Does any of the LSP-powered editors come any close to the smartness of IntelliJ? This is a genuine question, the last time I checked (over a decade ago) nothing could really compete to IntelliJ, everything else felt dumb.
- matklad 4y ago
  For the 20% of the things you use 80% of the time (basic completion, goto definition, syntax highlighting), I'd say advanced LSP servers like rust-analyzer are pretty close to what IntelliJ provides. But the heavy tail of 80% of the features which are used relatively rarely is very heavy. The biggest one there is the "change signature" refactor. LSP today can't really express that.
  - qwerty456127 4y ago
    Nice to know, thank you.
edding4500 5y ago
I got my hands on LSP and developed a language extension for a DSL. I found the overall idea great. he only thing that grind my gears is that currently, LSP focuses on some very common language support features and some Microsoft / VS Code inventions such as CodeLenses. The latter being a feature of the VS Code API, I am wondering whether the LSP will stay a platform agnostic protocol.
- carapace 5y ago
  Of course it won't [stay a platform agnostic protocol]. They'll leverage it as soon as they think they can get away with it. MS owns Github. They have already rolled out AI auto-completion. This LSP will be used as another way to get between users (in this case the users are devs) and the machine and "extract rent". Ultimately, complexity itself is the enemy.
sicp-enjoyer 5y ago
The batch style tools, where you launch it with stdin input, and get back results to stdout I still think are the majority of operations an IDE want to run. It's when you have a tool that has ongoing async communication which enables the editing the author mentions that LSP is really useful. Implementing a goofy protocol for each tool is a pain.
PaulHoule 5y ago
(1) WebStorm does a pretty good job as a Javascript IDE so far as I'm concerned (doesn't seem that you need TypeScript) (2) Maybe it is better today but historically Eclipse was a workable Java IDE but the "plugins" for other languages pretended to work but didn't really work, at least not as well as Java.
radarsat1 5y ago
I feel really alone being happy without autocomplete. Doesn't anyone else find it annoying and distracting to have your IDE or editor constantly throwing "suggestions" at you? I am quite happy with that feature turned off. I guess I do use it in Python REPL quite often though. But there at least you have to prompt it by pressing tab.
- matklad 5y ago
  I am in a similar position actually -- I don't find autocomplete to be the main feature of IDE, it doesn't add that much over dumb hippie-expand. Code navigation is much more important.
  - Sindisil 5y ago
    100,000% this. If I could get the navigation and analysis (go to def/decl/parent/etc., show/jump to uses/children/impls/etc.) all on its own, without auto-complete, type hints, or syntax highlighting, I'd be a pretty happy camper. I'm fine with features that are on-demand (on-demand completion, type info, actions, etc.). They add occasional value without distraction. About the only other things I care about in a development environment are compile output navigation (optionally with isolated warnings & errors, but full output is mandatory), configurable run and tooling integration (doesn't need to be close integration, either, just flexible and configurable). Gravy/Frosting (depending upon your preferred metaphor) would be debugger and documentation integration. Both are hard to do well, and can usually be accomplished using a decent run/tooling interface.
- _0ffh 5y ago
  You're not alone! =)
  - radarsat1 5y ago
    good to know! there are dozens of us!
- dllthomas 5y ago
  I think I'm with you in that I like autocomplete but I want to ask for it; prompting feels too busy.
bern4444 5y ago
The existence of multiple LSP clients demonstrates the success of LSP and is a strength, not a weakness as the author suggests. Often, it's because LSP clients have differences not in interacting with the LSP Server, but in how that is surfaced to the user. Some LSP clients are written in different languages, and that can make some implementations faster than others. This is why LSP is a success. It allows for clients to be written in many ways but for all of them to do the most basic thing they need to (go to def, rename, refactor, search symbols etc). Multiple LSP clients is a success and offers users more choice about how they want to interact with their code. Do I want a search for symbols to open in a new window, to replace the current buffer, in a searchable list etc.
munificent 5y ago
> Dart, despite being a from-scratch, relatively modern language, ended up with three implementations (host AOT compiler, host IDE compiler (dart-analyzer), on-device JIT compiler). We also had an earlier analyzer written in Java that was mostly discarded and rewritten as well as two compilers to JavaScript that share less code than you would expect. Some of the redundancy is because, as the author correctly states, the needs of an IDE are quite different from the needs of a batch compiler. Likewise, an ahead-of-time whole program compiler has different constraints from a JIT compiler running a program from source. But a lot of the redundancy in Dart's case is incidental. We have a fairly large, distributed team with a lot of autonomy and sometimes that led to people doing their own thing instead of putting in the effort into coordinating and sharing more code. Also, the language changed radically from Dart 1.0 to 2.0. 1.0 was a dynamically typed scripting language designed to be run from source in a native VM directly embedded in a browser with a separate optional type system mostly used at dev time. Dart 2.0 is a fully statically-typed language with a more typical compilation process. That monumental change led to some duplication and technical debt that we are slowly paying down by sharing more and more of the front end over time. The path I see most new languages take is: 1. Start with a command-line batch mode compiler. 2. Eventually get popular enough that users clamor for real IDE support. 3. Cobble together an IDE plug-in based on the batch-mode compiler's front end. 4. Discover that the latency is horrific and slowly and painfully realize you need to write a new front end architected for IDE use. 5. Now you have two front ends. I believe the throughput loss of a front-end designed for interactive IDE use is less than people realize. So I suspect that a more efficient path for any new language is to design your front end for interactive IDE use from day one.
- matklad 5y ago
  > So I suspect that a more efficient path for any new language is to design your front end for interactive IDE use from day one. Maybe! One problem is that this is going to be harder than starting with a batch compiler: you might not get to 2. due to the lack of resources. Another plausible trajectory is the following: 1. you start with a non-self hosting batch compiler, whose explicit goals are simplicity, language-designer iteration speed, and short time horizon, and which has deliberately minimal capabilities when it comes to error messages and other user-facing niceties. 2. using this bootstrap compiler, get to the 1.0 of the language. 3. implement a second, self hosting compiler with a long-term focus, interactive capabilities and all that stuff. 4. either discontinue bootstrap compiler, or explicitly support it as a minimal implementation for the purposes of differential fuzzing and specification.
andrewla 5y ago
To an extent the thesis here (that M*N is not insurmountable) has a grain of truth, but depends on the idea that an editor will support some specific protocol for that communications. From the article: > Rather, a language should implement a server which speaks some protocol, an editor needs to implement language agnostic APIs for providing completions and such, and, if both the language and the editor are not esoteric, someone who is interested in both would just write a bit of glue code to bind the two together Yes, this is the ideal world, but the first editor to actually go ahead an do that was VSCode, and the protocol they chose was LSP, so everyone else fell in line, because having a protocol that worked with one editor was at least a workable start. Another example of this is syntax highlighting, which is definitely N*M -- people have written syntax highlighting plugins for every conceivable language for almost every editor, which always requires some manual patchwork. Editors are increasingly migrating over to tree-sitter for this support, which seems to have become the de facto standard, and as a long-time vim user and occasional syntax highlighting maintainer, this cannot come soon enough.
stalfosknight 5y ago
I was hoping this was an article about Lumpy Space Princess. I am disappointed.
sally1620 5y ago
These are only hard problems in Unix world. VisualBasic and Delphi always had really good auto-complete.
deleted 4y ago
[deleted]
garfieldnate 4y ago
I've felt for a while that this is one of the most amazing advancements in my field in my lifetime. What's missing for me is an updated standard curriculum on building compilers; we shouldn't be teaching students how to just build batch compilers anymore. We need a standard set of algorithms that we use to create interactive compilers and IDE support, including refactoring.

373 points by afdbcreid 5y ago

It's worth noting that typescript/js support in VSCode doesn't come from an LSP. It's actually tangled with VSCode itself

salmo 5y ago

Ugh. This whole thing is exhausting because everything is so oversimplified (as is any conversation on the topic) and there isn’t a right answer. You can’t compare the architecture of VSCode, vim, emacs, VS, IntelliJ, etc. and come to any conclusion. They are completely different tools, built at different times, with different goals, by different (dis)organizations. Emacs was a bunch of primitives that were wrapped in Lisp, and you could do anything. It was a day 1 goal. Ignoring the things that evolved to become normal in the Lisp on top is weird. It was extensible by design and completely open ended, with packages evolving to being part of the expected toolchain. Vim took vi and bolted on some scripting interface that kept being pushed, adding extensibility by demand. Now we have neovim embedding lua which is more emacs-like, and driving vim features as well. But it got to add higher level “primitives” because it’s a new implementation. IDEs like VS, Borland X, Sun Studio, etc. were purpose built tools for a specific toolchain package. Only after the fall of traditional IDE/RAD environments did VS start trying to be an everything IDE. Eclipse and Netbeans started as Java IDEs in Java, were all focused on architecture, and enabled plugins. People started making plugins to support all the languages (and then totally break the IDE by installing the wrong combo). As eclipse, well eclipsed, NetBeans you started seeing Eclipse “distros” of canned plug-in combos to not implode. I know folks with an eclipse install per codebase they work on. IntelliJ got the architecture part from Eclipse, but stayed very hand curated like the old RADs and IDEs. You can install all the things in IntelliJ Ultimate or buy the more focused language specific ones (PyCharm, CLion, etc). The plugins are less flaky because they’re predominantly 1st party. The language specific ones make each toolchain ergonomic like a traditional IDE. VSCode got the benefit of being the newest. Strip down below an “IDE” but above a text editor exposing primitives. Use web tech so the kids can make stuff because Java isn’t the lingua franca it once was. It could look back for a better sweet spot. So whatever. Use what you find productive and comfortable. I wander between probably 10 languages. I tend to have an editor and terminal side by side. It’s comfortable for me. I prefer the editor+ experience more than the integrated IDE one. I also am generally mouse-adverse. But that’s me and I’m not “right.” I like my neovim acting like VSCode more than VSCode acting like vim. But it’s because it matches what my hands do already. When I do !!fmt or whatever and it doesn’t work, it takes me out of my flow. And when editing remotely, I still get vi/vim just without bells and whistles so my hands still work. Many just never have that use case. That comes at a cost of maintaining a crazy config for me vs clicking for plugins. It used to be 10-15 vim plugins. Now it’s more LSP + a few. But I revision control it and revisit every 6 months or so. That trade off’s worth it to me, maybe not to you. Some folks still love their full IDEs with tighter integration with a toolchain and language-specific features ergonomically placed. They likely use Java/C#/C++ and have to dabble in html+js at times. Use and make what you want. Make an LSP or don’t. Integrate them into something or don’t. All of the experiences described in the article still exist. No one has to use any of them. Follow your heart (or I guess your hands). Saying one is best is like saying some kind of shoes are best for everyone. They’re tools, but ones we live in. Wear what’s comfortable. In the words of a song by Sheryl Crow that I don’t particularly like “If it makes you happy, then why the hell are you so sad?”

jez 5y ago

I think this post hits the nail on the head. Before LSP, you could google for “$language Vim support” and find some results but more often than not, you were going to get nothing. Once LSP gained popularity, the question shifted to “why doesn’t $language have a language server yet?” which set an easy goal post for languages and compilers to go from 0 to 1 in terms of IDE support.

wodenokoto 5y ago
But isn’t that exactly what he starts of arguing against? The first chart shows that before lsp you needed special language support for each language for each editor, which is unfeasable. After lsp you just need lsp support for each language. The author then goes on to say “this is wrong”
- aaron_m04 5y ago
  Right. My takeaway after reading the whole post was "so it really _is_ M * N after all".
- ubertaco 5y ago
  I don't think the author is quite saying "this is wrong". It seems like he's more saying "this is an oversimplification, and the reality is a more-nuanced version of that statement" -- specifically, that rather than "now languages just build a language server, and editors just accept LSP", it's that "now languages just build a language server, and editors just accept LSP, plus a bit of LSP-specific configuration to point the editor at the particular LSP for a context." Breaking things down a bit further, there are three scenarios discussed in the article: 1. M × N -- every language needs a from-scratch, bespoke plugin for every editor 2. M + N -- the "standard explanation" for LSP, where you build one LSP client implementation per editor, one LSP server implementation per language, and you're done 3. The reality of what exists today: one LSP server implementation per language, one LSP client implementation per editor, and a tiny bit of (usually-end-user-supplied) configuration in your editor to glue those two sides together. The key difference between scenario 1 and scenario 3 is that scenario 1 requires someone with _very deep_ knowledge of a language's structure and semantics to implement a fully-fledged plugin for _every_ editor, usually requiring deep knowledge of that editor (or vice-versa), where scenario 3 allows both ends to wrap that deep knowledge up behind a standardized interface so that any shmuck can write like <100 lines of lua to glue the two ends together (speaking from my own experience as an "any shmuck" using neovim)

kqr 5y ago

> Notably, the two-sided market problem was solved by Microsoft, who were a vendor of both languages (C# and TypeScript) and editors (VS Code and Visual Studio), and who were generally loosing in the IDE space to a competitor (JetBrains). Talk about commoditising your complement! ---- Unrelated: how are LSP implementations and Emacs support generally these days? When I last tried 2–3 years ago the Emacs-specific plugins for various languages were still more reliable and predictable than their LSP counterparts, but has this changed a lot? If so, I might be inclined to try again.

vim-guru 5y ago
It's catching up. You can use them in tandem though and ignore the features that overlap by configuration. I use Eglot with a bunch of different languages. Clojure is the only one where I still use other packages for dealing with similar things.
- ashton314 5y ago
  Eh, gonna be really hard to be Emacs' prowess in the land of Lisps. This rings pretty true to me: they all work well together.
- yewenjie 5y ago
  Can you please comment on your experience with Eglot? Is it better/ faster?
ashton314 5y ago
LSP support with lsp-mode is fantastic with at least Rust. I use Elixir at my day job and while it does a good job of doing 98% of what I want (go to definition, find references (iffy; Elixir is difficult to analyze like this), code lenses, autocomplete, etc.) there are a few rough patches. This is likely due to shortcomings with the lang server implementation, not the lsp-mode library itself. Like I said, editing Rust is awesome in Emacs; saw someone working on Rust in VS Code and didn't notice anything that Emacs couldn't match. See also lsp-ui; I don't turn this on personally, but it does give you some significant bling.

vim-guru 5y ago

I have feelings about the last paragraph there. Most protocols are limiting by nature, so although LSP has made wonders to a lot of languages for a lot of editors out there; I'm stuck thinking that it might have stifled progress for some as well. I don't have a proposed solution or anything, just pondering if we could have done better.

OtomotO 5y ago
We could always have done better, but the nice thing is, that most of the time, we can. Will it take time and energy? Sure! Such is life. At least it's not as bad as e.g. JavaScript.
rob74 5y ago
Well (Ok, I'm partly contradicting the article here), most of the things LSP supports (autocomplete, jump to definition, ...) have been pretty standard features of more advanced IDEs (the original Visual Studio, the various JetBrains IDEs, even good ol' Delphi) for decades now. So the "problem domain" is pretty well defined already, LSP is just standardizing the interfaces.
- dmitriid 5y ago
  Advanced IDEs are advanced because they provide so much more beyond basic jump to definition and autocomplete (which, again, can be very advanced and be context-aware, provide code shortcuts etc.). It's good that tooling has improved across the board for many languages with LSP, but there's so much more still.
chii 5y ago
> it might have stifled progress for some as well which language got stifled as a result of the existence of LSP? Would that language still have stifled anyway without LSP?
- vim-guru 5y ago
  I haven't looked much into the implementation details of LSP, so I was not aware of custom-commands upon writing my comment. Maybe it hasn't stifled at all, but efforts have definitely been split, as prior art is still maintained years after LSP's launch

p00f 5y ago

> Contrast this with Emacs and Vim. They just don’t have proper completion as an editor’s extension point. Rather, they expose low-level cursor and screen manipulation API, and then people implement competing completion frameworks on top of that! This is just wrong, https://vimhelp.org/insert.txt.html#compl-omni https://vimhelp.org/insert.txt.html#compl-omni The first occurrence of "omni" I can find in the git log is this commit from 2012: https://github.com/vim/vim/commit/5d3a8038b6a59e6f1b219f27ec77d6067e643ac1 https://github.com/vim/vim/commit/5d3a8038b6a59e6f1b219f27ec... which means it's even older

the-smug-one 5y ago
I want to further bang about on this: This is a plain false statement. Emacs's completion-at-point was (C-h C-f) introduced at 23.2, in May of 2010. company-mode has commits going back to 2009.
- monsieurbanana 5y ago
  That's his point, people use company mode because Emacs doesn't have built-in "modern" completion UI. Completion at point uses the minibuffer, right? If so I wouldn't call it a real alternative, specially without minibuffers plug-ins like ivy and co. I don't think the Emacs situation is bad at all, but I agree 100% that it needs third party plug-ins to be anything close to what people expect of an editor in 2022.
  - the-smug-one 5y ago
    His point is poorly expressed in that case. Let's look at the quote again. >Contrast this with Emacs and Vim. They just don’t have proper completion as an editor’s extension point. Rather, they expose low-level cursor and screen manipulation API, and then people implement competing completion frameworks on top of that! People don't implement competing completion frameworks on top of "low-level cursor and screen manipulation API". There are more high-level APIs included in Emacs, such as completion-at-point. To quote company-mode's website: >The CAPF back-end provides a bridge to the standard completion-at-point-functions facility, and thus works with any major mode that defines a proper completion function. To me the author misrepresents what Emacs has built-in and what package writers use. Technically company-mode isn't third party, it's part of GNU Emacs. It needs to be downloaded, yes, but it's not third party. Emacs needs a package recommendation engine (I know VS Code has this, awesome!).

v-yadli 5y ago

One particular problem of M+N is that it requires LSP to cover all the wanted features provided by server and supported by client. This is not always the case for existing servers. Semantic highlighting wasn't in the protocol until 3.0 (iirc), and many servers have extra non-standard endpoints. So the real world situation is that there's a core feature set allowing M+N, and additionally, custom endpoints and features requiring MN. Nevertheless the non-standard MN endpoints at least follow the "shape" of LSP -- same communication channel and message format, and that makes MN much easier than before already! As the core protocol evolves, more MN features will be decoupled to M+N.

jez 5y ago
This is definitely true but like you mention because there’s a common communication channel and message format, these language-specific RPC endpoints exist whereas before LSP, language servers didn’t have “custom” features because there weren’t IDE-first language servers at all. There are already a more than a couple LSP features I wish existed, but in the mean time it’s good enough to have the (typed! and using rich data structures!) RPC endpoints exist for enterprising users to wire up to their IDE’s presentation layer.
orlp 5y ago
I also would like to note that this could provide a chilling effect for programming language design in the future. If LSP's become ultra-mainstream it might carve out a path of least resistance that prevent future innovation and adoption of language features incompatible with the protocol. Although I must admit I have no idea what such features could look like.
- oblio 5y ago
  Standardization ossifies some things but also allows building many things on top of a stable base. Innovation could just happen at a higher level.
  - TuringTest 5y ago
    > Innovation could just happen at a higher level. That, or retro-backing into the protocol novel features required by those experimental languages. In my experience, industry-wide ossification and chilling effects appear not because the environment makes it difficult to build new designs, but because it makes it difficult to think outside the current paradigm and imagine novel features. Once those innovative ideas are out of the box and many people understand their need, the frameworks needed to support them are created. I know because right now we're experiencing such a paradigm shift in terms of note-taking and knowledge building (abandoning WYSIWYG word processors in favor of networked-thinking bidirectional-linked graphs of notes), which IMHO sooner or later will extend to programming tools as well.

skybrian 5y ago

The problem with this argument is that it's based on only a few of the most popular IDE's that the author happens to remember, but there were a bunch of others. For Java, you might remember NetBeans or VisualAge. There's a long tail. Some of these IDE's supported more than one language. For example, VisualAge also supported C++ and SmallTalk. But each IDE supported a different set of languages with implementations of varying quality, they each had their own complicated plugin framework, and that's the N:M problem. Even if it were just IntelliJ and Eclipse, that's still implementing support for a new language twice from the ground up, and you'll be doing it in Java both times, which is a lot of work for a team working on a new programming language. You can look at the Wikipedia page to get a sense of how many different IDE's and languages there are out there: https://en.wikipedia.org/wiki/Comparison_of_integrated_development_environments https://en.wikipedia.org/wiki/Comparison_of_integrated_devel...

zmmmmm 5y ago
> you'll be doing it in Java both times I honestly think this is highly underrated as a reason. Teams working on a language want to program in that language. It doesn't really matter if it's complex, or they have to repeat the work for multiple IDEs. They love doing that. They will actually compete to do that (look how many posts on HN these days are just "$MUNDANE_TOOL written in Rust"). The main thing LSP did is it moved the boundary of the implementation to the other side of the language barrier.

hsn915 5y ago

I tried a few language plugins for VSCode that were all based on LSP, and they were all broken. So for me, the question is wrong. It's not "Why is LSP so great?". The question should be "Why are people so hyped about tech that's based on questionable assumptions and that hasn't proved itself?" The reason VSCode invented LSP is that different programming languages don't have a reliable way to intercommunicate (other than as separate processes communicating over a socket).

adwn 5y ago
> The reason VSCode invented LSP is that different programming languages don't have a reliable way to intercommunicate (other than as separate processes communicating over a socket). The processes-communicating-via-IPC architecture is way underrated for plugins. It's much easier to separate the plugin from the host, and to cleanly stop or restart the plugin. For example, sometimes (rarely) rust-analyzer hits a bug and starts consuming 100% CPU. When that happens, I just kill that process and VS Code transparently restarts rust-analyzer; I don't have to restart the IDE, and all other plugins are unaffected as well. That would be practically impossible if rust-analyzer were a library instead of an independent process.
- zozbot234 5y ago
  That's because separate processes are not "plugins" at all, but part of an outside toolset that may or may not be supported by a thin "plugin"-like IDE component. The patterns involved are completely different; the separate-processes pattern was popularized by *nix, whereas plugins were always more common in Windows and Mac.
  - adwn 5y ago
    The concept of a "plugin" isn't restricted to dynamically linking shared libraries, but rather defined by the concept of having an independent host application, which can load additional functionality at runtime. It doesn't matter whether this is achieved through shared libraries, interpreted code, or separate processes.
- 5y ago

IshKebab 5y ago

> If the standard theory were correct, then, before LSP, we would have lived in a world where some languages has superb IDE support in some editors. For example, IntelliJ would have been great at Java, Emacs at C++, Vim at C#, etc. My recollection of that time is quite different Interesting theory but my recollection of that time is pretty much exactly that. Lots of IDEs with great Java support (IntelliJ, Netbeans, Eclipse), a very small number with good CJJ support (Visual Studio, Qt Creator, maybe KDevelop or Code::Blocks if you're generous with "good"), and not much else. How many IDEs had great Delphi support? As far as I know exactly one - the Delphi IDE. Ok there were a couple of exceptions - Eclipse had decent C++ support IIRC and IntelliJ sort of had support for loads of languages, but that was a business whose main product is IDEs. I think the alternative suggestion is also true. They're both good reasons for the success of LSP.

junon 5y ago

I recently switched my neovim config over to LSP from ncm. My experience is now much buggier, slower, features work some days and not others, syntax highlighting just stops working randomly, trash files get spewn all over my repositories that I have to .git/info/exclude, etc. Not sure what the Right Direction is when it comes to text editing but this isn't it.

davidatbu 5y ago
Assuming you're talking about this ncm[0], are you aware that ncm is a " completion framework for neovim", which is different from an LSP? ncm is either an LSP client itself, or, it is talking to neovim's internal LSP client, to get completions from LSPs. It's also important to note that LSPs almost always provide you more than auto completion (for example, go to definition, go to implementation, find references). Do let me know if you have more questions, neovim and LSPs are my "daily driver" as a dev, so to speak. [0] https://github.com/ncm2/ncm2 https://github.com/ncm2/ncm2
- junon 5y ago
  Yep I'm aware of what it is, I used it for years and years, thanks. NCM ancillary plugins that interacted with e.g. libclang were more what I was referring to, but I'd hoped the average reader could deduce that. Those features of LSP I do not use, namely because LSP is a buggy mess. Some files have wild syntax errors reported for them that don't exist when compiling, but not others. Half the time, the auto-completion simply doesn't do anything. Nothing pops up, nothing auto-completes, etc. And now, just a few weeks after switching over, without changing a thing about my config, the arrow keys for selection have completely stopped working inexplicably.
  - davidatbu 4y ago
    So, synthesizing your reply here with your reply to @mjlbach, it seems to me like you are attributing to the whole LSP ecosystem (by saying things like "LSP is a buggy mess") (1) problems with treesitter, and (2) problems with clangd, one LSP server implementation. I feel like if your comments' utility to the conversation would be greatly improved by prefacing them with something like "in my experience with clangd, treesitter and neovim". I say this because my experience with LSPs is the polar opposite of yours[0]. While I believe there's value to sharing our personal experiences, I think we have to be careful not to present our personal experience (especially with (neo)vim setups, which tend to vary wildly from one to the next) as representative of the whole ecosystem. [0] I also have nothing but praise for most of the LSPs I've used (clangd, typescript-language-server, rust-analyzer, clangd, pyright, css-language-server, sumneko/lua-language-server, of the top of my head).

pjmlp 5y ago

> To get a decent IDE support, you either used a language supported by JetBrains (IntelliJ or ReSharper) or. VS, Eclipse, Netbeans, Delphi, C++ Builder, KDevelop, QtCreator,... > Notably, the two-sided market problem was solved by Microsoft, who were a vendor of both languages (C# and TypeScript) and editors (VS Code and Visual Studio), and who were generally loosing in the IDE space to a competitor (JetBrains) Nope, the IDE space on Windows has always been owned by Microsoft since Borland dropped the ball with their management crisis where to go next. VSCode and LSP were created out of Monaco and Erich Gamma's stewardship, yep one of the four on the patterns book, and original Eclipse team. "VS Code an Overnight Success… 10 years in the making" https://www.youtube.com/watch?v=hilznKQij7A https://www.youtube.com/watch?v=hilznKQij7A

Ygg2 5y ago
> VS, Eclipse, Netbeans, Delphi, C++ Builder, KDevelop, QtCreator,... Speak for yourself. Editing C# is much better in Rider than VStudio. Eclipse was a competitior, but I dread having to use it for anything nowadays. CLion just doesn't work correctly, yet. For Rust VS Code is better, but only because of awesome rust-analyzer plugin.
- pjmlp 5y ago
  Not keeping up with the times with VS? Eclipse is still my get to go, InteliJ wants me to buy Clion + InteliJ licenses for features Eclipse does out of the box. VSCode isn't an IDE, plus Rust is the new kid in town.
  - Ygg2 5y ago
    > VSCode isn't an IDE Add enough plugins and it will be. Rust being new is one part, other part is that Jetbrains really insists on their plugins vs LSP. I get Rust errors in Clion on programs that run normally. Bugs that rust-analyzer/cargo don't report.
    - kuschku 5y ago
      Bolting 200 things to a bicycle doesn't make it a truck, and no amount of addons can turn VS Code into an IDE. The I means Integrated, and VS Code with plugins and LSPs is anything but.

Shadonototra 5y ago

LSP is a great thing, one of the very few things i appreciate from microsoft

lf-non 5y ago

One thing I have difficulty understanding is that given MS is (was?) the primary proponent of LSP, and the spec was originally written with TS in mind, and VSCode has first class support for LSP, why do community projects like typescript-language-server (along with a bunch of others like the one by sourcegraph) need to exist ? Why can't arbitrary LSP clients connect directly to tsserver (maintained by MS) ? [1] https://github.com/typescript-language-server/typescript-language-server https://github.com/typescript-language-server/typescript-lan...

yanis_t 5y ago
I think the problem is that the tsserver for historical reasons even though is quite similar, still does NOT implement the LSP protocol. That's why you need another level here.
richdougherty 5y ago
> Why can't arbitrary LSP clients connect directly to tsserver (maintained by MS) ? Can you explain a bit more about why they can't connect?
- norman784 5y ago
  tsserver isn't LSP compliant, iirc they started with tsserver way before LSP was out, but since then didn't migrated to LSP. I think they didn't migrated because: - It's a big selling point for vscode. - It might hard to migrate what they have to LSP, they might implement a lot of non LSP standards. - Migrate to LSP could possibly delay new features because of the rewrite in tsserver.
- lf-non 5y ago
  Because tsserver (maintained by MS) is not an LSP server. There is an open issue [1] that has been open since 2020. [1] https://github.com/microsoft/TypeScript/issues/39459 https://github.com/microsoft/TypeScript/issues/39459
paskozdilar 5y ago
> Why can't arbitrary LSP clients connect directly to tsserver (maintained by MS) ? As far as I'm informed, they can. For example, NeoVim language client has a pretty wide range of officially supported servers [0], including tsserver. [0] https://github.com/neovim/nvim-lspconfig/blob/master/doc/server_configurations.md https://github.com/neovim/nvim-lspconfig/blob/master/doc/ser...

DrBazza 5y ago

It's not IDEs in 2022 that use LSP, it's editors. As a grey-beard, M x N wasn't "small". From memory, notepad, notepad++, scite, vim, emacs, netbeans, eclipse, VS studio, IDEA, sublime text, bbedit, plus a number of linux only ones. Languages, VB, C++, Pascal/Delphi, Fortran, Java, C# (from 1998/9?). The problem was tight coupling of languages to IDEs, in particular VB, and C++ to VS, and Java to Eclipse.

davidatbu 5y ago

Tangentially related: If a talented folks out there could please help out with (to the best of my knowledge) the only LSP for Kotlin out there[0], that would be amazing! I'm really attracted to Kotlin as a language[1], but, last I checked, IntelliJ is the only way to get great tooling for Kotlin, which is a show stopper for me since I've invested a lot into my neovim config. [0] https://github.com/fwcd/kotlin-language-server/projects/1 https://github.com/fwcd/kotlin-language-server/projects/1 [1] What attracts me to Kotlin is that it is actually statically typed (ruling out Python/JS/TS), with a relatively expressive type system (ruling out Golang), great type inference, not verbose (ruling out Java), not too involved with low level details (ruling out Rust), not too steep of a learning curve (ruling out Rust again), and a great cross-platform community (ruling out Swift).

ZeroGravitas 5y ago
I guess you're aware but the Jetbrains folks (who do Kotlin and the IDE range) have been a bit cold towards LSP since it's basically commodifying their USP for their IDEs (rich, language specific integration) which is why kotlin LSP is a bit of a no man's land.
- Tmpod 5y ago
  Yeah, that's true. I can see their reasoning, but I'd hope they wouldn't be so cold.
- zozbot234 5y ago
  Their USP is native, efficient tooling for something that's usually implemented in managed platforms like Electron/JS or the JVM. They would in fact benefit from a rich LSP ecosystem, since their tooling would be more easily applicable across languages.
  - ZeroGravitas 5y ago
    I don't use them, but aren't their IDEs still JVM?

codeflo 5y ago

This is a “correction” of a strawman only: The left diagram is exactly what would have been necessary to achieve the same result. Since that’s clearly infeasible, most of the arrows didn’t actually exist, and never would have. That was always the point. It’s precisely because the left picture is impractical that something like the LSP was necessary. Somehow, the author misunderstood the original argument, dismissed it, independently discovered the actual argument, and now claims they’re the one who found the true reason for LSP, when in reality, it’s just the standard argument.

afdbcreid 5y ago
No. The point is that while we need analyzer for each language, the connections between the editors & the analyzers have no problems to be N*M because they're very small. So we don't really need a protocol like LSP. But the rise of this protocol caused people to write independent, non-per-editor analyzers, and this is indeed necessary. I completely agree with that.
- codeflo 5y ago
  Hmm, maybe I misunderstood the author’s point slightly, but I find what you suggest the argument was even less convincing. Here’s the history: Every editor has a language abstraction already. And some language analyzers had an “editor abstraction”, perhaps notably Roslyn for C# with its programmable refactoring rules that worked in multiple editors. None of that was new. But that’s still N*M if you do the math, so that graph was never completely filled — for the obvious reason that this would have been practically impossible. No mystery about it.
  - mgdlbp 5y ago
    And considering how VB.NET was incorporated first-class besides C#, Roslyn seems rather ahead of its time, with both language abstraction at the compiler level and language-agnostic analysers based on semantic info. I still can't get over the awesomeness of being able to just throw together a fully supported custom analyser from a template—unique to Roslyn to this day, AFAIK.

numToStr 5y ago

> And try to write more about how to implement language servers — I feel like there’s still not enough knowledge about this out there. Yes, please! There is next to nothing about "How to write a language server using LSP". All I can find is "how to write a language server for VSCode". - Written by a neovim user

ZeroGravitas 5y ago
I think the idea is that those should be the same though. Like the move to standardise web extensions so a 'chrome' extension works in obscure browsers too.
matklad 5y ago
Well, I think it’s a bit more than that already :-) https://m.youtube.com/playlist?list=PLhb66M_x9UmrqXhQuIpWC5VgTdrGxMx3y https://m.youtube.com/playlist?list=PLhb66M_x9UmrqXhQuIpWC5V... https://rust-analyzer.github.io/blog/2020/07/20/three-architectures-for-responsive-ide.html https://rust-analyzer.github.io/blog/2020/07/20/three-archit...

yanis_t 5y ago

LSP is what made me make a full switch to Vim[0]. Neovim even made it part of its API out-of-the-box, and that's a perfectly reasonable move given how much benefit it brings. [0] https://www.vimfromscratch.com/articles/vim-and-language-server-protocol https://www.vimfromscratch.com/articles/vim-and-language-ser...

nvarsj 5y ago

Great article! LSP brought me back to Emacs. Real IDE support with best in class editing abilities is bliss. Microsoft really nailed it with LSP (albeit, I agree with the caveats the implementation has some warts). Jetbrains must be sweating - more and more of my colleagues and coworkers are using VSCode these days. I think Jetbrains should seriously consider LSPifying their backends. If they don't, they risk someone else building a better JVM LSP and then they are going to lose a lot of market share. Plus it gives them the opportunity to shape the LSP spec. I'd happily pay Jetbrains right now a sub just for the backend if I can use it with my editor of choice.

kstrauser 5y ago
Same for me. I used Emacs for many years, but other editors (especially VSCode) were getting better language support and it was starting to feel long in the tooth. With LSP support, it’s amazing again. I’m happy to be back in my favorite environment.
zmmmmm 5y ago
Eclipse already did (integrate JVM LSP support). It kinda sucks still in lots of ways, but at the same time, it is pretty surprising how well it works and is totally usable, eg: for working on React. If more work is done on it then it's definitely going to start change things a bit (in part also because Eclipse itself has improved dramatically in the last few years).
Shadonototra 5y ago
intellij isn't popular just for code completion people use intellij because of the various plugin/features/debugging for all the jvm ecosystem/frameworks https://www.jetbrains.com/idea/features/#jvm-frameworks https://www.jetbrains.com/idea/features/#jvm-frameworks similar to how Rider became immensely popular (1st class Unity Engine support) and now Rider C++ support for .sln solution and 1st class Unreal Engine support intellij's problem is their plugin API and development cycle, for a while you needed to restart the whole IDE whenever you wanted to recompile your plugin, as a result, the 3rd-party plugin ecosystem is very poor hopefully they fix all the annoying things with Fleet https://www.jetbrains.com/fleet/ https://www.jetbrains.com/fleet/ they went with JVM so it gives them a poor start already (no comparatif benefit over VSCode and electron, same bloatshit, but they can eat some market share by providing a better out of the box experience)

kqr 5y ago

I might be a bit contrarian here, but > In particular, add assist/code action/(bulb) as a first-class UX concept already. It’s the single most important UX innovation of IDEs, which is very old at this point. Its outright ridiculous that this isn’t a standard interface across all editors. In the grand scheme of things, this particular super-local tactical optimisation seems like entirely the wrong problem to focus on with all the inefficiencies that come with badly designed software. Whether I write public double Hours { get { return _seconds / 3600; } } or public double Hours => _seconds / 3600; is really not going to make a big difference to the quality of my software. Is this where we want to focus our attention? The day I get a little pop-up that says - "are you sure you need to invent your own time abstraction here?", or - "this is a derived value that should only be used in presentation and not business logic or persistance", or - "the customer has not been using the feature that this code is relevant for in six months, are you sure you want to spend time on this?" then we can start to rave about the single most important UX innovations.

turtleyacht 5y ago
Interesting! So the problem of LSPs (or code tools like IDEs) approaches automating the evaluation of "one-level up" abstractions similar to the feedback during code reviews.

oxff 5y ago

Rust-Analyzer on VS-Code is probably the best development experience I've ever had. Makes me want to gouge my eyes out whenever I have to use something else.

zabil 5y ago

I think LSP is great. I've used it in one my side projects to quickly add support for VS Code. We took a more direct approach. Instead of having a separate LSP server our main program runs on LSP server mode. This made a lot of things like distribution and interaction with the core functionality simpler. LSP also solves a big pain by making it easy to build tooling when teams choose to implement an external DSL's.

jmull 5y ago

I don’t know why the author spends so much time “debunking” the M * N argument for LSP. It seems a minor side point. It’s a bit incoherent anyway. The author’s argument seems to be that if you have language-provided pluggable language servers and editors that can host these language server plugins appropriately, then the M * N shrinks down to a small problem, without the help of LSP. Well, OK, but that’s actually the original LSP M * N argument, except with LSP replaced with something just like LSP.

matklad 5y ago
The point is that you don’t need common protocol. A bunch of language-specific protocols would have worked! It’s interesting to ask why that didn’t happen.
- codeflo 5y ago
  But in fact that did happen. Several languages started to implement “editor neutral” code analysis and refactoring. That worked, for the most popular editor/language combinations, but not as a general solution because of N*M.
  - pas 5y ago
    Can you please give a few examples?
    - nerdponx 5y ago
      OCaml (Merlin), Common Lisp (SWANK), Clojure (nREPL), Python (Jedi), even Idris 2 (its IDE Mode).
- bzxcvbn 5y ago
  Is it really interesting? Every editor would have had to support a bunch of different protocols, instead of just one. That's back to square one.
- nerdponx 5y ago

mekster 5y ago

I've just dug through the Neovim ecosystem trying to upgrade my 5 years old .vimrc and I feel it has come a long way since LSP became a thing. It can now actually act like real IDE as the editor knows about the code pretty well. https://github.com/folke/trouble.nvim https://github.com/folke/trouble.nvim https://github.com/tanvirtin/vgit.nvim https://github.com/tanvirtin/vgit.nvim https://github.com/kosayoda/nvim-lightbulb https://github.com/kosayoda/nvim-lightbulb Some features look pretty close to what you get in JetBrains. I can see some parts are kind of rough until you Google enough to fix/customize to your needs but if I take enough time, I might as well feel like its a decent replacement for a real IDE. For a starter, AstroNvim helped me figure out what the juicy plugins are these days. It seems NvChad and LunarVim are preconfigured competitors to it. I wonder if JetBrains one day gets left behind for not using LSP ecosystem but perhaps Fleet might implement it.

KrishnaShripad 5y ago
Can't recommend AstroNvim enough as a starting point to Nvim. It is the perfect bootstrap needed (though I would disable their integration with the scroll plugin — Neoscroll. Prefer the native scroll instead).
- mekster 5y ago
  Exactly what I did. Not sure if people like that slow scrolling.

david_draco 5y ago

There are N programming languages and M libraries (e.g., dates, file-formats, databases, data analysis method X). Currently, each language (Go, Rust, Python, Julia), needs to reimplement each library from scratch, which is a huge effort and delays language readiness for users. Maybe building another programming language can be accelerated by deriving its M libraries from another, using some kind of abstract libraries or translation methods, through a service similar to LSP.

matklad 5y ago
That’s what deno is doing IIRC! They rely heavily on design work that went into web standards and Go standard library, and harness open-source community to deliver implementation.
- oblio 5y ago
  That's in no way what he's suggesting and it's something every language in creation has done: getting inspiration from existing languages. He's saying about actual working code, something akin to universal libraries you could use from any language. Or maybe some auto-translation/hooking up of libraries from another language, through some sort of "library protocol".
- bartlomieju 5y ago
  Bartek from Deno here, this is mostly true. The runtime heavily relies on the Web standards, while Go was a heavy inspiration for Deno's standard library (deno_std [0]) in the early days. Since then, we've been relying on Web standards even more and the standard library started seeing more modules added that were based on popular npm modules. On another note: huge thanks for rust-analyzer, not only is it my main daily driver, but we also used it as a reference for implementation of "deno lsp" - the language server that ships with Deno. [0] https://github.com/denoland/deno_std https://github.com/denoland/deno_std
  - matklad 4y ago
    <3 I can't say I am a user of deno today, but my feelings about deno remind me my feelings about Rust circa 2015: it's not clear if the thing succeeds, but, if it does, that would be a huge boon for the whole industry, as finally "better is better" would take over "worse is better". Deno has all the potential to become the first really dependable scripting environment! So, I wish you luck and hope one day I'll migrate my jekyll+asciidoctor blog to deno :-)

chrismorgan 5y ago

> But don’t make LSP itself a first class concept. Why not? I’m interested in any concrete reasons why doing so would be a bad idea, why you should maintain independent extension points and bind LSP to them, rather than providing LSP-shaped extension points, as it were.

matklad 5y ago
Several things: _First_, a general clean architecture considerations -- protocol is "view", it shouldn't infect the model. _Second_, LSP is a way to consume semantic info, but there might be others. A different protocol might come along, or you might want to embed the server as a library somewhere. _Third_, LSP is language agnostic, so, to implement LSP, you'd have to dumb-down internal rich language specific representation to a language agnostic one. The system will be more evolvable if such dumbing down happens near the edge of the system. _Fourth_, LSP really just stretches the surface of whats possible. Language servers should push the boundaries of whats possible to both deliver a better UX for a specific language as well as to pressure the protocol itself to implement more advanced features. The design guideline we use in rust-analyzer is that we are building not for LSP/VS Code, but for a hypothetical, rust-specific perfect IDE whose frontend is capable of doing anything we want.

wheelerof4te 5y ago

I see that the days when people just sat down with their favorite code editor and wrote code are long gone.

pas 5y ago
my favorite editor was the turbo pascal one! (then Delphi, then PHP, then PHP with unit tests and something like TDD/BDD, then entirely too much Bash, and then python, and then TS, Scala, Rust) I never learned to code without a superfast feedback loop. (maybe I've never learned to code!!!?) oh, okay, there was a C++ project (a music recognizer - shazam clone) and I've mostly hacked that together in vim until the compiler stopped complaining about move semantics and right-references.
h2odragon 5y ago
I've argued with others over the utility of "Do What You Think I Meant" editors. I feel the difference is significant; the kiddies are trying to draw using pantographs and exoskeleton rigs, in terror of the possibilities they might explore had they not the artificial constraints on their tooling.

qwerty456127 5y ago

Does any of the LSP-powered editors come any close to the smartness of IntelliJ? This is a genuine question, the last time I checked (over a decade ago) nothing could really compete to IntelliJ, everything else felt dumb.

matklad 4y ago
For the 20% of the things you use 80% of the time (basic completion, goto definition, syntax highlighting), I'd say advanced LSP servers like rust-analyzer are pretty close to what IntelliJ provides. But the heavy tail of 80% of the features which are used relatively rarely is very heavy. The biggest one there is the "change signature" refactor. LSP today can't really express that.
- qwerty456127 4y ago
  Nice to know, thank you.

edding4500 5y ago

I got my hands on LSP and developed a language extension for a DSL. I found the overall idea great. he only thing that grind my gears is that currently, LSP focuses on some very common language support features and some Microsoft / VS Code inventions such as CodeLenses. The latter being a feature of the VS Code API, I am wondering whether the LSP will stay a platform agnostic protocol.

carapace 5y ago
Of course it won't [stay a platform agnostic protocol]. They'll leverage it as soon as they think they can get away with it. MS owns Github. They have already rolled out AI auto-completion. This LSP will be used as another way to get between users (in this case the users are devs) and the machine and "extract rent". Ultimately, complexity itself is the enemy.

sicp-enjoyer 5y ago

The batch style tools, where you launch it with stdin input, and get back results to stdout I still think are the majority of operations an IDE want to run. It's when you have a tool that has ongoing async communication which enables the editing the author mentions that LSP is really useful. Implementing a goofy protocol for each tool is a pain.

PaulHoule 5y ago

(1) WebStorm does a pretty good job as a Javascript IDE so far as I'm concerned (doesn't seem that you need TypeScript) (2) Maybe it is better today but historically Eclipse was a workable Java IDE but the "plugins" for other languages pretended to work but didn't really work, at least not as well as Java.

radarsat1 5y ago

I feel really alone being happy without autocomplete. Doesn't anyone else find it annoying and distracting to have your IDE or editor constantly throwing "suggestions" at you? I am quite happy with that feature turned off. I guess I do use it in Python REPL quite often though. But there at least you have to prompt it by pressing tab.

matklad 5y ago
I am in a similar position actually -- I don't find autocomplete to be the main feature of IDE, it doesn't add that much over dumb hippie-expand. Code navigation is much more important.
- Sindisil 5y ago
  100,000% this. If I could get the navigation and analysis (go to def/decl/parent/etc., show/jump to uses/children/impls/etc.) all on its own, without auto-complete, type hints, or syntax highlighting, I'd be a pretty happy camper. I'm fine with features that are on-demand (on-demand completion, type info, actions, etc.). They add occasional value without distraction. About the only other things I care about in a development environment are compile output navigation (optionally with isolated warnings & errors, but full output is mandatory), configurable run and tooling integration (doesn't need to be close integration, either, just flexible and configurable). Gravy/Frosting (depending upon your preferred metaphor) would be debugger and documentation integration. Both are hard to do well, and can usually be accomplished using a decent run/tooling interface.
_0ffh 5y ago
You're not alone! =)
- radarsat1 5y ago
  good to know! there are dozens of us!
dllthomas 5y ago
I think I'm with you in that I like autocomplete but I want to ask for it; prompting feels too busy.

bern4444 5y ago

The existence of multiple LSP clients demonstrates the success of LSP and is a strength, not a weakness as the author suggests. Often, it's because LSP clients have differences not in interacting with the LSP Server, but in how that is surfaced to the user. Some LSP clients are written in different languages, and that can make some implementations faster than others. This is why LSP is a success. It allows for clients to be written in many ways but for all of them to do the most basic thing they need to (go to def, rename, refactor, search symbols etc). Multiple LSP clients is a success and offers users more choice about how they want to interact with their code. Do I want a search for symbols to open in a new window, to replace the current buffer, in a searchable list etc.

munificent 5y ago

> Dart, despite being a from-scratch, relatively modern language, ended up with three implementations (host AOT compiler, host IDE compiler (dart-analyzer), on-device JIT compiler). We also had an earlier analyzer written in Java that was mostly discarded and rewritten as well as two compilers to JavaScript that share less code than you would expect. Some of the redundancy is because, as the author correctly states, the needs of an IDE are quite different from the needs of a batch compiler. Likewise, an ahead-of-time whole program compiler has different constraints from a JIT compiler running a program from source. But a lot of the redundancy in Dart's case is incidental. We have a fairly large, distributed team with a lot of autonomy and sometimes that led to people doing their own thing instead of putting in the effort into coordinating and sharing more code. Also, the language changed radically from Dart 1.0 to 2.0. 1.0 was a dynamically typed scripting language designed to be run from source in a native VM directly embedded in a browser with a separate optional type system mostly used at dev time. Dart 2.0 is a fully statically-typed language with a more typical compilation process. That monumental change led to some duplication and technical debt that we are slowly paying down by sharing more and more of the front end over time. The path I see most new languages take is: 1. Start with a command-line batch mode compiler. 2. Eventually get popular enough that users clamor for real IDE support. 3. Cobble together an IDE plug-in based on the batch-mode compiler's front end. 4. Discover that the latency is horrific and slowly and painfully realize you need to write a new front end architected for IDE use. 5. Now you have two front ends. I believe the throughput loss of a front-end designed for interactive IDE use is less than people realize. So I suspect that a more efficient path for any new language is to design your front end for interactive IDE use from day one.

matklad 5y ago
> So I suspect that a more efficient path for any new language is to design your front end for interactive IDE use from day one. Maybe! One problem is that this is going to be harder than starting with a batch compiler: you might not get to 2. due to the lack of resources. Another plausible trajectory is the following: 1. you start with a non-self hosting batch compiler, whose explicit goals are simplicity, language-designer iteration speed, and short time horizon, and which has deliberately minimal capabilities when it comes to error messages and other user-facing niceties. 2. using this bootstrap compiler, get to the 1.0 of the language. 3. implement a second, self hosting compiler with a long-term focus, interactive capabilities and all that stuff. 4. either discontinue bootstrap compiler, or explicitly support it as a minimal implementation for the purposes of differential fuzzing and specification.

andrewla 5y ago

To an extent the thesis here (that M*N is not insurmountable) has a grain of truth, but depends on the idea that an editor will support some specific protocol for that communications. From the article: > Rather, a language should implement a server which speaks some protocol, an editor needs to implement language agnostic APIs for providing completions and such, and, if both the language and the editor are not esoteric, someone who is interested in both would just write a bit of glue code to bind the two together Yes, this is the ideal world, but the first editor to actually go ahead an do that was VSCode, and the protocol they chose was LSP, so everyone else fell in line, because having a protocol that worked with one editor was at least a workable start. Another example of this is syntax highlighting, which is definitely N*M -- people have written syntax highlighting plugins for every conceivable language for almost every editor, which always requires some manual patchwork. Editors are increasingly migrating over to tree-sitter for this support, which seems to have become the de facto standard, and as a long-time vim user and occasional syntax highlighting maintainer, this cannot come soon enough.

stalfosknight 5y ago

I was hoping this was an article about Lumpy Space Princess. I am disappointed.

sally1620 5y ago

These are only hard problems in Unix world. VisualBasic and Delphi always had really good auto-complete.

deleted 4y ago

[deleted]

garfieldnate 4y ago

I've felt for a while that this is one of the most amazing advancements in my field in my lifetime. What's missing for me is an updated standard curriculum on building compilers; we shouldn't be teaching students how to just build batch compilers anymore. We need a standard set of algorithms that we use to create interactive compilers and IDE support, including refactoring.