13 ms·

How LSP could have been better

• warble 3y ago
  What the hell is LSP?
  • Brybry 3y ago
    I think it's https://en.wikipedia.org/wiki/Language_Server_Protocol https://en.wikipedia.org/wiki/Language_Server_Protocol but those words definitely do not exist in the post
    • Joeboy 3y ago
      It's a shame the parent comment was downvoted and this is currently languishing at the bottom of the comments.
  • duxup 3y ago
    https://microsoft.github.io/language-server-protocol/ https://microsoft.github.io/language-server-protocol/
  • teraflop 3y ago
    If you click the "Why LSP?" link in the first sentence of the article, or type "lsp" into Google, you'll get your answer: > The Language Server Protocol (LSP) defines the protocol used between an editor or IDE and a language server that provides language features like auto complete, go to definition, find all references etc.
  • everyone 3y ago
    Lumpy Space Princess https://en.wikipedia.org/wiki/List_of_Adventure_Time_characters#Lumpy_Space_Princess https://en.wikipedia.org/wiki/List_of_Adventure_Time_charact...
    • codetrotter 3y ago
      Article title: How Lumpy Space Princess could have been better Article content in full: She couldn’t.
      • DiabloD3 3y ago
        Can't improve on perfection.
  • reywood 3y ago
    My first thought was Liskov Substitution Principle
  • moring 3y ago
    Why the downvote? It took me quite some reading before I understood it, given that LSP can refer to multiple things. I, too, thought about the Liskov Substitution Principle first.
• zyedidia 3y ago
  Does anyone know why LSP uses UTF-16 for encoding columns? It seems like everyone agrees it is a bad choice, so I'm curious about the original reasoning. Are there any benefits at all to using UTF-16, or was it something to do with Microsoft legacy code?
  • hgs3 3y ago
    There was a lengthy discussion on this [1]. UTF-16 was used because it was convenient: it's what Microsoft API's and JavaScript already use (the latter being the language VS Code is written in). [1] https://github.com/microsoft/language-server-protocol/issues/376 https://github.com/microsoft/language-server-protocol/issues...
    • mardifoufs 3y ago
      That thread was infuriating. Since when does an encoding format have an evangelical task force? I'm all for UTF8 everywhere but wow some of the replies were super cringe. Even when the proposal of "UTF-16 default, UTF-8 optional" was made to keep backwards compatibility, it was not enough. It has to be UTF8 because it's superior technically, as if that's the only consideration! I agree they should've just picked one, but I still don't think the maintainers needed a refresher on what is UTF-8 every 3 comments.
      • sillysaurusx 3y ago
        Count me among the UTF-8 everywhere absolutists. There are two ways to encode text: UTF-8, and a worse choice. But I wouldn’t be annoying about it. I’d just tut tut from afar. (Though if the decision is still up in the air, I’d argue as passionately as any preacher to persuade our fellow devs to adopt our lord and savior UTF-8 into their hearts and minds.)
        • mardifoufs 3y ago
          Yeah I would absolutely take utf8 everywhere. I hate dealing with anything else. But I think the worst part was that the maintainer was clear that he/she wasn't debating this on a technical level. Like, they weren't trying to decide which encoding was better. From what I understand it was more about how best to deal with the (at the time) current design choices without breaking the current implementations, and feedback from actual implementers.
• raincole 3y ago
  Recently I stumbled upon this issue: https://github.com/joaotavora/eglot/discussions/1127 https://github.com/joaotavora/eglot/discussions/1127 I don't know enough about emacs and LSP to see the full picture, but it seems that both eglot's and corfu's maintainers, assumably very competent programmers, can't find a solution for this. I only skimmed the thread. My understanding is that LSP dumps a long list of completion candidates at once and they can't decide a cache strategy that works well with existing code...? If someone who know more about this please enlighten us.
  • b5n 3y ago
    I'm curious to know what `company` does differently here than `corfu`. As a longtime user I couldn't be happier: https://company-mode.github.io/ https://company-mode.github.io/
    • NoGravitas 3y ago
      It appears to be a cache-busting issue, if I understand the bug discussion correctly. company-capf always busts the completion-in-region cache, corfu never does unless you wrap your completion function in a cache-buster function.
  • jhoechtl 3y ago
    I wonder if this is still the case as most comments petered out end of March. There was a nasty bug discovered in Emacs JSONRPC-Library, which did cause spurious LSP connect errors and likely other issues in LSP server communication https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=8bf4cdcf79bc3254a9169f28f68922ab83bd4e78 https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=8bf4c... Do note that many users might have not been aware of this issue. The fixed library is part of Emacs 29. If you are not on Emacs 29 you might pull in a more recent JSONRPC-package from ELPA in which case it will shadow the inbuilt/pre-shipped library.
    • raincole 3y ago
      I stumbled upon this issue because my eglot still constantly gets out of sync with LSP. (emacs 29.1, latest stable corfu and eglot)
• deleted 3y ago
  [deleted]
• fiddlerwoaroof 3y ago
  The way a language server should work, imo, is it holds all the code and serves projections of the code on the fly to the editor. The user of the editor makes changes to the code and commits it back, at which point the language server parses the code and integrates it into the codebase, pretty-printing it to files as necessary for source control. But, the file layout should be an implementation detail that the editor knows nothing about.
  • slimsag 3y ago
    That's exactly how LSP is designed. The editor e.g. requests 'the user is hovering over code <here>, what should I show?' and the server responds with some Markdown text. Or 'the user wants to navigate to the thing under their cursor <here>, where should I go?' and the server responds with a file/line. Once you try to use LSP for anything not in that form.. it's not so fun.
    • fiddlerwoaroof 3y ago
      No, this is the opposite of how LSP is designed: LSP assumes the editor is looking at a file and this results in a complicated sync dance between editor state and file state. I want the editor to look at temporary buffers served up from the language server and have the code be “written” by sending the temporary buffer back to the language server which handles writing it out to files itself. As far as I know, the only languages with something even vaguely like this are Pharo Smalltalk (with its git integration) and unison
      • fiddlerwoaroof 3y ago
        The advantage here is this enables some interesting workflows, like “show me this method and every definition that overrides it in a single view” or “show me this method and all the methods it calls directly”. So you can project the interesting part of your code base into a temporary editor buffer and then edit it and the language server takes care of persistence. Also, it ends the formatting wars because the in-repository format is disconnected from the user’s preferred format.
• hgs3 3y ago
  > The problem, column is counted using UTF-16 code units. Rather than fixing the problem by using code points, they made it worse by allowing any encoding. Now, not only do you need to support UTF-16, but also UTF-8 and UTF-32 [1]. [1] https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#positionEncodingKind https://microsoft.github.io/language-server-protocol/specifi...
  • sillysaurusx 3y ago
    Or use UTF-8 everywhere to fix the problem. Code points have their own issues (as does UTF-8, but on balance these seem like better engineering tradeoffs).
    • celeritascelery 3y ago
      Using UTF-8 only has better engineering trade offs if you represent text as UTF-8. If you don’t (like JavaScript) then it’s just extra complexity. It’s no better than UTF-16. Only advantage it has is it’s more common among newer languages. Code points is the only thing that makes sense for a multi-language protocol. It’s unambiguous and every Unicode client can talk in code points, even if they use some exotic encoding.
  • chrismorgan 3y ago
    You don’t need to; UTF-16 is mandatory (ugh), but support for other encodings is negotiated (see ClientCapabilities.positionEncoding and ServerCapabilities.positionEncoding).
    • layer8 3y ago
      This seems somewhat pointless, because for compatibility you need to support UTF-16 anyway, so why bother implementing any of the other encodings in addition.
      • chrismorgan 3y ago
        Because if the server and client both use UTF-8 internally, they can get better performance by agreeing to use UTF-8.
• kstrauser 3y ago
  I’m fighting with it right now. LSP specifies several configuration mechanisms but there isn’t a “blessed” one. This is a pain point because: I want to write Python. There’s one FOSS language server (pylsp) that’s a better fit for my needs than the other Python servers. I want to use a certain commercial editor. The editor and the language server don’t share a common configuration method. Ergo, I can’t use the editor I’d like to write Python the way I want. It’s no one’s fault. The editor works just fine with other servers, for Python and for lots of other languages. The server is also operating within spec, and plenty of editors support its configuration method. However, they don’t play well with each other. I can still use the editor with that server as long as I’m ok with the default settings, but I’m not. It would’ve been spiffy to have a preferred baseline method that all editors and language servers spoke, but they went the enterprisey “here are a hundred options - pick a few!” overly configurable direction. Bummer. I really, really like that editor.
  • foota 3y ago
    What is configuration in this case?
    • kstrauser 3y ago
      In this specific case, enabling specific plugins, configuring their behavior, adjusting maximum line lengths, that sort of thing.
      • duped 3y ago
        That seems out of scope of what LSP deals with
        • kstrauser 3y ago
          That’s exactly the kind of thing LSP deals with. You have to get those settings into the server somehow, and the main methods are for the editor to send them to the server, or for the server to load them from somewhere in the filesystem or similar. A lot of those settings (like line length) are configured in the editor by the user. It makes a lot more sense to have a mechanism to communicate them over the same protocol than to use a janky method like writing to a file with a specific name, or some other out-of-band method.
• dom2 3y ago
  One thing that has annoyed me with some personal implementation work is why does DAP use a cosmetically similar but not following spec version of JSONRPC? Why not just use JSONRPC especially considering LSP already does?
  • schemescape 3y ago
    I would assume DAP predates (and possibly motivated) JSONRPC (edit: so it might have been working off of a draft spec). Hopefully someone knowledgeable can comment.
    • schemescape 3y ago
      Note: This was definitely not correct! JSON-RPC apparently started in 2005.
  • duped 3y ago
    Because of organizational disorganization within the VS Code development teams. LSP and DAP are basically the VSCode APIs for implementing language support and debuggers but bolted onto an RPC layer without synchronization or consistency between the people/teams that develop them.
    • sakras 3y ago
      Conway’s law at its finest!
  • durbat 3y ago
    This post explains the reasons https://github.com/microsoft/vscode-debugadapter-node/issues/58#issuecomment-246387621 https://github.com/microsoft/vscode-debugadapter-node/issues...
• duped 3y ago
  I feel like the RPC protocol (minus a good text representation) that LSP needs is ONC RPC. It's ancient, to be sure, but it is stupid simple to implement either with a ton of boilerplate or with a compiler to generate the structs and serializers/deserializers for you.
• bumbledraven 3y ago
  Jonathan Blow criticized the performance and complexity of LSP in his talk at DevGAMM 2019 titled "Preventing the Collapse of Civilization" (timestamp 42m26s; https://youtu.be/pW-SOdj4Kkk?t=2546 https://youtu.be/pW-SOdj4Kkk?t=2546): > In the programming language world, there's this thing called Language Server Protocol that is pretty much the worst thing I've ever heard of. There are proponents of this all over, building systems for it right now that are going to be living on your computer tomorrow, or maybe even today. As far as I can tell, it's basically a more complicated, slower way to do libraries. > Say you've got an editor for some programming language and you want to be able to do stuff that we've been doing for decades already. For example, look up the declaration of an identifier by clicking on it, or have tooltips that say, "What type is this value?" Well, they say the way you should do that is—you know, you have your editor and then it's a hassle to make plugins. This is the made-up problem. It's a hassle to make plugins for all these different things. So in order to standardize, you're going to run a server on your machine. Then your editor talks over a socket to the server, and the server talks back and gives you the answer. This approach has now turned your single program into a distributed system. > The flaw in this whole line of thinking, that none of these people seem to actually think about, is that there's nothing special about looking up the location of an identifier in your code. That's just an API, like we have all the time for everything. So the obvious next step, if you're saying that we should architect our APIs like this, is to do this for other tasks. Now your editor, or whatever program, is going to be talking to multiple of these things. If you ever want to author anything for this, you now have to author and debug components of a distributed system where state is not located in any central place. We all know how fun that is, right? > But of course, libraries are not that simple. Libraries use other libraries. So what happens at that point is you're running all these servers on your system, and who knows, some of them are going to go down and have to restart. People are synchronizing with each other—no, this is a disaster. And people are actively building this right now, while we're spending all this time overcomplicating stuff that we used to be able to do in 1960. (Transcript produced by asking GPT-4 to clean up the raw output from youtubetranscript.com: https://chat.openai.com/share/e1bc0e87-f79e-4958-98e7-b93c1b510a21 https://chat.openai.com/share/e1bc0e87-f79e-4958-98e7-b93c1b...)
  • deleted 3y ago
    [deleted]
• __MatrixMan__ 3y ago
  I hope we're nearing the end of the era where we're editing text in a bubble of tooling that understands ASTs but has to map everything back to text. I look forward making tabs vs spaces a viewer setting and not an author choice.
  • raincole 3y ago
    People have kept saying this since paredit...
    • __MatrixMan__ 3y ago
      Yes, but we've been getting closer since then. Unison, for instance, is a step in this direction.
  • ianlevesque 3y ago
    Agreed. Even beyond that it would be great for merging & conflict resolution too.
    • __MatrixMan__ 3y ago
      Right. Line-wise diff is such a crude instrument.
      • mfashby 3y ago
        You can already get some tools e.g. https://difftastic.wilfred.me.uk/introduction.html https://difftastic.wilfred.me.uk/introduction.html which try to do semantic rather than purely textual diffs
  • akhosravian 3y ago
    Tabs vs spaces has been a viewer setting for decades for tab users.
    • globular-toast 3y ago
• oaiey 3y ago
  I do not follow the argumentation. LSPs are created for text editors. They are focused on lightweight text editor presentation. If you want to expose AST, you either start implementing clients again (defeating the original purpose) or you start being 3-7 levels down the abstraction madness. You can do that and projecting ASTs instead of representing text (looking at you JetBrains MPS) but even the best IDE companies in the world has abondon that in favor of being text based. With that LSP would not solve the n x m problem of editors to languages but would require all text editors to reinvent themselves into something completely different.
  • cmrdporcupine 3y ago
    Part of the problem is that the LSP protocol & model simply can't work with systems that aren't text-file-based; like a Smalltalk, say. Or stored procedures in a database. Or other heterodox models. It makes the (probably reasonable) assumption that all development happens in file-based text editors, and imposes the text editor model. That's fine, but there are other ways to organize code objects, and the LSP ecosystem is mainly useless to them. Which is frustrating for the people working in or on that model, but maybe of no concern to others and probably no reason VSCode etc would want to have focused on the possibilities of that alternative. But it means people doing the heterodox will have to reinvent the wheel completely instead of partially.
    • oaiey 3y ago
      But that is something you can fix when you virtualize the file system. In the end it is text. And I think there is some movement. See other comments.
• Ozzie_osman 3y ago
  I've read a bunch of articles about LSP, and some of the docs, but still not entirely sure what it is. I mean, abstractly I understand, but to truly understand what it's capable of doing and when I might need to use it... Does anyone have any good pointers?
  • kstrauser 3y ago
    At a high level, it’s a standard way for an editor to ask another program for information about a source code file: What are the function definitions here? What are the available completions for this text? What errors are there? Can you reformat this in standard way? Where is the thing one line 23, row 41 defined? Instead of every single editor having to understand C and Python and JavaScript and Rust and SQL and Ruby, they only have to know how to talk to a language server using a standard protocol. And if you’re writing a new language, you can make a language server for it, and then every editor that knows how to use LSP can be used to comfortably write code in that new language. The old way is that M editors want to support N languages, someone has to write M times N language parsers. With LSP, they can write N. That’s way less work.
  • ww520 3y ago
    LSP stands for Language Server Protocol. It's a communication protocol between the clients (text editors, debuggers, source viewers, etc) and the language servers. Typically a language compiler runs over the source files and terminates once it's done. All the compilation states are gone. The language server is the compiler itself running for a long time; its aim is to maintain the compiled states of the source files alongside of the editing session in the editor client. It's ready to answer requests from the editor in regarding the source code. E.g. The editor loads a source file. It sends a LSP request to the language server saying the file is opened. The language server would compile the file and maintain its compiled states. The compilation process might involve other source files as well. The language server reports any errors or warnings back to the editor. The editor shows them to the user right the way. The user modifies a function and hits save. The editor sends a file-changed request to the language server. It recompiles the file and sends back any errors or warnings. The user places the cursor at a function call and issues the jump-to-definition command. The editor sends a LSP request to the language server to look up the location of the definition of the function. The language server has all the compiled states of the source files and knows where the function definition is. It sends back the target location (file and line number). The editor can switch to the target file and jump to the line number. LSP defines a whole bunch of these requests and responses (e.g. looking up the type of a variable, definition of a value, auto-completion suggestion of function parameters, etc). A language server for a language can implement these and suddenly all the clients that talk LSP can utilize the features.
• KirillPanov 3y ago
  "How" did the word "how" end up in the HN title for this? Spin?
• grumpyprole 3y ago
  The biggest problem with LSP is that it's a lowest common denominator solution. If I try to edit OCaml using an LSP solution, I don't get features like "query type of symbol", presumably because JavaScript doesn't need it.
  • pxc 3y ago
    Wait, does that feature mean what is sounds like? That sounds too basic to be fundamentally beyond LSP.
    • _flux 3y ago
      Yeah, it sounds like it should support it. Traditionally though OCaml editor integrations have also supported not only asking just the type of a symbol, but of an expression. I wonder if LSP can do that, because that function needs some interactive scoping of the query, not just a single point, or I suppose it can work if hovering over parenthesis but if precedency needs to be accounted for, it would be difficult to understand what the user wants to see. I've _really_ enjoyed the expression type queries in the past, but I haven't coded OCaml for a while :/.
      • skitter 3y ago
        That's doable in LSP - the server knows about the selection, not just the cursor position. E.g. if I use rust-analyzer (in Helix, so there aren't any protocol extension in play), select an expression and request hover info, rust-analyzer shows the expression's type.
  • teaearlgraycold 3y ago
    [flagged]
  • runeks 3y ago
    Hovering over the name of a variable in Haskell shows the type. It sounds like your OCaml LSP server just hasn't implemented this yet.
  • 3y ago
• Syzygies 3y ago
  Wow. There was a tiny typo. I found myself thinking, someone who reasons like this should have a protocol for fixing typos, if they eat their own dog food. I was floored to see the "fix typo" link at the bottom. Submitted. I now wonder if the typo was on purpose.
  • matklad 3y ago
    No, I am just spelling-impaired (even in my mother tongue) :)
    • DiabloD3 3y ago
      As long as you keep writing, its all good.
    • Syzygies 3y ago
      I was expressing amazement that you took the time to provide a "fix typo" mechanism. In the real world, many of us would take the time to fix a splinter in a well-traveled floor. I love that your appreciation for robust protocol includes a mechanism for moving the digital world in that direction.
  • junon 3y ago
    This is a strange position to take. "Someone who uses logic and reasoning should churn out pristine, flawless work."
    • Syzygies 3y ago
      You misunderstand me. I am amazed by the author's attention to robust protocol design. No one wants to be "that person" who observes a missing space before an open parenthesis, so usually smoothing out flawless work is not a collaborative process. I found myself daydreaming how blogs would work if the author turned their brilliant attention to the process of blogging itself. They would solve such a problem, to encourage collaboration. I was therefore stunned to see a "Fix Typo" link at the bottom. Perhaps this has become widespread, but I'd never noticed it before, and the logic of having this author provide such a link amazed me. Part of what we do in comments is appreciate various aspects of a work. I was appreciating this aspect of this work.
• junon 3y ago
  LSP is just Microsoft's EEE[0] into the open source space, as they've done with a number of things (including Typescript and buying GitHub/NPM, WSL, etc). Visual Studio was a meme for a long time because it was so heavy and enterprisey, so to capture more market share they made the cool new shiny version VS Code, and wanted to compete with e.g. Atom whilst still having their grip on the ecosystem as a whole. So LSP was born. It has a lot of problems. It only really works well with VS Code (yes, there are exceptions, but not many, and they aren't paid for by a mega corporation). This is Microsoft's way. They try to sway the developer community through force, not persuasion. [0] https://en.m.wikipedia.org/wiki/Embrace,_extend,_and_extinguish https://en.m.wikipedia.org/wiki/Embrace,_extend,_and_extingu...
  • throwaway50182 3y ago
    What exactly did they embrace and extend with LSP? And what's going to get extinguished? They created it and others adopted it... They could've simply integrated TypeScript into VSCode using a proprietary protocol to keep people locked to it. Instead they took care to make a well documented and reasonably open protocol that others can use for their own language support, and to integrate TypeScript into other IDEs. I really don't understand what you think they should've done instead. Do you think that the world where every compiler has its own API is better? You're still free to do so - just develop an extension for VSCode and/or all the other IDEs instead of a single LSP server implementation.
    • Xophmeister 3y ago
      To be cynical, I guess we're at the "embrace and extend" part; "extinguish" hasn't happened yet, but it could. Microsoft controls the LSP and VS Code is very popular; thus Microsoft has all the leverage in the LSP space. I'm not sure what "going rogue" would look like, on Microsoft's part, but with all the power, it could do so without adversely affecting itself.
      • eddythompson80 3y ago
        To be more cynical all products from all companies are in the “embrace and extend” phase
• uoaei 3y ago
  An article about LSP should really start by defining LSP.
• zubairq 3y ago
  Very interesting article and comments!
• silok 3y ago
  Overall very informative. Really like the comparison with the Dart analytics protocol and the Jetbrains protocol. I think the comments on state synchronization with subscription based feature providers sounds very promising. Hopefully LSP specification can evolve over time and incorporate some of these ideas!
• elashri 3y ago
  I never understood LSP in a good manner. Can someone explain it to me and how it differs?. Shouldn't the protocol be a standard way to implement things like completions for different languages?
• nerdix 3y ago
  I've always found it interesting that MS basically started LSP but C# has probably the worst LSP implementation I've ever seen. It's flat out broken. Rust, Go, Dart, etc all vastly superior. I recently opened up a powershell script in nvim and even powershell's LSP is better. I guess there is a new C# LSP with C# Dev Kit and it seems better in vs code but I haven't tried to use it with nvim yet.
• laurmaedje 3y ago
  I have a question about the point on highlighting in particular. With a subscription-based model, don't you lose out on the ability to partially highlight a file based on the visible ranges in the editor? Unless you re-subscribe to the highlighting whenever the visible ranges change, the language server wouldn't know which slice(s) of the document to highlight after a change comes in.