9 ms·

Go team adding a Go language server to core

218 points by sqs 8y ago

wenc 8y ago
Curious about language servers (especially those implementing LSP): how do they achieve low-latency responses? My understanding is that JSON-RPC is used, which is relatively high overhead. Are there lighter weight alternatives? (gRPC?)
- tragic 8y ago
  Because they're typically running locally they're quick, or at least as quick as your hardware will allow. The json-related latency is typically dwarfed by the latency from actually doing the functionality (parsing the code for code completion, etc). So the real work is in having a good story for incremental compilation and a generally quick and robust compile toolchain. I guess that Go is in a good position here ...
- int_19h 8y ago
  The key word is "relatively". You're comparing it to protocols that are designed to process much heavier loads than what typing generates. And don't forget that processing is usually asynchronous, and can be spun off to a separate thread, so you get one just for protocol handling.
- scns 8y ago
  The xi editor uses json to communicate with its backend. The author wrote about negligible overhead
wasd 8y ago
I wish every language would do this.
flaque 8y ago
Possibly dumb question: what is a language server in this context and what’s it used for outside of the AST package?
- blcArmadillo 8y ago
  Look at https://microsoft.github.io/language-server-protocol/ https://microsoft.github.io/language-server-protocol/ for more info. It's a standard interface IDEs can use for things such as autocomplete rather than every IDE having to reinvent the wheel.
  - int_19h 8y ago
    To add to this: the beauty of the protocol is that it uses JSON over stdin/stdout for communication. So if your language can make console apps, you can write a language server in it that works with any compliant editor/IDE - no FFI necessary, no need to learn how to author extensions for that editor etc. And, conversely, if your editor/IDE can fork a child process and redirect its standard streams, you can support any language server and not care about how it's implemented. I think this more than anything else has contributed to its rapid adoption, especially on Unix-likes.
    - hawski 8y ago
      Why was this downvoted? Is it not factual? If it's not, it would be far more helpful if it would be pointed out. If it's implemented as stated by int_19h then it's my favourite interface - stdio instead of FFI. So one could implement a server even in Brainfuck.
      - int_19h 8y ago
        The protocol itself is transport-agnostic - it just needs a bidirectional stream, and the spec doesn't define what it is. In practice, both stdin/stdout and sockets are used - the latter usually for language servers that are "heavyweight", such that it is advantageous to spin up the server once, "prime" it (let it do any indexing etc), and then reuse it between concurrent sessions. The standard streams are usually the default choice for simplicity reasons, though. Implementing a server in Brainfuck is theoretically possible, but only if you ignore the heat death of the universe as the upper time boundary. ~
  - 8y ago
dev_dull 8y ago
> Tooling is so important to the developer experience for any language, and a language server is one of the most important kinds of tooling there is (because the user interacts with it on virtually every keystroke in their editor). Here’s how languages backed by large corporations win. They have the resources and the optics to know why these non-fun features are important.
- stcredzero 8y ago
  Since when is tooling non-fun? This depends on a lot of factors.
  - asdkhadsj 8y ago
    Exactly. Hell, I consider tooling to be some of the most fun (and often difficult) things I work on in my free time. If I can make something easier for my normal dev environment I see huge gains ~40 hours a week. That's pretty damn awesome and flashy to me. It's definitely not easy, but imo it's well worth my time.
  - ehsankia 8y ago
    I agree that tooling can be fun. What probably many more people would agree is non-fun is good documentation.
  - kjeetgill 8y ago
    I'd say tooling is so much fun it derails most of my pet projects. This is why there are 500 different static site generators produced a week.
  - hayden592 8y ago
    I actually prefer to write dev tools. Implementing my product manager next feature will always be less satisfying than helping a co-worker.
- georgewfraser 8y ago
  As someone who has worked on a Java language server [1] on-and-off for 2.5 years, for fun, I disagree that tooling is not fun :) [1] https://github.com/georgewfraser/vscode-javac https://github.com/georgewfraser/vscode-javac
asdkhadsj 8y ago
Does anyone know if this is introducing more features than the existing Go LSP[1]? I'm all in favor of tooling support - I'm just curious if this actually adds anything to those of us already using a Go LSP. Thoughts? [1]: https://github.com/sourcegraph/go-langserver https://github.com/sourcegraph/go-langserver
- froh 8y ago
  If I'm reading the linked article right this is about moving exactly the go language server you are referring to into the Google maintained set of go core libraries and tools. So this first is independent from which features it supports. Now as it is open source, you bet both Google and sourcegraph will continue adding features as appropriate.
- __float 8y ago
  I don't think this adds anything currently -- but the comment this post links to is by a Sourcegraph engineer :)
- deleted 8y ago
  [deleted]
- sqs 8y ago
  Sourcegraph CEO here (and author of a comment in the link + contributor to https://github.com/sourcegraph/go-langserver https://github.com/sourcegraph/go-langserver). The idea is that with a Go language server becoming a core part of Go, it will have a lot more resources invested into it and it will surpass where https://github.com/sourcegraph/go-langserver https://github.com/sourcegraph/go-langserver is now.
adiusmus 8y ago
Every language has support ... except the ones that don’t. https://langserver.org/ https://langserver.org/
vbezhenar 8y ago
It's interesting that Jetbrains refuses to leverage those language servers and writes their own parsers for every supported language.
- gameswithgo 8y ago
  very few languages implement the RLS well. For instance Rust does it, but its incomplete and very slow. I bought a quieter CPU fan specifically because of it!
  - int_19h 8y ago
    It's more a function of language complexity than anything else. LSP for something like C, Java or TypeScript is not that hard, because the language is fairly straightforward to parse, and not ambiguous. But something like C++ or Rust, where you have to deal with templates/generics/traits/... and type inference starts getting gnarly. And something like JS or Python is hard because there are no static types at all, and you have to do inference. But LSP itself doesn't really make it easier or harder. If you look at other code completion implementations (in IDEs that don't use LSP), they're usually at a comparable level, if they exist.
  - jayflux 8y ago
    Have you used RLS recently? It’s improved quite a bit for me
- CzechTech 8y ago
  Good. Great even. If the implementation is top notch, mantained and from the 1st party, why the hell not, but most of the times, language server implementations are half assed, slow, "kinda" working messes that are all but useless in the long run. The PHP language server implementations are a great example of this. A buch of broken, incomplete, underperforming projects maintained by a single person (or straight up abandoned). Compare that with Phpstorm and you have just compared a rusty tricycle with a spaceship.
  - mixmastamyk 8y ago
logicallee 8y ago
2 unrelated questions. (tl;dr at bottom) Background ---------- The two major advantages of Go are its clear, unambiguous and explicit syntax (unfortunately ugly) and very small (minimal) feature set, so that it is possible to reason about anything without any high context. The former means it is possible to pick up Go in a day or two and the latter means that there are no real tricks after a few months. That's it. All this has been done in an excellent way. There is no class system. Everything is simple. Explicit typing works. These design trade-offs have led to a language that is VERY easy to go back to after working in any other language. Almost all simple syntax mistakes by the programmer are caught before something successfully compiles. Once a programmer is used to it, which happens quickly, Go is the only language people write pages of code in that compiles on the first pass and does everything they hoped. "Less is exponentially more." History ------- Due to the history of Go it was meant to be a systems language. It didn't really succeed at replacing C anywhere, except the web server: I consider Go like a web-safe C. (In no small part due to Google's work meeting its own needs.) Question 1 ---------- What I don't understand is that since speed and being a systems language were explicit design goals, why the Go syntax is not suitable for tiny, embedded microcontrollers. If the answer is something about garbage collection, this doesn't really make sense for me, as good algorithms should scale from 0 upward. I don't see why there should be a huge jump from "not running Go yet" to "now you're running a Go runtime" that suddenly requires a large amount of free memory. Here is a post about this: https://medium.com/samsara-engineering/running-go-on-low-memory-devices-536e1ca2fe8f https://medium.com/samsara-engineering/running-go-on-low-mem... It seems this is high. I don't understand why Go doesn't take its core, primary strengths outlined in the "background" section above and expand it into the lowest of the low-end devices. Why can't I compile a Go program targeting a microcontroller with 2-8 MB of RAM without issue? Go was designed as a compiled systems language, to replace C. Not as a scripting language to replace Python. Question 2 ---------- Since explicit (if very ugly) syntax that keeps people from making mistakes, and whose specifications can be kept in mind all at once, is a strength of the language and syntax, why can't I develop desktop GUI's in Go? It is a fast, compiled language. You might say the answer to that is that a windowing system needs a class system but I disagree - that can't possibly be any more true than the idea that a web app needs a class system. Clearly people make web apps in Go without a class hierarchy. Nobody makes desktop apps (I mean the class of applications that web browsers, IDE's, photo editing software, basically everything on a list like this - https://helpdeskgeek.com/free-tools-review/best-freeware-programs-for-windows/ https://helpdeskgeek.com/free-tools-review/best-freeware-pro... ) in Go. Of course, when it debuted you could say, "That's just not what the team focused on", since it was backed by Google they needed it on the server. But at some point you'd expect first someone to make a make-do GUI toolkit, then other people to add on and make something better, and by now it should be an obvious language for making a Winzip or Winrar competitor. Or an antivirus. Or an Audacity or VLC-type program. Or an OpenOffice clone. But nobody does these things. Any of these things. Why not? The simple language has nothing missing that languages which are used for these applications have. I don't buy the idea of the ecosystem entirely lacking these. Why don't people make GUI stuff in Go? I don't understand why it shouldn't be an obvious language choice. I just wouldn't reach for Go when it comes to scripting a GUI application. Why not? Summary / tl;dr --------------- 1. I would like to know what it would take to make the simple specifications behind Go, which are protected by knights in shining armor, an obvious choice for embedded microcontroller applications in 2018. 2. I would like to know what it would take to make the simple specifications behind Go, which are protected by knights in shining armor, an obvious choice for desktop applications in 2018.
shabbyrobe 8y ago
I'm not sure I fully understand the provenance of the code here, but I could just be being a bit thick. The files in the change request show the copyright as "2018 The Go Authors", but some of the commentary hints that this is code donated from the Sourcegraph implementation. If this is the case, does that mean Sourcegraph has transferred the copyright to the Go authors?
pokstad 8y ago
I’m going to guess this is in response to many community supported dev tools breaking in v1.11 when modules are turned on. Specifically tools like gocode, godef, and gogetdoc.
fithisux 8y ago
This is great news. Hopefully we will see others follow.
mark_l_watson 8y ago
Good idea. Languages like Haskell benefit by having a language server that Emacs, VisualCode, etc. can all use. It just make good sense to do the work once and let all editors and IDEs use it.
lobo42 8y ago
Cool! We wrapped the VS Code extension in the protocol to support it in gitpod.io. See here https://github.com/theia-ide/go-language-server https://github.com/theia-ide/go-language-server Would be really awesome to use something maintained by the people who know Go best :)

218 points by sqs 8y ago

wenc 8y ago

Curious about language servers (especially those implementing LSP): how do they achieve low-latency responses? My understanding is that JSON-RPC is used, which is relatively high overhead. Are there lighter weight alternatives? (gRPC?)

tragic 8y ago
Because they're typically running locally they're quick, or at least as quick as your hardware will allow. The json-related latency is typically dwarfed by the latency from actually doing the functionality (parsing the code for code completion, etc). So the real work is in having a good story for incremental compilation and a generally quick and robust compile toolchain. I guess that Go is in a good position here ...
int_19h 8y ago
The key word is "relatively". You're comparing it to protocols that are designed to process much heavier loads than what typing generates. And don't forget that processing is usually asynchronous, and can be spun off to a separate thread, so you get one just for protocol handling.
scns 8y ago
The xi editor uses json to communicate with its backend. The author wrote about negligible overhead

wasd 8y ago

I wish every language would do this.

flaque 8y ago

Possibly dumb question: what is a language server in this context and what’s it used for outside of the AST package?

blcArmadillo 8y ago
Look at https://microsoft.github.io/language-server-protocol/ https://microsoft.github.io/language-server-protocol/ for more info. It's a standard interface IDEs can use for things such as autocomplete rather than every IDE having to reinvent the wheel.
- int_19h 8y ago
  To add to this: the beauty of the protocol is that it uses JSON over stdin/stdout for communication. So if your language can make console apps, you can write a language server in it that works with any compliant editor/IDE - no FFI necessary, no need to learn how to author extensions for that editor etc. And, conversely, if your editor/IDE can fork a child process and redirect its standard streams, you can support any language server and not care about how it's implemented. I think this more than anything else has contributed to its rapid adoption, especially on Unix-likes.
  - hawski 8y ago
    Why was this downvoted? Is it not factual? If it's not, it would be far more helpful if it would be pointed out. If it's implemented as stated by int_19h then it's my favourite interface - stdio instead of FFI. So one could implement a server even in Brainfuck.
    - int_19h 8y ago
      The protocol itself is transport-agnostic - it just needs a bidirectional stream, and the spec doesn't define what it is. In practice, both stdin/stdout and sockets are used - the latter usually for language servers that are "heavyweight", such that it is advantageous to spin up the server once, "prime" it (let it do any indexing etc), and then reuse it between concurrent sessions. The standard streams are usually the default choice for simplicity reasons, though. Implementing a server in Brainfuck is theoretically possible, but only if you ignore the heat death of the universe as the upper time boundary. ~
- 8y ago

dev_dull 8y ago

> Tooling is so important to the developer experience for any language, and a language server is one of the most important kinds of tooling there is (because the user interacts with it on virtually every keystroke in their editor). Here’s how languages backed by large corporations win. They have the resources and the optics to know why these non-fun features are important.

stcredzero 8y ago
Since when is tooling non-fun? This depends on a lot of factors.
- asdkhadsj 8y ago
  Exactly. Hell, I consider tooling to be some of the most fun (and often difficult) things I work on in my free time. If I can make something easier for my normal dev environment I see huge gains ~40 hours a week. That's pretty damn awesome and flashy to me. It's definitely not easy, but imo it's well worth my time.
- ehsankia 8y ago
  I agree that tooling can be fun. What probably many more people would agree is non-fun is good documentation.
- kjeetgill 8y ago
  I'd say tooling is so much fun it derails most of my pet projects. This is why there are 500 different static site generators produced a week.
- hayden592 8y ago
  I actually prefer to write dev tools. Implementing my product manager next feature will always be less satisfying than helping a co-worker.
georgewfraser 8y ago
As someone who has worked on a Java language server [1] on-and-off for 2.5 years, for fun, I disagree that tooling is not fun :) [1] https://github.com/georgewfraser/vscode-javac https://github.com/georgewfraser/vscode-javac

asdkhadsj 8y ago

Does anyone know if this is introducing more features than the existing Go LSP[1]? I'm all in favor of tooling support - I'm just curious if this actually adds anything to those of us already using a Go LSP. Thoughts? [1]: https://github.com/sourcegraph/go-langserver https://github.com/sourcegraph/go-langserver

froh 8y ago
If I'm reading the linked article right this is about moving exactly the go language server you are referring to into the Google maintained set of go core libraries and tools. So this first is independent from which features it supports. Now as it is open source, you bet both Google and sourcegraph will continue adding features as appropriate.
__float 8y ago
I don't think this adds anything currently -- but the comment this post links to is by a Sourcegraph engineer :)
deleted 8y ago
[deleted]
sqs 8y ago
Sourcegraph CEO here (and author of a comment in the link + contributor to https://github.com/sourcegraph/go-langserver https://github.com/sourcegraph/go-langserver). The idea is that with a Go language server becoming a core part of Go, it will have a lot more resources invested into it and it will surpass where https://github.com/sourcegraph/go-langserver https://github.com/sourcegraph/go-langserver is now.

adiusmus 8y ago

Every language has support ... except the ones that don’t. https://langserver.org/ https://langserver.org/

vbezhenar 8y ago

It's interesting that Jetbrains refuses to leverage those language servers and writes their own parsers for every supported language.

gameswithgo 8y ago
very few languages implement the RLS well. For instance Rust does it, but its incomplete and very slow. I bought a quieter CPU fan specifically because of it!
- int_19h 8y ago
  It's more a function of language complexity than anything else. LSP for something like C, Java or TypeScript is not that hard, because the language is fairly straightforward to parse, and not ambiguous. But something like C++ or Rust, where you have to deal with templates/generics/traits/... and type inference starts getting gnarly. And something like JS or Python is hard because there are no static types at all, and you have to do inference. But LSP itself doesn't really make it easier or harder. If you look at other code completion implementations (in IDEs that don't use LSP), they're usually at a comparable level, if they exist.
- jayflux 8y ago
  Have you used RLS recently? It’s improved quite a bit for me
CzechTech 8y ago
Good. Great even. If the implementation is top notch, mantained and from the 1st party, why the hell not, but most of the times, language server implementations are half assed, slow, "kinda" working messes that are all but useless in the long run. The PHP language server implementations are a great example of this. A buch of broken, incomplete, underperforming projects maintained by a single person (or straight up abandoned). Compare that with Phpstorm and you have just compared a rusty tricycle with a spaceship.
- mixmastamyk 8y ago

logicallee 8y ago

2 unrelated questions. (tl;dr at bottom) Background ---------- The two major advantages of Go are its clear, unambiguous and explicit syntax (unfortunately ugly) and very small (minimal) feature set, so that it is possible to reason about anything without any high context. The former means it is possible to pick up Go in a day or two and the latter means that there are no real tricks after a few months. That's it. All this has been done in an excellent way. There is no class system. Everything is simple. Explicit typing works. These design trade-offs have led to a language that is VERY easy to go back to after working in any other language. Almost all simple syntax mistakes by the programmer are caught before something successfully compiles. Once a programmer is used to it, which happens quickly, Go is the only language people write pages of code in that compiles on the first pass and does everything they hoped. "Less is exponentially more." History ------- Due to the history of Go it was meant to be a systems language. It didn't really succeed at replacing C anywhere, except the web server: I consider Go like a web-safe C. (In no small part due to Google's work meeting its own needs.) Question 1 ---------- What I don't understand is that since speed and being a systems language were explicit design goals, why the Go syntax is not suitable for tiny, embedded microcontrollers. If the answer is something about garbage collection, this doesn't really make sense for me, as good algorithms should scale from 0 upward. I don't see why there should be a huge jump from "not running Go yet" to "now you're running a Go runtime" that suddenly requires a large amount of free memory. Here is a post about this: https://medium.com/samsara-engineering/running-go-on-low-memory-devices-536e1ca2fe8f https://medium.com/samsara-engineering/running-go-on-low-mem... It seems this is high. I don't understand why Go doesn't take its core, primary strengths outlined in the "background" section above and expand it into the lowest of the low-end devices. Why can't I compile a Go program targeting a microcontroller with 2-8 MB of RAM without issue? Go was designed as a compiled systems language, to replace C. Not as a scripting language to replace Python. Question 2 ---------- Since explicit (if very ugly) syntax that keeps people from making mistakes, and whose specifications can be kept in mind all at once, is a strength of the language and syntax, why can't I develop desktop GUI's in Go? It is a fast, compiled language. You might say the answer to that is that a windowing system needs a class system but I disagree - that can't possibly be any more true than the idea that a web app needs a class system. Clearly people make web apps in Go without a class hierarchy. Nobody makes desktop apps (I mean the class of applications that web browsers, IDE's, photo editing software, basically everything on a list like this - https://helpdeskgeek.com/free-tools-review/best-freeware-programs-for-windows/ https://helpdeskgeek.com/free-tools-review/best-freeware-pro... ) in Go. Of course, when it debuted you could say, "That's just not what the team focused on", since it was backed by Google they needed it on the server. But at some point you'd expect first someone to make a make-do GUI toolkit, then other people to add on and make something better, and by now it should be an obvious language for making a Winzip or Winrar competitor. Or an antivirus. Or an Audacity or VLC-type program. Or an OpenOffice clone. But nobody does these things. Any of these things. Why not? The simple language has nothing missing that languages which are used for these applications have. I don't buy the idea of the ecosystem entirely lacking these. Why don't people make GUI stuff in Go? I don't understand why it shouldn't be an obvious language choice. I just wouldn't reach for Go when it comes to scripting a GUI application. Why not? Summary / tl;dr --------------- 1. I would like to know what it would take to make the simple specifications behind Go, which are protected by knights in shining armor, an obvious choice for embedded microcontroller applications in 2018. 2. I would like to know what it would take to make the simple specifications behind Go, which are protected by knights in shining armor, an obvious choice for desktop applications in 2018.

shabbyrobe 8y ago

I'm not sure I fully understand the provenance of the code here, but I could just be being a bit thick. The files in the change request show the copyright as "2018 The Go Authors", but some of the commentary hints that this is code donated from the Sourcegraph implementation. If this is the case, does that mean Sourcegraph has transferred the copyright to the Go authors?

pokstad 8y ago

I’m going to guess this is in response to many community supported dev tools breaking in v1.11 when modules are turned on. Specifically tools like gocode, godef, and gogetdoc.

fithisux 8y ago

This is great news. Hopefully we will see others follow.

mark_l_watson 8y ago

Good idea. Languages like Haskell benefit by having a language server that Emacs, VisualCode, etc. can all use. It just make good sense to do the work once and let all editors and IDEs use it.

lobo42 8y ago

Cool! We wrapped the VS Code extension in the protocol to support it in gitpod.io. See here https://github.com/theia-ide/go-language-server https://github.com/theia-ide/go-language-server Would be really awesome to use something maintained by the people who know Go best :)