21 ms·

Golang’s most important feature is invisible

117 points by geekydev 5y ago

ferdowsi 5y ago
Well, golang.org does specifically call this out at the top: > Built-in concurrency and a robust standard library I think that the joy of Go's async-first design really makes sense most to people who have experienced the awful pains of colored async-await in other languages.
bestinterest 5y ago
What about project loom? Will that not solve this case providing you start the server with a virtual thread pool? I guess thats what he was getting at however, not having to do that.
- tomohawk 5y ago
  Having used both languages for many years, it was unexpectedly irritating to pick up Java again after doing golang for a few years. So much unneeded complexity to do simple things.
  - azth 5y ago
    What are some examples of unneeded complexity?
    - adgjlsfhk1 5y ago
      All of it? For a specific example, try doing any functional programming in Java and compare it to pretty much any other language.
      - 41b696ef1113 5y ago
        I am far from a Java guy, but that seems like a poor example. Functional programming is not in the wheelhouse of Java. In the same way, it would be unfair to criticize Haskell for its object-oriented style.
      - aphexairlines 5y ago
        Java's not helpful for FP, but that's an odd example to bring up in a discussion about Go, which is also statement-oriented.
      - azth 5y ago
        Incidentally, golang is much worse for FP.
      - kaba0 5y ago
        It really is not that bad, and it is at least possible, both to incorporate it partly (stream api, immutable objects, especially with records) or go full-on with vavr.
    - doctor_eval 5y ago
      While not strictly the language itself, I find the Java standard library API to be unnecessarily verbose, with often obtuse defaults. Go’s library is sometimes overly terse, but after having experienced both in production, I much prefer Go’s style. An example is one of the date parsing functions (I can’t remember details now) which will happily accept an invalid date string (eg 2021-02-35) and silently convert it to a valid date (2021-03-07) despite declaring a parsing Exception. You see, you only get the exception if you call setLenient(false) or if the date string can’t be coerced. For the life of me, I cannot think of a use case for this behaviour, and it certainly violates the principle of least surprise.
      - blibble 5y ago
        the replacement for SimpleDateFormat/Calendar came with Java 8 (released in 2014)
        doctor_eval 5y ago
        Sure, but it is an example of the kind of crappy APIs that littered Java for a long time.
        hbogert 5y ago
        Crappy apis is not beholden to java. Have you looked at go's timer API? Try to re-use one without inadvertently causing a data race, or get wrong behaviour.
        vips7L 5y ago
        Or any of go’s file api’s… https://fasterthanli.me/articles/i-want-off-mr-golangs-wild-ride https://fasterthanli.me/articles/i-want-off-mr-golangs-wild-...
      - eknkc 5y ago
        On the other hand, Go’s time format and parse logic is the biggest pile of shit I’ve ever seen in an API.
    - deleted 5y ago
      [deleted]
    - grey-area 5y ago
      Inheritance.
      - azth 5y ago
        Inheritance does have its place. I've seen verbose and error prone workarounds in golang due to its lack of inheritance.
        grey-area 5y ago
        Even the author of Java, James Gosling, disagrees with you. https://www.infoworld.com/article/2073649/why-extends-is-evil.html https://www.infoworld.com/article/2073649/why-extends-is-evi... As to workarounds for inheritance in Go, that sounds like trying to write in another language but in Go, which is of course not advisable in any language.
        azth 5y ago
        > Even the author of Java, James Gosling, disagrees with you. From the article: > Interface inheritance (the implements relationship) is preferable. You should avoid implementation inheritance whenever possible. Key words: preferable, and whenever possible. It's sometimes preferable to have inheritance and not possible to leave it out without unneeded extra complexity. Furthermore, nowadays interfaces in Java (as well as Kotlin, Scala, C# - all of which are much more expressive and have superior modeling capability than golang) can have default implementations, which increases their application even more and reduces the need for explicit inheritance. Not so in golang since interfaces cannot have default implementations there. > As to workarounds for inheritance in Go, that sounds like trying to write in another language but in Go, which is of course not advisable in any language. Not quite, it's just where having inheritance would have simplified the design, and improved readability.
        grey-area 5y ago
        He quite clearly states in the article he would ‘leave out classes’ (i.e. implementation inheritance). You didn’t quote the clearest statement in the article, just preceding your excerpt!? Again on interfaces, sometimes less is more. Many people prefer the inverted interfaces of go, declared at point of use. It’s fine to prefer the opposite, but these are deliberate choices, not mistakes or failures. Re inheritance simplifying a design, it does the opposite in my experience, unless by simplify you mean hide program flow and state.
- closeparen 5y ago
  Loom is an awesome research project that might begin to be relevant in the actual practice of production Java once it is 1-2 LTS releases old.
  - lostcolony 5y ago
    Yeah; really hard to guess at the effect a feature that hasn't been finalized, has been in progress for years, and has yet to land in a release in any form, will actually have in an environment that is as slow to roll out updates and adopt new features as enterprise Java dev tends to be. Optimistically we may be able to answer op's question sometime in 2024.
travisd 5y ago
My impression of the article is just that the author is a fan of green threads (goroutines). Green threads don’t really make sense without an integration into some kind of IO runtime, so calling out how Go does do this seems moot. Other languages with green threads also with this way (Julia comes to mind).
- karmakaze 5y ago
  The article is not very well informed. It doesn't even make a distinction between os threads and green threads. > To get this performance out of Java you would need to add threadpools, futures or some other async library. What Java needs to be as efficient is the work of Project Loom: Fibers and Continuations. Go also has value types (e.g. arrays and nested structs with inline memory layout) that's the work of Java's Project Valhalla.
- zik 5y ago
  goroutines aren't really green threads, they're more lightweight than conventional green thread implementations due to having a segmented stack. Millions of goroutines can run comfortably on a standard machine. This isn't possible with any green thread implementation I've seen. https://blog.cloudflare.com/how-stacks-are-handled-in-go/ https://blog.cloudflare.com/how-stacks-are-handled-in-go/
  - kortilla 5y ago
    They are cheaper but conceptually aren’t any different.
  - lucian1900 5y ago
    You can run that many Erlang processes or more. Millions is typical of green thread implementations.
  - nasretdinov 5y ago
    By the way, while Go did initially have segmented stacjs, they've long abandoned it in favour of a contiguous one.
bborud 5y ago
It isn’t that it’s invisible. It’s just that you didn’t get it until now. I am not saying that to belittle the author. This is more an observation of how programming seems to have gone from understanding both yourself and what you are doing to shopping bits and pieces and then gluing them together.
- ferdowsi 5y ago
  Not really, "invisible" is a fair way to describe complexity that developers never interact with. Especially when most other popular languages forces developers to deal with irritating async/await bookkeeping when it comes comes to concurrency.
  - zozbot234 5y ago
    How so? The point of async/await is to reduce bookkeeping. The Go model is at its core an implementation of M:N user threads, and it creates plenty of complexity wrt. e.g. interfacing with other languages. Which is why M:N threading was mostly abandoned elsewhere.
    - klabb3 5y ago
      > it creates plenty of complexity wrt. e.g. interfacing with other languages Hmm I'm not sure I get this point. Do you mean in comparison with async? Async cross language interfaces should need the same type of continuation mechanisms (eg poll or completion based interop with an event loop) as green threads, no? I think of async as a primarily syntactically explicit construction, while both look similar in ABI, with a virtual stack. If not, I'd love to know why.
      - int_19h 5y ago
        Async with explicit await is basically just syntactic sugar for continuation callbacks. The "virtual stack" that results can thus be expressed entirely in terms that the C ABI understands: a callback is just a C function pointer + pointer to the struct representing the virtual stack frame.
hahnchen 5y ago
Love the tldr at the start: “ The Go standard runtime allows for asynchronous coding while allowing the developer a simply synchronous interface”
ridiculous_fish 5y ago
How does Go handle syscalls which can't be multiplexed? If I call stat() from 500 goroutines, what does the Go runtime do regarding kernel threads?
- tedunangst 5y ago
  500 kernel threads.
smallerfish 5y ago
Techempower benchmarks beg to differ. https://www.techempower.com/benchmarks/#section=data-r20&hw=ph&test=query https://www.techempower.com/benchmarks/#section=data-r20&hw=... There's a lot of Java above the first occurrence of Go. Also - nobody writes Java web applications using the Sun httpserver. It's verbose and old.
- TheDong 5y ago
  It seems to me like the go ecosystem would rather have a less performant language if it means more similar code between projects. Take for example json: Go's stdlib json implementation is wildly slow. It has to be implemented in reflection, it can't use any code-generation anything, and for backwards compatibility it can never trim any feature, no matter whether it has a performance implication for all users of the package. It's well known to be a slow implementation. In java, json isn't part of the stdlib. There's jackson if you want one flavor of fast json, jetty ships its own json serializer, etc. This means that if you pick up a new java codebase written in a different framework, it's possible you have to learn the right pattern to json serialize things. By nature of these third party libraries in java being out of the stdlib, they've been able to grow and become more performant and compete with each other. Go sits on the opposite side. The stdlib http implementation is a little slow. The stdlib json implementation is incredibly slow. No one wants to build these features outside of the stdlib though, and people would rather every codebase have the same looking go code, even if that means the actual code itself is slower.
  - sofixa 5y ago
    > Go sits on the opposite side. The stdlib http implementation is a little slow. The stdlib json implementation is incredibly slow. No one wants to build these features outside of the stdlib though, and people would rather every codebase have the same looking go code, even if that means the actual code itself is slower. What? There are multiple json parsing and http server implementations for Go.
    - TheDong 5y ago
      They exist (fasthttp and fastjson for example), but no one actually uses them. I should have written "no one wants to use third-party versions of these packages", and I don't think that small semantic issue changes my comment meaningfully. Do you take issue with the way I'm presenting the go ecosystem other than that small pedantic point?
ricardobeat 5y ago
Which part of the example go code looks synchronous? It’s using a callback function. That’s a hallmark of async code, nothing invisible. I also can’t quite grasp the comparison with Java. It’s passing a class instance to the server handler, and that could be doing anything. Nothing there tells you how async or not the server implementation is, and that `os.write()` call might be using a thread pool underneath.
- foolfoolz 5y ago
  agreed. the examples don’t show the real pain. it is when every method returns Future or Promise. and you have to map each result. and many languages don’t have good support for that so you end up with lots of nesting. writing synchronous function signatures is much simpler to maintain. there’s 2 language level syntax features i see that make this better: one is scalas for comprehension. this lets you write async code that reads like imperative synchronous lines. the other is async/await, which let’s you write synchronous code that behaves like async
- recroad 5y ago
  I thought I was the only one who wasn’t getting it.
- kgeist 5y ago
  It's not about the callback at all, the example is poor and the explanation is lacking. In Go, many standard I/O functions use async I/O under the hood. When you call such a function (inside the handler callback, that is), the current goroutine is suspended yielding to other goroutines, so waiting for I/O doesn't block the OS thread inside the request handler. It's basically what you'd do in C# with async/await, except it's implicit and you don't have to mark functions as asynchronous manually. I.e. you write simple synchronous code and Go makes it asynchronous under the hood. There are several disadvantages, though: it's not tunable, and if there's a CGO call (call to a C func) which uses I/O itself it will block an entire worker thread because the Go runtime isn't aware of any of it. As far as I know, Java's new virtual threads have the same idea.
  - spongechameleon 5y ago
jeffbee 5y ago
Unfortunately the batteries-included runtime of Go effectively forecloses any really high performance server code that scales well to huge machines. What it offers is moderately good and that's all you'll ever get. A Go web server will work OK on 8 cores and it will only go a bit faster on 32 cores and not at all faster, possibly slower, on 256 cores. Meanwhile Java async programs are still scaling up. These results are apparent if you look at the gRPC multi-language benchmarks. There's only one way to write a server in Go and that has pros and cons.
- technosamay20 5y ago
  Thanks for sharing this insight. Is this a issue with just the web server. If I have a standalone task, which tires to use 128 cores and manages parallel tasks using goroutines communicating via channels - will I be able to successfully leverage 128 cores.
  - jeffbee 5y ago
    If your tasks are large enough to dominate, then the non-scalability of the goroutine scheduler and the network stack won’t be an issue. For example, if your task is compressing some huge hunk of a video and it takes a second to do it, then it will scale fine. Then again, you can get this scalability from anything, including `xargs`.
- rawoke083600 5y ago
  Interesting.. Any sources or examples ?
recursivedoubts 5y ago
I think what the author is saying is that go has very good async support, but it is no less visible than it is in java, from what I can see. Certainly better syntax, but still visible. I am working on a front end scripting language that does try to make this stuff invisible: https://hyperscript.org/docs/#async https://hyperscript.org/docs/#async It's slow as all get out, but it does, for the most part, hide all the async stuff from the script writer so they can write stuff like: fetch /someurl put the results into #a-div wait 2s transition #a-div's opacity to 0 remove #a-div And all the async stuff in there (the fetch, the wait, the transition) all happen w/o callbacks, async annotations, etc.
- Yoric 5y ago
  That's going to make data race conditions invisible, no? How will developers cope with that?
  - kgeist 5y ago
    Data races happen when there are multiple writers/readers of the same data. If one virtual thread of execution is multiplexed on several OS threads/workers/callbacks/you name it, there's still only one reader/writer at a time, because different parts of a multiplexed function don't run at the same time, they go one after another sequentially as before, with the difference is that there are now implicit continuation points where execution can hop from thread to thread (callback to callback). Can't say for hyperscript, but generally you need to protect your data with the same synchronization primitives as before, such as locks for example. It's only incompatible with thread locals and thread affinity, I'm not sure if JS has those.
    - Yoric 5y ago
      If your data is composite (i.e. a JavaScript object or array) and you have several Promise-based flows of execution (technically not a virtual thread by most definitions of the term, but still a scheduled flow of execution) that end up reading/writing sub-parts of your piece of data in unpredictable order. That's basically a data race, in JavaScript. I've had to debug some of those. As a side-note, some forms of data races (for perhaps a slightly looser definition of the term) can even happen without explicit asynchronicity, just with callbacks/synchronous events.
halpert 5y ago
This type of synchronous code is most impressive to those who come from JavaScript where everything is clearly asynchronous. For instance, in Go you can read a file synchronously using os.ReadFile and it will generally be performant. In NodeJS, you need a callback to achieve the same result.
apatheticonion 5y ago
It's a weird one. The fact that callbacks in Go may be on a different thread means it's so damn easy to use a variable reference from an outer scope within in a callback. This increases the chances that someone inadvertently causes a race condition. Something that requires additional awareness and screening during code reviews. This by itself isn't an indictment of the language as threading is essential to quality GUI applications and as such Go is a great language for writing client applications - which it is largely not used for. You really need to evaluate the value the overhead Go's thread management brings when used in the context of web servers - where we typically split processes up using container orchestration, making them functionally threaded any way. Comparatively, TypeScript running on Node or Deno is single threaded yet asynchronous. Using it as a web server means you have a better type system which follows a similar structural typing model to go (interfaces and such) and you don't have to screen for race conditions. In K8 or ECS, you spawn as many processes as you have CPUs and Bob's your unkle. Personally, I would love it if Go focused on GUI application development but I can _bearally_ write GTK apps with it. Windows and MacOS applications are near-to or impossible to write. Don't even talk to me about Web Assembly. Such a missed opportunity.
- tgv 5y ago
  That’s a weird critique of Go: you make it sound as if you must use threads. But nothing stops you from using callbacks and promises (which have become a bit more comfy with the addition of generics). And if the there’s one domain where I see an advantage in using inheritance, it’s GUIs.
Thorrez 5y ago
Just looking at the numbers, Go is 71% faster. That could be due to the Go HTTP library being more efficient than the Java HTTP library, rather than language or runtime features.
- lenkite 5y ago
  Nobody uses that internal Oracle HTTP server - it is not meant for production. It isn't even in the Java namespace -your static analysers will complain about using 'com.sun.*' packages. Use the Java Jetty server and Go will hide itself in a corner in shame when benchmarked.
  - vips7L 5y ago
    Undertow has better performance for the example program: https://www.reddit.com/r/programming/comments/sanusb/comment/htv5cnu/ https://www.reddit.com/r/programming/comments/sanusb/comment...
est 5y ago
> TLDR: The Go standard runtime allows for asynchronous coding while allowing the developer a simply synchronous interface. I am happy for Golang users but Gevent is more "synchronous" and easier to write.
- kgeist 5y ago
  >I am happy for Golang users but Gevent is more "synchronous" and easier to write. Can you give an example to substantiate your post?
bvrmn 5y ago
Hmm. If you pack real blocking code (instead of writing a couple of bytes) into Go's handler it would definitely turn into crippled turtle under high concurrency load. There is no magic in Go's runtime.
pcwalton 5y ago
Every operating system kernel also presents a synchronous interface to asynchronous I/O. What Go provides is more efficient multithreading in certain cases, primarily because the kernel scheduler doesn't need to be involved on context switches. But this could be fixed at the kernel level with something like Windows' fibers or Google's SwitchTo patches.
frizlab 5y ago
https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ https://vorpus.org/blog/notes-on-structured-concurrency-or-g...
torginus 5y ago
I kind of disagree with the article's premise - that the fact that Go uses async/await like Js and C#, but presents them as green threads is a good thing. Imo it creates false expectations in programmers, who are surprised, that long-running compute code doesn't get preempted, since the compiler doesn't insert preemption points. It also robs us of the possibility, of writing event-driven UI applications, where one can make an asynchronous call on the UI thread safely, which will get marshalled back to the UI thread.
- nasretdinov 5y ago
  By the way, Go does have async preemption. It didn't have it in version 1.0, but it does now. https://medium.com/a-journey-with-go/go-asynchronous-preemption-b5194227371c https://medium.com/a-journey-with-go/go-asynchronous-preempt...
KingOfCoders 5y ago
The author is right in principle, i I think Golangs async scheduler which automatically moves blocking code outside Golangs goroutine scheduler is amazing, but his code and explanations are all wrong. also "Java’s green threads" shows a deep lack of understanding. The author shows a junior level of understanding which co tradicts "I’ve worked professionally with Go for a couple of years"
- PYTHONDJANGO 5y ago
  If you want to show a senior level of understanding coding discussions, please understand that you always should write with a scientific mindset. This means: not only say "x is good" or "x is bad" without explanation like a little child - but also deliver the source of your knowledge or the proof. Then your contributions will be seen as mature additions to collective wisdom.

117 points by geekydev 5y ago

Well, golang.org does specifically call this out at the top: > Built-in concurrency and a robust standard library I think that the joy of Go's async-first design really makes sense most to people who have experienced the awful pains of colored async-await in other languages.

bestinterest 5y ago

What about project loom? Will that not solve this case providing you start the server with a virtual thread pool? I guess thats what he was getting at however, not having to do that.

tomohawk 5y ago
Having used both languages for many years, it was unexpectedly irritating to pick up Java again after doing golang for a few years. So much unneeded complexity to do simple things.
- azth 5y ago
  What are some examples of unneeded complexity?
  - adgjlsfhk1 5y ago
    All of it? For a specific example, try doing any functional programming in Java and compare it to pretty much any other language.
    - 41b696ef1113 5y ago
      I am far from a Java guy, but that seems like a poor example. Functional programming is not in the wheelhouse of Java. In the same way, it would be unfair to criticize Haskell for its object-oriented style.
    - aphexairlines 5y ago
      Java's not helpful for FP, but that's an odd example to bring up in a discussion about Go, which is also statement-oriented.
    - azth 5y ago
      Incidentally, golang is much worse for FP.
    - kaba0 5y ago
      It really is not that bad, and it is at least possible, both to incorporate it partly (stream api, immutable objects, especially with records) or go full-on with vavr.
  - doctor_eval 5y ago
    While not strictly the language itself, I find the Java standard library API to be unnecessarily verbose, with often obtuse defaults. Go’s library is sometimes overly terse, but after having experienced both in production, I much prefer Go’s style. An example is one of the date parsing functions (I can’t remember details now) which will happily accept an invalid date string (eg 2021-02-35) and silently convert it to a valid date (2021-03-07) despite declaring a parsing Exception. You see, you only get the exception if you call setLenient(false) or if the date string can’t be coerced. For the life of me, I cannot think of a use case for this behaviour, and it certainly violates the principle of least surprise.
    - blibble 5y ago
      the replacement for SimpleDateFormat/Calendar came with Java 8 (released in 2014)
      - doctor_eval 5y ago
        Sure, but it is an example of the kind of crappy APIs that littered Java for a long time.
        hbogert 5y ago
        Crappy apis is not beholden to java. Have you looked at go's timer API? Try to re-use one without inadvertently causing a data race, or get wrong behaviour.
        vips7L 5y ago
        Or any of go’s file api’s… https://fasterthanli.me/articles/i-want-off-mr-golangs-wild-ride https://fasterthanli.me/articles/i-want-off-mr-golangs-wild-...
    - eknkc 5y ago
      On the other hand, Go’s time format and parse logic is the biggest pile of shit I’ve ever seen in an API.
  - deleted 5y ago
    [deleted]
  - grey-area 5y ago
    Inheritance.
    - azth 5y ago
      Inheritance does have its place. I've seen verbose and error prone workarounds in golang due to its lack of inheritance.
      - grey-area 5y ago
        Even the author of Java, James Gosling, disagrees with you. https://www.infoworld.com/article/2073649/why-extends-is-evil.html https://www.infoworld.com/article/2073649/why-extends-is-evi... As to workarounds for inheritance in Go, that sounds like trying to write in another language but in Go, which is of course not advisable in any language.
        azth 5y ago
        > Even the author of Java, James Gosling, disagrees with you. From the article: > Interface inheritance (the implements relationship) is preferable. You should avoid implementation inheritance whenever possible. Key words: preferable, and whenever possible. It's sometimes preferable to have inheritance and not possible to leave it out without unneeded extra complexity. Furthermore, nowadays interfaces in Java (as well as Kotlin, Scala, C# - all of which are much more expressive and have superior modeling capability than golang) can have default implementations, which increases their application even more and reduces the need for explicit inheritance. Not so in golang since interfaces cannot have default implementations there. > As to workarounds for inheritance in Go, that sounds like trying to write in another language but in Go, which is of course not advisable in any language. Not quite, it's just where having inheritance would have simplified the design, and improved readability.
        grey-area 5y ago
        He quite clearly states in the article he would ‘leave out classes’ (i.e. implementation inheritance). You didn’t quote the clearest statement in the article, just preceding your excerpt!? Again on interfaces, sometimes less is more. Many people prefer the inverted interfaces of go, declared at point of use. It’s fine to prefer the opposite, but these are deliberate choices, not mistakes or failures. Re inheritance simplifying a design, it does the opposite in my experience, unless by simplify you mean hide program flow and state.
closeparen 5y ago
Loom is an awesome research project that might begin to be relevant in the actual practice of production Java once it is 1-2 LTS releases old.
- lostcolony 5y ago
  Yeah; really hard to guess at the effect a feature that hasn't been finalized, has been in progress for years, and has yet to land in a release in any form, will actually have in an environment that is as slow to roll out updates and adopt new features as enterprise Java dev tends to be. Optimistically we may be able to answer op's question sometime in 2024.

travisd 5y ago

My impression of the article is just that the author is a fan of green threads (goroutines). Green threads don’t really make sense without an integration into some kind of IO runtime, so calling out how Go does do this seems moot. Other languages with green threads also with this way (Julia comes to mind).

karmakaze 5y ago
The article is not very well informed. It doesn't even make a distinction between os threads and green threads. > To get this performance out of Java you would need to add threadpools, futures or some other async library. What Java needs to be as efficient is the work of Project Loom: Fibers and Continuations. Go also has value types (e.g. arrays and nested structs with inline memory layout) that's the work of Java's Project Valhalla.
zik 5y ago
goroutines aren't really green threads, they're more lightweight than conventional green thread implementations due to having a segmented stack. Millions of goroutines can run comfortably on a standard machine. This isn't possible with any green thread implementation I've seen. https://blog.cloudflare.com/how-stacks-are-handled-in-go/ https://blog.cloudflare.com/how-stacks-are-handled-in-go/
- kortilla 5y ago
  They are cheaper but conceptually aren’t any different.
- lucian1900 5y ago
  You can run that many Erlang processes or more. Millions is typical of green thread implementations.
- nasretdinov 5y ago
  By the way, while Go did initially have segmented stacjs, they've long abandoned it in favour of a contiguous one.

bborud 5y ago

It isn’t that it’s invisible. It’s just that you didn’t get it until now. I am not saying that to belittle the author. This is more an observation of how programming seems to have gone from understanding both yourself and what you are doing to shopping bits and pieces and then gluing them together.

ferdowsi 5y ago
Not really, "invisible" is a fair way to describe complexity that developers never interact with. Especially when most other popular languages forces developers to deal with irritating async/await bookkeeping when it comes comes to concurrency.
- zozbot234 5y ago
  How so? The point of async/await is to reduce bookkeeping. The Go model is at its core an implementation of M:N user threads, and it creates plenty of complexity wrt. e.g. interfacing with other languages. Which is why M:N threading was mostly abandoned elsewhere.
  - klabb3 5y ago
    > it creates plenty of complexity wrt. e.g. interfacing with other languages Hmm I'm not sure I get this point. Do you mean in comparison with async? Async cross language interfaces should need the same type of continuation mechanisms (eg poll or completion based interop with an event loop) as green threads, no? I think of async as a primarily syntactically explicit construction, while both look similar in ABI, with a virtual stack. If not, I'd love to know why.
    - int_19h 5y ago
      Async with explicit await is basically just syntactic sugar for continuation callbacks. The "virtual stack" that results can thus be expressed entirely in terms that the C ABI understands: a callback is just a C function pointer + pointer to the struct representing the virtual stack frame.

hahnchen 5y ago

Love the tldr at the start: “ The Go standard runtime allows for asynchronous coding while allowing the developer a simply synchronous interface”

ridiculous_fish 5y ago

How does Go handle syscalls which can't be multiplexed? If I call stat() from 500 goroutines, what does the Go runtime do regarding kernel threads?

tedunangst 5y ago
500 kernel threads.

smallerfish 5y ago

Techempower benchmarks beg to differ. https://www.techempower.com/benchmarks/#section=data-r20&hw=ph&test=query https://www.techempower.com/benchmarks/#section=data-r20&hw=... There's a lot of Java above the first occurrence of Go. Also - nobody writes Java web applications using the Sun httpserver. It's verbose and old.

TheDong 5y ago
It seems to me like the go ecosystem would rather have a less performant language if it means more similar code between projects. Take for example json: Go's stdlib json implementation is wildly slow. It has to be implemented in reflection, it can't use any code-generation anything, and for backwards compatibility it can never trim any feature, no matter whether it has a performance implication for all users of the package. It's well known to be a slow implementation. In java, json isn't part of the stdlib. There's jackson if you want one flavor of fast json, jetty ships its own json serializer, etc. This means that if you pick up a new java codebase written in a different framework, it's possible you have to learn the right pattern to json serialize things. By nature of these third party libraries in java being out of the stdlib, they've been able to grow and become more performant and compete with each other. Go sits on the opposite side. The stdlib http implementation is a little slow. The stdlib json implementation is incredibly slow. No one wants to build these features outside of the stdlib though, and people would rather every codebase have the same looking go code, even if that means the actual code itself is slower.
- sofixa 5y ago
  > Go sits on the opposite side. The stdlib http implementation is a little slow. The stdlib json implementation is incredibly slow. No one wants to build these features outside of the stdlib though, and people would rather every codebase have the same looking go code, even if that means the actual code itself is slower. What? There are multiple json parsing and http server implementations for Go.
  - TheDong 5y ago
    They exist (fasthttp and fastjson for example), but no one actually uses them. I should have written "no one wants to use third-party versions of these packages", and I don't think that small semantic issue changes my comment meaningfully. Do you take issue with the way I'm presenting the go ecosystem other than that small pedantic point?

ricardobeat 5y ago

Which part of the example go code looks synchronous? It’s using a callback function. That’s a hallmark of async code, nothing invisible. I also can’t quite grasp the comparison with Java. It’s passing a class instance to the server handler, and that could be doing anything. Nothing there tells you how async or not the server implementation is, and that `os.write()` call might be using a thread pool underneath.

foolfoolz 5y ago
agreed. the examples don’t show the real pain. it is when every method returns Future or Promise. and you have to map each result. and many languages don’t have good support for that so you end up with lots of nesting. writing synchronous function signatures is much simpler to maintain. there’s 2 language level syntax features i see that make this better: one is scalas for comprehension. this lets you write async code that reads like imperative synchronous lines. the other is async/await, which let’s you write synchronous code that behaves like async
recroad 5y ago
I thought I was the only one who wasn’t getting it.
kgeist 5y ago
It's not about the callback at all, the example is poor and the explanation is lacking. In Go, many standard I/O functions use async I/O under the hood. When you call such a function (inside the handler callback, that is), the current goroutine is suspended yielding to other goroutines, so waiting for I/O doesn't block the OS thread inside the request handler. It's basically what you'd do in C# with async/await, except it's implicit and you don't have to mark functions as asynchronous manually. I.e. you write simple synchronous code and Go makes it asynchronous under the hood. There are several disadvantages, though: it's not tunable, and if there's a CGO call (call to a C func) which uses I/O itself it will block an entire worker thread because the Go runtime isn't aware of any of it. As far as I know, Java's new virtual threads have the same idea.
- spongechameleon 5y ago

jeffbee 5y ago

Unfortunately the batteries-included runtime of Go effectively forecloses any really high performance server code that scales well to huge machines. What it offers is moderately good and that's all you'll ever get. A Go web server will work OK on 8 cores and it will only go a bit faster on 32 cores and not at all faster, possibly slower, on 256 cores. Meanwhile Java async programs are still scaling up. These results are apparent if you look at the gRPC multi-language benchmarks. There's only one way to write a server in Go and that has pros and cons.

technosamay20 5y ago
Thanks for sharing this insight. Is this a issue with just the web server. If I have a standalone task, which tires to use 128 cores and manages parallel tasks using goroutines communicating via channels - will I be able to successfully leverage 128 cores.
- jeffbee 5y ago
  If your tasks are large enough to dominate, then the non-scalability of the goroutine scheduler and the network stack won’t be an issue. For example, if your task is compressing some huge hunk of a video and it takes a second to do it, then it will scale fine. Then again, you can get this scalability from anything, including `xargs`.
rawoke083600 5y ago
Interesting.. Any sources or examples ?

recursivedoubts 5y ago

I think what the author is saying is that go has very good async support, but it is no less visible than it is in java, from what I can see. Certainly better syntax, but still visible. I am working on a front end scripting language that does try to make this stuff invisible: https://hyperscript.org/docs/#async https://hyperscript.org/docs/#async It's slow as all get out, but it does, for the most part, hide all the async stuff from the script writer so they can write stuff like: fetch /someurl put the results into #a-div wait 2s transition #a-div's opacity to 0 remove #a-div And all the async stuff in there (the fetch, the wait, the transition) all happen w/o callbacks, async annotations, etc.

Yoric 5y ago
That's going to make data race conditions invisible, no? How will developers cope with that?
- kgeist 5y ago
  Data races happen when there are multiple writers/readers of the same data. If one virtual thread of execution is multiplexed on several OS threads/workers/callbacks/you name it, there's still only one reader/writer at a time, because different parts of a multiplexed function don't run at the same time, they go one after another sequentially as before, with the difference is that there are now implicit continuation points where execution can hop from thread to thread (callback to callback). Can't say for hyperscript, but generally you need to protect your data with the same synchronization primitives as before, such as locks for example. It's only incompatible with thread locals and thread affinity, I'm not sure if JS has those.
  - Yoric 5y ago
    If your data is composite (i.e. a JavaScript object or array) and you have several Promise-based flows of execution (technically not a virtual thread by most definitions of the term, but still a scheduled flow of execution) that end up reading/writing sub-parts of your piece of data in unpredictable order. That's basically a data race, in JavaScript. I've had to debug some of those. As a side-note, some forms of data races (for perhaps a slightly looser definition of the term) can even happen without explicit asynchronicity, just with callbacks/synchronous events.

halpert 5y ago

This type of synchronous code is most impressive to those who come from JavaScript where everything is clearly asynchronous. For instance, in Go you can read a file synchronously using os.ReadFile and it will generally be performant. In NodeJS, you need a callback to achieve the same result.

apatheticonion 5y ago

It's a weird one. The fact that callbacks in Go may be on a different thread means it's so damn easy to use a variable reference from an outer scope within in a callback. This increases the chances that someone inadvertently causes a race condition. Something that requires additional awareness and screening during code reviews. This by itself isn't an indictment of the language as threading is essential to quality GUI applications and as such Go is a great language for writing client applications - which it is largely not used for. You really need to evaluate the value the overhead Go's thread management brings when used in the context of web servers - where we typically split processes up using container orchestration, making them functionally threaded any way. Comparatively, TypeScript running on Node or Deno is single threaded yet asynchronous. Using it as a web server means you have a better type system which follows a similar structural typing model to go (interfaces and such) and you don't have to screen for race conditions. In K8 or ECS, you spawn as many processes as you have CPUs and Bob's your unkle. Personally, I would love it if Go focused on GUI application development but I can _bearally_ write GTK apps with it. Windows and MacOS applications are near-to or impossible to write. Don't even talk to me about Web Assembly. Such a missed opportunity.

tgv 5y ago
That’s a weird critique of Go: you make it sound as if you must use threads. But nothing stops you from using callbacks and promises (which have become a bit more comfy with the addition of generics). And if the there’s one domain where I see an advantage in using inheritance, it’s GUIs.

Thorrez 5y ago

Just looking at the numbers, Go is 71% faster. That could be due to the Go HTTP library being more efficient than the Java HTTP library, rather than language or runtime features.

lenkite 5y ago
Nobody uses that internal Oracle HTTP server - it is not meant for production. It isn't even in the Java namespace -your static analysers will complain about using 'com.sun.*' packages. Use the Java Jetty server and Go will hide itself in a corner in shame when benchmarked.
- vips7L 5y ago
  Undertow has better performance for the example program: https://www.reddit.com/r/programming/comments/sanusb/comment/htv5cnu/ https://www.reddit.com/r/programming/comments/sanusb/comment...

est 5y ago

> TLDR: The Go standard runtime allows for asynchronous coding while allowing the developer a simply synchronous interface. I am happy for Golang users but Gevent is more "synchronous" and easier to write.

kgeist 5y ago
>I am happy for Golang users but Gevent is more "synchronous" and easier to write. Can you give an example to substantiate your post?

bvrmn 5y ago

Hmm. If you pack real blocking code (instead of writing a couple of bytes) into Go's handler it would definitely turn into crippled turtle under high concurrency load. There is no magic in Go's runtime.

pcwalton 5y ago

Every operating system kernel also presents a synchronous interface to asynchronous I/O. What Go provides is more efficient multithreading in certain cases, primarily because the kernel scheduler doesn't need to be involved on context switches. But this could be fixed at the kernel level with something like Windows' fibers or Google's SwitchTo patches.

frizlab 5y ago

https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ https://vorpus.org/blog/notes-on-structured-concurrency-or-g...

torginus 5y ago

I kind of disagree with the article's premise - that the fact that Go uses async/await like Js and C#, but presents them as green threads is a good thing. Imo it creates false expectations in programmers, who are surprised, that long-running compute code doesn't get preempted, since the compiler doesn't insert preemption points. It also robs us of the possibility, of writing event-driven UI applications, where one can make an asynchronous call on the UI thread safely, which will get marshalled back to the UI thread.

nasretdinov 5y ago
By the way, Go does have async preemption. It didn't have it in version 1.0, but it does now. https://medium.com/a-journey-with-go/go-asynchronous-preemption-b5194227371c https://medium.com/a-journey-with-go/go-asynchronous-preempt...

KingOfCoders 5y ago

The author is right in principle, i I think Golangs async scheduler which automatically moves blocking code outside Golangs goroutine scheduler is amazing, but his code and explanations are all wrong. also "Java’s green threads" shows a deep lack of understanding. The author shows a junior level of understanding which co tradicts "I’ve worked professionally with Go for a couple of years"

PYTHONDJANGO 5y ago
If you want to show a senior level of understanding coding discussions, please understand that you always should write with a scientific mindset. This means: not only say "x is good" or "x is bad" without explanation like a little child - but also deliver the source of your knowledge or the proof. Then your contributions will be seen as mature additions to collective wisdom.