5 ms·

What do you think about goroutines and channels?

hackcasual 11y ago
Using go for goroutines and channels is a bit like using Perl for regular expressions. The features have been added in a way that makes them easy to use and serves as a nice idiomatic platform, but fundamentally it's functionality other languages can provide via library support.
- dgellow 11y ago
  +1 on this. Clojure's core.async[0] is the perfect example of an implementation of CSP as a library. Even JS can be used to implement such concepts via the use of generators[1]. [0] https://github.com/clojure/core.async https://github.com/clojure/core.async [1] https://github.com/ubolonton/js-csp https://github.com/ubolonton/js-csp
  - masklinn 11y ago
    > +1 on this. Clojure's core.async[0] is the perfect example of an implementation of CSP as a library. With the added convenience that shared mutability is pretty much nonexistent.
  - afandian 11y ago
    Correct me if I'm wrong, but describing core.async as "a library" isn't perfect in the context of a golang discussion. Doesn't the `go` macro rewrite the abstract syntax tree / JVM bytecode to make e.g. the `!<` macro co-operate with the channel? https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async/impl/ioc_macros.clj https://github.com/clojure/core.async/blob/master/src/main/c... That's not something that could be done with golang as far as I know.
    - dgellow 11y ago
      I'm not sure to understand your point. Could your clarify?
      - afandian 11y ago
        `hackcasual` was saying that certain language features can be added as a library rather than needing to be integrated into the core language > "fundamentally it's functionality other languages can provide via library support" You were saying that CSP can be added as a library, citing Clojure's core.async. All I was saying was that the way in which core.async was implemented doesn't feel like a great example of a 'library' in the sense that most people would understand in the context of a discussion about Golang. Golang is a static, compiled-to-machine-code language without macros (in the LISP or C sense) or homoiconicity. The reason core.async can be implemented as a library in Clojure is that it has these things. If you're talking about adding CSP to a language just by adding a library and without having to get into the internals of the language, core.async isn't a good example. Again, happy to be corrected.
        dgellow 11y ago
        Well core.async is a Clojure library so it uses features available in Clojure. I don't see how it would affect the fact that it is a library. I've also linked to js-csp, a JS library obviously not implemented using macros. I can also find other examples of implementation as libraries, but I have no experience with them: - Scala: https://github.com/rssh/scala-gopher https://github.com/rssh/scala-gopher - F#: https://github.com/Hopac/Hopac https://github.com/Hopac/Hopac - C++: http://www.cs.kent.ac.uk/projects/ofa/c++csp/ http://www.cs.kent.ac.uk/projects/ofa/c++csp/
        afandian 11y ago
        Wasn't saying it can't be done! Just nit-picking at the particular example wrt to golang.
        dgellow 11y ago
  - muraiki 11y ago
    I used to feel this way until I realized the limitations of core.async. In Go I don't have to worry about whether the particular functions I'm calling, especially IO-related, are blocking or not, as Go will create new lightweight goroutines as necessary to deal with all that. With core.async, if I use blocking IO inside of a coroutine I risk causing thread starvation. See http://martintrojer.github.io/clojure/2013/07/07/coreasync-and-blocking-io/ http://martintrojer.github.io/clojure/2013/07/07/coreasync-a... Maybe things have changed since 2013, but I feel like this is a fundamental limitation of running on the JVM vs what Go can provide in its runtime. Edit: Also, it appears to be much easier to simply "run out" of Clojure coroutines than Go goroutines, but perhaps that's also changed. Anyways, my point is that by core.async operating as a macro you still can't overcome limitations of the underlying runtime, whereas Go's runtime was purposely-built to support goroutines.
    - nwalkr 11y ago
      Small remark: Go I/O layer is safe to use only with network I/O. Only network I/O plays nice with goroutines. File I/O or everything else treated as syscall by Go runtime might turn you program into 10k-os-threads-monster. Scheduler will be creating new OS threads to replace those locked on syscalls until thread limit is reached and whole program crashes. Only way to prevent it is to restrict your syscall layer into fixed-size goroutine pool. I had an interesting case recently - my app serves some data from tons of files laying in NAS, accessing it by NFS mount and one day NAS hunged completely, every I/O call to it was lasting forever. Even 'ls /mount-point-of-nas' was just doing nothing forever until Ctrl-C. In my case I've applied poweroff-poweron cycle to NAS, and everything went right in minutes, just as NAS booted. And after it I wondered, what if my server was written in Go, instead of Erlang... And, BTW, you can never be sure, that underlying libraries of your code are safe to use.
  - riwsky 11y ago
    It's also a perfect example of the limitations of that approach - core.async had to make serious compromises in its interface because it was 'just a library': expressions with <!'s and other calls can't just be pulled into functions or for-comprehensions like normal code. That's not to say it's poorly-done, or not useful - it is well done and useful, and those compromises are in line with clojure's goal of integrating well with host vms. It's just an example of how builtins can be "simpler" sometimes. https://github.com/clojure/core.async/wiki/Go-Block-Best-Practices#unsupported-constructs-and-other-limitations-in-go-blocks https://github.com/clojure/core.async/wiki/Go-Block-Best-Pra...
- tormeh 11y ago
  I tried making a CSP library for C. It was not very pleasant. At best you end up with something slightly significantly less safe than POSIX threads, but now with message passing. While C is an extreme example it's certainly not true that all languages can add CSP/actors/whatever in an appetizing form through a library. Anyway, there's a reason the phrase "tacked-on" has such negative connotations.
  - sevensor 11y ago
    What's your take on libmill?
- exDM69 11y ago
  > but fundamentally it's functionality other languages can provide via library support. Implementing goroutines and channels requires language and runtime support for green threads that are n:m multiplexed on top of native threads. It can not be implemented as a library in most languages, at least not efficiently. Any language with thread support can set up threads and put a concurrent queue between them, but that's hardly the same thing. Languages such as Go, Erlang and Haskell do this. Interestingly, early versions of Rust had green threads (iirc) but later migrated to using native threads only.
  - adrusi 11y ago
    Any language that has continuations, or at least thread-safe coroutines, can implement goroutines and channels. This includes scheme and lua. Also any language where the stack can be directly manipulated can implement those thread-safe coroutines, so that opens up C/C++ and perl and possibly some others.
    - exDM69 11y ago
      Yes, all the languages you mention have the necessary language and runtime support required, probably a handful of others too (but by no means every language out there). C and C++ are a bit different because you need to resort to assembly and know details about the target arch to do stack switching but that's acceptable.
  - iofj 11y ago
    No ? Qt, Gtk, ... all do it.
    - exDM69 11y ago
      Huh? Care to elaborate on this? As far as I know, GTK (and Qt IIRC) use a single threaded event loop. That's not at all the same thing (albeit can be used for similar things).
      - iofj 11y ago
cloud_monk 11y ago
I think the way Go addresses concurrency is simple and straight-forward. It's trivial to write concurrent applications. If your target is writing semi-low-level infrastructure software I think Go is a great choice. It's definitely got a lot of fans in the world of people writing software for DevOpsy type applications. From a personal standpoint, it's missing a lot of the features I like, namely ADTs, list comprehensions, folds, maps, etc. But that's just my personal style, not something wrong with Go. Go programs don't necessarily always look pretty but you can usually understand them after a minimal amount of study because the language is so simple.
- dgellow 11y ago
  > I think the way Go addresses concurrency is simple and straight-forward. It's trivial to write concurrent applications. Yes, CSP[0] is a very interesting concept. But it's not something unique to the go language. [0] https://en.wikipedia.org/wiki/Communicating_sequential_processes https://en.wikipedia.org/wiki/Communicating_sequential_proce...
- pkolaczk 11y ago
  "I think the way Go addresses concurrency is simple and straight-forward. It's trivial to write concurrent applications." Except when it is not. Message passing style of concurrency is just a dual of the classical blocking concurrency with critical sections, mutexes, monitors and conditional variables. An actor is a dual of a critical section. Actor's mailbox is a dual of a mutex. Sending/receiving messages is a dual of wait/notify. With any complex CSP program you can have all the same problems: race conditions, starvation, deadlocks (livelocks) etc.
cmrdporcupine 11y ago
They're fine concepts, but... On the little project I was tasked with using Go with at Google I got slapped down by the readability reviewers for using them. I think this is an interesting construct, but not sure that community really knows how to use them well? Erlang at least is consistent on this -- it has that hammer well tuned and isn't afraid to pound nails with it.
- gendoikari 11y ago
  Mmmm I'm puzzled. I don't think we are consulting the same community... Goroutines are everywhere in all the main go projects. Goroutines and channels are one of the main reasons Go exists.
  - cmrdporcupine 11y ago
    It's quite possible. After my experience with Go code reviews internally at Google, though, I am not eager to go back. I'd work on a project if I was paid to do it, but I wouldn't start one or advocate for it.
  - mike_hearn 11y ago
    "Readability reviews" at Google are somewhat notorious for imposing fairly arbitrary style choices extremely rigidly, for instance 80 characters per line (woe betide you if even one line in a 5000 line patch is 81 characters...). It doesn't sound so surprising to me that the people behind such a process might have decided that one of Go's primary selling points is 'confusing', given that the Go authors appear to believe their colleagues can't handle a brilliant language!
  - kungfooguru 11y ago
    He said at Google not the community. I wouldn't doubt there is a difference.
    - gendoikari 11y ago
      I'm pretty sure at Google they know why they created Go, why they're using Go, and so on... The assumption that Google reviewers had something against goroutine puzzles me a bit. Maybe the problem was not about goroutines, but about how they were used... I don't know, I'm guessing...
tveita 11y ago
Goroutines are for all practical purposes threads. Threaded code is generally thought to be difficult to write correctly, in ways that can't be solved just by making the threads cheaper to spin up. Queues ("channels") are a good way to limit complexity of threaded code by treating each process as an agent. Besides some syntactical sugar Go doesn't really support this better than most other languages with threading, like Java. Go has a weird thing going on where channels are sometimes used as a kind-of-replacement for iterators, which is error-prone since the "obvious" way to do it doesn't allow the consumer to stop the generator without a side channel. This can lead to buggy code that leaks goroutines, since goroutines can not be garbage collected. One of the best ways to reduce the complexity of threaded code is immutability - preventing race conditions by making sure a structure never changes while being read. Curiously, Go has no way to mark an object as immutable, and does nothing to detect or prevent objects being unsafely accessed from different threads.
- skj 11y ago
  > Queues ("channels") are a good way to limit complexity of threaded code by treating each process as an agent. Besides some syntactical sugar Go doesn't really support this better than most other languages with threading, like Java. The magic of channels comes with select{}. Considering them to be only threadsafe queues is really missing out. Supporting select{} in other languages is possible, but difficult and rare.
  - bmurphy1976 11y ago
    Also, by default channels are zero length and block. What is a zero length queue in other languages? Doesn't even make sense! Zero length channels are a key part of coordinating concurrent threads in Go.

Using go for goroutines and channels is a bit like using Perl for regular expressions. The features have been added in a way that makes them easy to use and serves as a nice idiomatic platform, but fundamentally it's functionality other languages can provide via library support.

dgellow 11y ago
+1 on this. Clojure's core.async[0] is the perfect example of an implementation of CSP as a library. Even JS can be used to implement such concepts via the use of generators[1]. [0] https://github.com/clojure/core.async https://github.com/clojure/core.async [1] https://github.com/ubolonton/js-csp https://github.com/ubolonton/js-csp
- masklinn 11y ago
  > +1 on this. Clojure's core.async[0] is the perfect example of an implementation of CSP as a library. With the added convenience that shared mutability is pretty much nonexistent.
- afandian 11y ago
  Correct me if I'm wrong, but describing core.async as "a library" isn't perfect in the context of a golang discussion. Doesn't the `go` macro rewrite the abstract syntax tree / JVM bytecode to make e.g. the `!<` macro co-operate with the channel? https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async/impl/ioc_macros.clj https://github.com/clojure/core.async/blob/master/src/main/c... That's not something that could be done with golang as far as I know.
  - dgellow 11y ago
    I'm not sure to understand your point. Could your clarify?
    - afandian 11y ago
      `hackcasual` was saying that certain language features can be added as a library rather than needing to be integrated into the core language > "fundamentally it's functionality other languages can provide via library support" You were saying that CSP can be added as a library, citing Clojure's core.async. All I was saying was that the way in which core.async was implemented doesn't feel like a great example of a 'library' in the sense that most people would understand in the context of a discussion about Golang. Golang is a static, compiled-to-machine-code language without macros (in the LISP or C sense) or homoiconicity. The reason core.async can be implemented as a library in Clojure is that it has these things. If you're talking about adding CSP to a language just by adding a library and without having to get into the internals of the language, core.async isn't a good example. Again, happy to be corrected.
      - dgellow 11y ago
        Well core.async is a Clojure library so it uses features available in Clojure. I don't see how it would affect the fact that it is a library. I've also linked to js-csp, a JS library obviously not implemented using macros. I can also find other examples of implementation as libraries, but I have no experience with them: - Scala: https://github.com/rssh/scala-gopher https://github.com/rssh/scala-gopher - F#: https://github.com/Hopac/Hopac https://github.com/Hopac/Hopac - C++: http://www.cs.kent.ac.uk/projects/ofa/c++csp/ http://www.cs.kent.ac.uk/projects/ofa/c++csp/
        afandian 11y ago
        Wasn't saying it can't be done! Just nit-picking at the particular example wrt to golang.
        dgellow 11y ago
- muraiki 11y ago
  I used to feel this way until I realized the limitations of core.async. In Go I don't have to worry about whether the particular functions I'm calling, especially IO-related, are blocking or not, as Go will create new lightweight goroutines as necessary to deal with all that. With core.async, if I use blocking IO inside of a coroutine I risk causing thread starvation. See http://martintrojer.github.io/clojure/2013/07/07/coreasync-and-blocking-io/ http://martintrojer.github.io/clojure/2013/07/07/coreasync-a... Maybe things have changed since 2013, but I feel like this is a fundamental limitation of running on the JVM vs what Go can provide in its runtime. Edit: Also, it appears to be much easier to simply "run out" of Clojure coroutines than Go goroutines, but perhaps that's also changed. Anyways, my point is that by core.async operating as a macro you still can't overcome limitations of the underlying runtime, whereas Go's runtime was purposely-built to support goroutines.
  - nwalkr 11y ago
    Small remark: Go I/O layer is safe to use only with network I/O. Only network I/O plays nice with goroutines. File I/O or everything else treated as syscall by Go runtime might turn you program into 10k-os-threads-monster. Scheduler will be creating new OS threads to replace those locked on syscalls until thread limit is reached and whole program crashes. Only way to prevent it is to restrict your syscall layer into fixed-size goroutine pool. I had an interesting case recently - my app serves some data from tons of files laying in NAS, accessing it by NFS mount and one day NAS hunged completely, every I/O call to it was lasting forever. Even 'ls /mount-point-of-nas' was just doing nothing forever until Ctrl-C. In my case I've applied poweroff-poweron cycle to NAS, and everything went right in minutes, just as NAS booted. And after it I wondered, what if my server was written in Go, instead of Erlang... And, BTW, you can never be sure, that underlying libraries of your code are safe to use.
- riwsky 11y ago
  It's also a perfect example of the limitations of that approach - core.async had to make serious compromises in its interface because it was 'just a library': expressions with <!'s and other calls can't just be pulled into functions or for-comprehensions like normal code. That's not to say it's poorly-done, or not useful - it is well done and useful, and those compromises are in line with clojure's goal of integrating well with host vms. It's just an example of how builtins can be "simpler" sometimes. https://github.com/clojure/core.async/wiki/Go-Block-Best-Practices#unsupported-constructs-and-other-limitations-in-go-blocks https://github.com/clojure/core.async/wiki/Go-Block-Best-Pra...
tormeh 11y ago
I tried making a CSP library for C. It was not very pleasant. At best you end up with something slightly significantly less safe than POSIX threads, but now with message passing. While C is an extreme example it's certainly not true that all languages can add CSP/actors/whatever in an appetizing form through a library. Anyway, there's a reason the phrase "tacked-on" has such negative connotations.
- sevensor 11y ago
  What's your take on libmill?
exDM69 11y ago
> but fundamentally it's functionality other languages can provide via library support. Implementing goroutines and channels requires language and runtime support for green threads that are n:m multiplexed on top of native threads. It can not be implemented as a library in most languages, at least not efficiently. Any language with thread support can set up threads and put a concurrent queue between them, but that's hardly the same thing. Languages such as Go, Erlang and Haskell do this. Interestingly, early versions of Rust had green threads (iirc) but later migrated to using native threads only.
- adrusi 11y ago
  Any language that has continuations, or at least thread-safe coroutines, can implement goroutines and channels. This includes scheme and lua. Also any language where the stack can be directly manipulated can implement those thread-safe coroutines, so that opens up C/C++ and perl and possibly some others.
  - exDM69 11y ago
    Yes, all the languages you mention have the necessary language and runtime support required, probably a handful of others too (but by no means every language out there). C and C++ are a bit different because you need to resort to assembly and know details about the target arch to do stack switching but that's acceptable.
- iofj 11y ago
  No ? Qt, Gtk, ... all do it.
  - exDM69 11y ago
    Huh? Care to elaborate on this? As far as I know, GTK (and Qt IIRC) use a single threaded event loop. That's not at all the same thing (albeit can be used for similar things).
    - iofj 11y ago

cloud_monk 11y ago

I think the way Go addresses concurrency is simple and straight-forward. It's trivial to write concurrent applications. If your target is writing semi-low-level infrastructure software I think Go is a great choice. It's definitely got a lot of fans in the world of people writing software for DevOpsy type applications. From a personal standpoint, it's missing a lot of the features I like, namely ADTs, list comprehensions, folds, maps, etc. But that's just my personal style, not something wrong with Go. Go programs don't necessarily always look pretty but you can usually understand them after a minimal amount of study because the language is so simple.

dgellow 11y ago
> I think the way Go addresses concurrency is simple and straight-forward. It's trivial to write concurrent applications. Yes, CSP[0] is a very interesting concept. But it's not something unique to the go language. [0] https://en.wikipedia.org/wiki/Communicating_sequential_processes https://en.wikipedia.org/wiki/Communicating_sequential_proce...
pkolaczk 11y ago
"I think the way Go addresses concurrency is simple and straight-forward. It's trivial to write concurrent applications." Except when it is not. Message passing style of concurrency is just a dual of the classical blocking concurrency with critical sections, mutexes, monitors and conditional variables. An actor is a dual of a critical section. Actor's mailbox is a dual of a mutex. Sending/receiving messages is a dual of wait/notify. With any complex CSP program you can have all the same problems: race conditions, starvation, deadlocks (livelocks) etc.

cmrdporcupine 11y ago

They're fine concepts, but... On the little project I was tasked with using Go with at Google I got slapped down by the readability reviewers for using them. I think this is an interesting construct, but not sure that community really knows how to use them well? Erlang at least is consistent on this -- it has that hammer well tuned and isn't afraid to pound nails with it.

gendoikari 11y ago
Mmmm I'm puzzled. I don't think we are consulting the same community... Goroutines are everywhere in all the main go projects. Goroutines and channels are one of the main reasons Go exists.
- cmrdporcupine 11y ago
  It's quite possible. After my experience with Go code reviews internally at Google, though, I am not eager to go back. I'd work on a project if I was paid to do it, but I wouldn't start one or advocate for it.
- mike_hearn 11y ago
  "Readability reviews" at Google are somewhat notorious for imposing fairly arbitrary style choices extremely rigidly, for instance 80 characters per line (woe betide you if even one line in a 5000 line patch is 81 characters...). It doesn't sound so surprising to me that the people behind such a process might have decided that one of Go's primary selling points is 'confusing', given that the Go authors appear to believe their colleagues can't handle a brilliant language!
- kungfooguru 11y ago
  He said at Google not the community. I wouldn't doubt there is a difference.
  - gendoikari 11y ago
    I'm pretty sure at Google they know why they created Go, why they're using Go, and so on... The assumption that Google reviewers had something against goroutine puzzles me a bit. Maybe the problem was not about goroutines, but about how they were used... I don't know, I'm guessing...

tveita 11y ago

Goroutines are for all practical purposes threads. Threaded code is generally thought to be difficult to write correctly, in ways that can't be solved just by making the threads cheaper to spin up. Queues ("channels") are a good way to limit complexity of threaded code by treating each process as an agent. Besides some syntactical sugar Go doesn't really support this better than most other languages with threading, like Java. Go has a weird thing going on where channels are sometimes used as a kind-of-replacement for iterators, which is error-prone since the "obvious" way to do it doesn't allow the consumer to stop the generator without a side channel. This can lead to buggy code that leaks goroutines, since goroutines can not be garbage collected. One of the best ways to reduce the complexity of threaded code is immutability - preventing race conditions by making sure a structure never changes while being read. Curiously, Go has no way to mark an object as immutable, and does nothing to detect or prevent objects being unsafely accessed from different threads.

skj 11y ago
> Queues ("channels") are a good way to limit complexity of threaded code by treating each process as an agent. Besides some syntactical sugar Go doesn't really support this better than most other languages with threading, like Java. The magic of channels comes with select{}. Considering them to be only threadsafe queues is really missing out. Supporting select{} in other languages is possible, but difficult and rare.
- bmurphy1976 11y ago
  Also, by default channels are zero length and block. What is a zero length queue in other languages? Doesn't even make sense! Zero length channels are a key part of coordinating concurrent threads in Go.