4 ms·

I guess I wish this was a bit more in-depth as to what "go" can do with channels or goroutines, but maybe I've just been using languages where all this is alrea

I guess I wish this was a bit more in-depth as to what "go" can do with channels or goroutines, but maybe I've just been using languages where all this is already possible. The article is a nice cursory glance, I just want to learn more :) I mean the first example it looks like is using a Mutex, and then (b)locking on it, and the second just looks like having a queue of messages (mailbox) that it rips through (like the actor pattern). Some questions I've got after reading this... How does the go-runtime (?) schedule these calls? Does it manage an internal thread pool? Is the scheduler asynchronous, parallel, or both? How do you manage contention or back pressure if I begin to flood messages to to one channel (or many)? How many channels can I have open and what are the limits? Can I still lock the system stupidly using channels, if so, how (or how not)? Edit: Truly, I'm curious because as I researched asynchronous programming and efforts to better future proof my career (years ago) as we began really increasing core counts-- Go never stood out. It's a fairly practical language, yes, but if I want a better paradigm for asynchronous programming the future it really isn't there (IMHO). BEAM stood out as something unique, the JVM stood out as something even more practical, and Rust stood out as something performant (with the serious caveat of not being C or C++), while Go has always seemed like an odd one to me... People talk about channels and goroutines like their special but they seem pretty damn run of the mill to me... WAT AM I MISSING?

gjs278 7y ago
you manage the channel backlog with semaphores and wait
SamReidHughes 7y ago
Go channels have a fixed capacity, by default zero, and their main special feature (compared to just having some concurrent_queue<T> type) is the select statement, which helps devs do things the right way. Nothing stops you from stupidly locking the system, e.g. by having a thread block itself by writing to the channel it’s responsible for consuming.
- jayd16 7y ago
  Is the select semantically different than something like this? bool TryWrite(T) How does go handle cancellation with the select syntax? I guess you have a cancel channel and select across both the cancel and the read channels? Is Go smart enough to sleep and not busy wait in that case?
  - TheDong 7y ago
    > Is the select semantically different than something like this? Yes. select can do a lot of things. It's overloaded to be something like 5 different things. Here, let me show you: chanOne := make(chan int) chanTwo := make(chan int) ctx := context.Background() // Receive whichever is ready first. Blocking select { case v := <-chanOne: case v := <-chanTwo: } // Non-blocking read select { case v := <-chanOne: default: } // Timeout and cancellation select { case v := <-chanOne: case <-time.After(1 * time.Second): // timeout case <-ctx.Done(): // cancel } // Write or read, whichever channel is ready first select { case chanOne <- 1: case v := <-chanTwo: } Another form of cancellation is closing channels, which causes reads to return the zero value immediately, and causes any writes to that closed channel to immediately panic (effectively abort the program).
    - jayd16 7y ago
      Neat. Most of the permutations are easily handled in other languages as well except for the full blocking (sleeping) case. Without a channel the scheduler can reason about, most languages would have to busy wait or possibly pull from multiple channels.
      - SamReidHughes 7y ago
        I think the biggest advantage is that it's a lot harder to screw up using a select statement than most equivalent API's you'd create in other languages that allow waiting on one of N channels/signals/channel/event operations, and branching based off that. Simply because you're either constructing a bunch of callback objects or you get some number N saying the Nth parameter had an event, and your code has to match N to the specific channel correctly.
Vendan 7y ago
Goroutines are scheduled onto system threads via M:N greenthreading. Each "system" thread can steal work from other threads, and there's some stuff so that blocking system calls don't use up your "goroutine system threads" (you can put a limit on the number of goroutines that run concurrently). Channels are fixed size on creation, either 0 (directly handing off items from 1 goroutine to another) or >0 (the channel has space allocated to hold that many items). If the channel is full(buffered, i.e. >0 size channel) or there's nothing waiting to receive the item(unbuffered channel), the sender sleeps until it can send the item. Channels are purely data structure, have as many as you want. You can lock the system fairly trivially, but Go can detect it in many cases and give you stack traces of all the places stuff is waiting to read/write (only if all goroutines deadlock). edit: And no, there's nothing really special about channels, just they play nice with the goroutine scheduler, so it's perfectly sensible to do lots of "wait on these channels" stuff inside your program, without having to have lots of OS threads and such. (goroutines are a bit more lightweight then OS threads)
lmm 7y ago
It's like what you'd do with actors on top of some kind of userspace scheduling (e.g. a thread pool that accepts tasks), yes. Writes to channels are implicitly yield points, so you write cooperative multitasking code (again, like you'd do in any language with userspace scheduling) with all the advantages that entails (no blocking), but without having to explicitly manage your sequencing/yielding. Since the whole language is built around the idea that this is going to be happening, a lot of the immediate problems with userspace scheduling go away: anything that's trying to do thread-local storage (e.g. web requests or database sessions getting bound to the current thread) will necessarily be goroutine-aware and do the right thing. There's no free lunch, of course: it becomes much harder to write code that is pinned to a single system thread (e.g. good luck interoperating with a C library that expects you to pass callbacks), or to use higher-performance blocking I/O in the cases where that's warranted. But it makes the 80% case very straightforward.
_tkzm 7y ago
https://www.youtube.com/watch?v=h0s8CWpIKdg https://www.youtube.com/watch?v=h0s8CWpIKdg

gjs278 7y ago

you manage the channel backlog with semaphores and wait

SamReidHughes 7y ago

Go channels have a fixed capacity, by default zero, and their main special feature (compared to just having some concurrent_queue<T> type) is the select statement, which helps devs do things the right way. Nothing stops you from stupidly locking the system, e.g. by having a thread block itself by writing to the channel it’s responsible for consuming.

jayd16 7y ago
Is the select semantically different than something like this? bool TryWrite(T) How does go handle cancellation with the select syntax? I guess you have a cancel channel and select across both the cancel and the read channels? Is Go smart enough to sleep and not busy wait in that case?
- TheDong 7y ago
  > Is the select semantically different than something like this? Yes. select can do a lot of things. It's overloaded to be something like 5 different things. Here, let me show you: chanOne := make(chan int) chanTwo := make(chan int) ctx := context.Background() // Receive whichever is ready first. Blocking select { case v := <-chanOne: case v := <-chanTwo: } // Non-blocking read select { case v := <-chanOne: default: } // Timeout and cancellation select { case v := <-chanOne: case <-time.After(1 * time.Second): // timeout case <-ctx.Done(): // cancel } // Write or read, whichever channel is ready first select { case chanOne <- 1: case v := <-chanTwo: } Another form of cancellation is closing channels, which causes reads to return the zero value immediately, and causes any writes to that closed channel to immediately panic (effectively abort the program).
  - jayd16 7y ago
    Neat. Most of the permutations are easily handled in other languages as well except for the full blocking (sleeping) case. Without a channel the scheduler can reason about, most languages would have to busy wait or possibly pull from multiple channels.
    - SamReidHughes 7y ago
      I think the biggest advantage is that it's a lot harder to screw up using a select statement than most equivalent API's you'd create in other languages that allow waiting on one of N channels/signals/channel/event operations, and branching based off that. Simply because you're either constructing a bunch of callback objects or you get some number N saying the Nth parameter had an event, and your code has to match N to the specific channel correctly.

Vendan 7y ago

Goroutines are scheduled onto system threads via M:N greenthreading. Each "system" thread can steal work from other threads, and there's some stuff so that blocking system calls don't use up your "goroutine system threads" (you can put a limit on the number of goroutines that run concurrently). Channels are fixed size on creation, either 0 (directly handing off items from 1 goroutine to another) or >0 (the channel has space allocated to hold that many items). If the channel is full(buffered, i.e. >0 size channel) or there's nothing waiting to receive the item(unbuffered channel), the sender sleeps until it can send the item. Channels are purely data structure, have as many as you want. You can lock the system fairly trivially, but Go can detect it in many cases and give you stack traces of all the places stuff is waiting to read/write (only if all goroutines deadlock). edit: And no, there's nothing really special about channels, just they play nice with the goroutine scheduler, so it's perfectly sensible to do lots of "wait on these channels" stuff inside your program, without having to have lots of OS threads and such. (goroutines are a bit more lightweight then OS threads)

lmm 7y ago

It's like what you'd do with actors on top of some kind of userspace scheduling (e.g. a thread pool that accepts tasks), yes. Writes to channels are implicitly yield points, so you write cooperative multitasking code (again, like you'd do in any language with userspace scheduling) with all the advantages that entails (no blocking), but without having to explicitly manage your sequencing/yielding. Since the whole language is built around the idea that this is going to be happening, a lot of the immediate problems with userspace scheduling go away: anything that's trying to do thread-local storage (e.g. web requests or database sessions getting bound to the current thread) will necessarily be goroutine-aware and do the right thing. There's no free lunch, of course: it becomes much harder to write code that is pinned to a single system thread (e.g. good luck interoperating with a C library that expects you to pass callbacks), or to use higher-performance blocking I/O in the cases where that's warranted. But it makes the 80% case very straightforward.

_tkzm 7y ago

https://www.youtube.com/watch?v=h0s8CWpIKdg https://www.youtube.com/watch?v=h0s8CWpIKdg