7 ms·

Three ways to think about Go channels

162 points by ingve 2y ago

jgrahamc 2y ago
I'll admit that I like Go channels because Hoare was the professor when I was doing my doctorate and so CSP was what I used in my thesis[1], but it's worth understanding what unbuffered channels give you: it's message passing with synchronization. They are very simple to reason about and make writing concurrent code a breeze. [1] https://blog.jgc.org/2024/03/the-formal-development-of-secure.html https://blog.jgc.org/2024/03/the-formal-development-of-secur...
- bsaul 2y ago
  i like the pattern, but message passing where the message is actually just a pointer to a mutable struct ( as allowed in go) kind of defeat the purpose, doesn't it ?
  - superb_dev 2y ago
    It kinda does yeah, you probably shouldn’t do that
    - adastra22 2y ago
      It’s fine if that’s the only (writable) pointer to the struct. Which is something go doesn’t enforce, but Rust does. I often dream of Go with Rust’s borrow checker. Sigh…
      - jimbokun 2y ago
        Wouldn't that be Rust?
        adastra22 2y ago
        Rust doesn’t have a direct analogue to go’s concurrency primitives. The pieces are there, but you have to jump through hoops to use them effectively.
  - dgb23 2y ago
    No, it’s an idiomatic way of expressing who’s in charge of a given piece of data. There’s nothing in the type system preventing data races. But that’s orthogonal to channels. They just help you to structure the code.
- bjoli 2y ago
  I come from the complete opposite side: I have been writing some concurrentML code in guile scheme recently, and it has really made me understand why I always disliked go's channels. There are just so many things that are just slightly wrong to make it unpleasant. Better than many other things, but still kind of frustrating. I really think unbuffered channels should have been the default.
hedora 2y ago
I’ve worked with a few large code bases that use channels. In all those code bases, they were about as maintainable as GOTO-based control flow, except that GOTO makes you have unique label names. All the channel-based code I’ve seen just has 100’s of call sites like “chan->send()” and “chan->recv()” sprinkled around, and doesn’t even have the discipline to put related senders and receivers in the same source file. At least old-school syntax like “GOTO foobar_step_23” and “LABEL foobar_step_23” is grepable). I greatly prefer programs that “color” functions sync/async, and that use small state machines to coordinate shared state only when necessary. Go technically supports this, but it doesn’t seem like it is idiomatic (unlike rust, the go compiler won’t help with data races, and unlike C++, people don’t assume that the language is a giant foot-gun).
- astockwell 2y ago
  Go does have the `- race` flag for test + build commands [1], which can help (not 100% iirc, but some) with datarace detection. [1] https://go.dev/doc/articles/race_detector https://go.dev/doc/articles/race_detector
- hot_gril 2y ago
  Golang should have async/await and exceptions, both of which can be implemented as syntactic sugar on top of existing features.
  - throwaway894345 2y ago
    Having used both async/await and goroutines, I much prefer the latter. But yes, goroutines and channels are powerful tools that require understanding and discipline.
  - randomdata 2y ago
    Go has exceptions. Always has.
    - shepherdjerred 2y ago
      Yes, but when writing idiomatic Go one would prefer error types over using panic (panic is Go's term for an exception). The parent is really complaining about idiomatic Go. Which is fair, because idiomatic Go feels like writing Java 6 (that is to say, making lots of compromises due to limitations of the language).
kirlfiend_grill 2y ago
[flagged]
onionisafruit 2y ago
I didn’t realize why channels are part of the language vs a standard library feature until I read this. Now it makes sense that it’s about compiler optimizations with goroutines.
adeptima 2y ago
Well writen. Will recommend to all newcomers from other languages. As for uncovered topics or part 2 - long running Go channels can be a nightmare. You need to implement kind of observability for them. You must find the way to stop/run again/upgrade and even version payload to handle your channels/goroutine. Common problem with channels overuse is so called goroutine leaks. Happens more often than most devs think. Especially, if lib writers initiate goroutines in init() to maintain cache or do some background cleanup job. It's good to scan all used packages for such surprises. You might also find concepts like "durable execution" or "workflow" engines down the road.
- Intermernet 2y ago
  One pattern for long running channels is to have another polling / heartbeat channel on the goroutine. This is often redundant as you can achieve the basic functionality with contexts, but it can be useful when you need to implement some reporting, or you need to implement more advanced load balancing or back off strategies.
parhamn 2y ago
Go channel behaviors are pretty annoying. For one I always forget the panic scenarios (e.g. writing to a closed channel), I feel like the type system could've done more here. I recently wrote a simple function that maps out tasks concurrently, can be canceled by a context.WithCancel, or if a task fails. The things that cancel the task mapper need to coordinate very carefully on both sides of the channel so that they're closed and publishers stop sending in the right sequence. The amount of switches/cancels/signals quickly explode around the coordination if you too cute on how to do it (e.g. read from the error channel to stop the work). Frankly I'm not sure I still got it right [1]. And this is probably the most unsettling part. Rereading the code I can't possibly remember the cancellation semantics and ordering of the short mess I created. Now I'm wondering if mutexes would've made for more understandable code. [1] https://gist.github.com/pnegahdar/1783f0a4e03dc9a3da43478994923956 https://gist.github.com/pnegahdar/1783f0a4e03dc9a3da43478994...
- jerf 2y ago
  Writing to a closed channel at all is generally a design smell. Generally this is consumers closing channels, which is not a good idea. Only producers should close channels, generally only a producer that is the sole owner of the channel, and then, being the sole owner, it should "know" that it closed the channel and by its structure never write to it again. You probably are overcomplicating matters and need just one top-level channel, which in modern Go is the one contained in a context.Context, to be the one and only stop signal in the system. I think everyone goes through a bit of a complication phase with channels, I recognize your issues in code I've written myself, this is definitely not a "only a bad person would have this problem" post. But, yes, there probably is an organization that solves this problem. There's an art to using them properly. A good 50% of that art may well be that consumers should never close channels. (Another one is the utility of sending a channel as part of the message in a different channel. It is intuitively easy to think that channels must be very expensive, but when they're not participating in a select, they're just a handful of machine words in RAM. It is perfectly sensible to send a message to a "server" process that contains the channel in it to send the reply to, because it only costs a few machine words in allocation and precisely the one sync operation it will ever participate in. Channels do not have to amortize their costs with lots of messages; a 1-message channel is practical. This also cleaned up some complicated code I had before, trying to prematurely optimize something that was already very cheap.) The other thing is, if you haven't looked at https://pkg.go.dev/golang.org/x/sync/errgroup https://pkg.go.dev/golang.org/x/sync/errgroup , you may want to. golang.org/x/ is the not-as-well-known-as-it-should-be extended standard library; things the Go team are not willing to put into the 1.0 backwards compatibility promise so they retain the ability to change things if necessary, but otherwise de facto as high quality as the standard library, modulo some reasonably well-labeled exceptions. Contra some claims that it is impossible to abstract in Go, many of these common concurrency patterns have been abstracted out and you can and should grab them off the shelf.
tapirl 2y ago
> One literal interpretation of Pike's quote is that message passing is different than sharing memory for pedantic reasons. Copying values between sender/receiver stacks is safer than sharing memory. But it is also possible to send values with pointers into channels, so that doesn't really prevent developers from abusing the model. I don't think avoiding shared memory is a top of mind consideration for developers deciding whether to use channels. I think the point of Pike's quote is that, when a goroutine gets a pointer received from a channel, it gets the ownership of the values referenced by the pointer and other goroutines give up the ownership. This is a discipline Go programmers should hold but not a rule enforced by the language.
- jerf 2y ago
  As I like to say, Rust may be one of the only languages that has built ownership right into its type system, but the problems that "creates" with programming in Rust are actually problems revealed by programming in Rust, not created. The ownership problems are 100% there in other languages too, it just isn't a compiler error. It can help everyone, in any threaded language, to be thinking like Rust, even if the compiler and type system do not directly help you. A simple and useful degenerate case of this is that whenever any message is sent, complete ownership of the entire transitively-reachable set of values contained in that message is by default transferred to the receiver. This is worthwhile even if you must explicitly construct some safe value to pass in order to maintain this promise. This keeps things generally easy to reason about without the full complication and power that Rust offers. At the very least, whenever I violate this, I have lots of comments about what and why on both sides of that transaction in my code base. (I do sort of wish there was a variant of a "go" statement that I could use that made me statically promise that all communication in and out of a given goroutine must be solely in the form of copied values, so I could guarantee that goroutine was an "actor". This is, admittedly, just me wishing very pie-in-the-sky. I doubt it could be turned into a practical proposal.)
  - tick_tock_tick 2y ago
    To be technical Rust "creates" a lot of errors that aren't their because it doesn't, and can't, have full understanding of the control flow. Well it's less create and more complains about non issues. It why overtime code that used to be invalid Rust has become allowed as they've improved the borrow checker.
cyberax 2y ago
I must admit, I dislike Go channels. They are a hell to debug, as they are unnamed. And they're anonymous, so you can easily get a stacktrace filled with thousands of identical stacktraces that you can't correlate with logs. Golang needs to have a way to manage the channels better. Naming them and waiting on them would simplify a lot of crusty stuff. Naming is becoming possible, goroutines can already have pprof labels (that are even inherited between goroutines!), so just adding pprof labels to stacktraces will help a lot. But unfortunately, Go creators are allergic to anything that brings thread-local variables closer.
- tombert 2y ago
  This almost kind of feels like you're reinventing the Actor model (e.g. Erlang/Elixir).
  - r14c 2y ago
    go concurrency is built on a similar concept (concurrent sequential processes), if I'm not mistaken. the languages aren't very far apart in this regard to start with!
    - tombert 2y ago
      Yeah, I'm actually pretty familiar with CSP (I'm getting a degree in a related topic using a timed variant of it), though Go breaks from the formal CSP defintion language in a few ways (if nothing else vanilla CSP doesn't officially allow buffered channels). I do find the Actor model a bit nicer at an engineering level, simply because I find being able to refer to things by name to be a bit more intuitive, and since Actors (at least in Erlang) can manage their own state it's a lot easier to kind of shape them into doing anything you want (e.g. buffered or non-buffered). I'm much less familiar with the formal definitions of Actors though.
- rdtsc 2y ago
  > They are a hell to debug, as they are unnamed. And they're anonymous That was one odd thing that stood out to me about Go. I am coming from Erlang so a process having process ID we can keep track of, terminate it, trace, etc, is fundamental to be able to reason about and operate a system. Advertising that they can handle millions of lightweight goroutines but then having no obvious way to identify them, and monitor their lifetimes is kind of strange. On the lower level, I sort of understand why they did it: they focused on typed channels. So they can have multiple channels of different types potentially talking to the same goroutine. Now, having both named goroutines and channels would be more complicated, and they tried to keep things "simple". Erlang's processes on the other hand, have implicit mailboxes, there is no "mailbox1" and "mailbox2" there is only one, but it's also easier there because there is no static typing. There things are simple because only processes have identities but not mailboxes.
liampulles 2y ago
As an avid Go user, I think async/await is probably a nicer construct for most usecases. But Go channels work fine as long as you keep to the basics and documented patterns. I can recommend making a utility function which accepts a set of anonymous functions and a concurrency factor. I've since extended this function with a version which accepts a rate limiter, jitter factor, and retry count. This handles most cases where I need concurrency (batches) in a simple and safe way.
- adastra22 2y ago
  I guess the grass is always greener on the other side. As a Rust user, I’m constantly thinking “why didn’t they just give us go channels instead of this crazy async/await nonsense?”
  - steveklabnik 2y ago
    Rust has one form of channels in the standard library. More kinds available from crates.io. If you prefer channels, use them!
    - adastra22 2y ago
      Because one crate uses one channel implementation, and another crate uses a different one… this is really something that should be solved at the language / stdlib level, as Go does, but Rust falls a little bit short. There mpmc and the Send trait, but they alone are insufficient. So instead we get crate dependency hell that is only somewhat mitigated by cargo, and colored functions due to the async implementation.
- brianolson 2y ago
  Rust async/await is less nice than Go coroutines. There are things you can't do and weird rules around Rust async code. Every Go chan allows multiple readers and multiple writers, but Rust stdlib and tokio default to single-reader queues.
- initplus 2y ago
  Channels and async/await aren’t really equivalent features. Beyond the fact that they both deal with concurrency. You can do channels (message passing) on top of async await.
doawoo 2y ago
Y’all should really give Erlang/Elixir a try… this stuff is so much more trivial to deal with in that ecosystem and it pains me that it doesn’t get as much attention as Go does.
- tail_exchange 2y ago
  I am curious about Elixir, but I really can't bring myself to spend time mastering a language with dynamic types. You are removing one class of errors from your application, but at the same time inviting a new one that has been solved decades ago. I'm aware that this is a very controversial take, because lots of people love duck typed languages, but after working in large codebases, they are a hard pass for me. There's a reason why TypeScript was created for JavaScript, Sorbet was created for Ruby, and type hints is so popular with Python. I think I've seen something about gradual typing being introduced in Elixir, but gradual typing is still a long ways from enforcing type safety. Until then, I'll stick with Go. If I am mistaken and there is a "TypeScript for Elixir", then I would love to know about it.
  - innocentoldguy 2y ago
    I've worked with strongly typed languages and duck typed languages. I've also worked a lot with Go and Elixir. In my opinion, the tradeoff between the elegance of Elixir's actor-based functional style vs. Go's warty imperative style is more compelling than static typing vs. duck typing. I love working with Elixir. Go is something I tolerate.
    - tail_exchange 2y ago
      That's a fair trade if you don't like Go, but I love working with it, so for me it has to be something more compelling than trading safe types for actor-based functional style. Go's imperativeness never bothered me.
  - azurelake 2y ago
    Elixir is in the process of adding a gradual type system right now FWIW.
JaggerFoo 2y ago
Timely article for me, since I'm thinking of using channels in a real-world application for the first time.
DylanSp 2y ago
Channels can be a useful primitive, but without more structure they're tough to reason about. I really wish that the Go team would provide more implementations of common patterns and publicize them; things like x/sync/errgroup are fantastic, and I'd love to see more.

162 points by ingve 2y ago

I'll admit that I like Go channels because Hoare was the professor when I was doing my doctorate and so CSP was what I used in my thesis[1], but it's worth understanding what unbuffered channels give you: it's message passing with synchronization. They are very simple to reason about and make writing concurrent code a breeze. [1] https://blog.jgc.org/2024/03/the-formal-development-of-secure.html https://blog.jgc.org/2024/03/the-formal-development-of-secur...

bsaul 2y ago
i like the pattern, but message passing where the message is actually just a pointer to a mutable struct ( as allowed in go) kind of defeat the purpose, doesn't it ?
- superb_dev 2y ago
  It kinda does yeah, you probably shouldn’t do that
  - adastra22 2y ago
    It’s fine if that’s the only (writable) pointer to the struct. Which is something go doesn’t enforce, but Rust does. I often dream of Go with Rust’s borrow checker. Sigh…
    - jimbokun 2y ago
      Wouldn't that be Rust?
      - adastra22 2y ago
        Rust doesn’t have a direct analogue to go’s concurrency primitives. The pieces are there, but you have to jump through hoops to use them effectively.
- dgb23 2y ago
  No, it’s an idiomatic way of expressing who’s in charge of a given piece of data. There’s nothing in the type system preventing data races. But that’s orthogonal to channels. They just help you to structure the code.
bjoli 2y ago
I come from the complete opposite side: I have been writing some concurrentML code in guile scheme recently, and it has really made me understand why I always disliked go's channels. There are just so many things that are just slightly wrong to make it unpleasant. Better than many other things, but still kind of frustrating. I really think unbuffered channels should have been the default.

hedora 2y ago

I’ve worked with a few large code bases that use channels. In all those code bases, they were about as maintainable as GOTO-based control flow, except that GOTO makes you have unique label names. All the channel-based code I’ve seen just has 100’s of call sites like “chan->send()” and “chan->recv()” sprinkled around, and doesn’t even have the discipline to put related senders and receivers in the same source file. At least old-school syntax like “GOTO foobar_step_23” and “LABEL foobar_step_23” is grepable). I greatly prefer programs that “color” functions sync/async, and that use small state machines to coordinate shared state only when necessary. Go technically supports this, but it doesn’t seem like it is idiomatic (unlike rust, the go compiler won’t help with data races, and unlike C++, people don’t assume that the language is a giant foot-gun).

astockwell 2y ago
Go does have the `- race` flag for test + build commands [1], which can help (not 100% iirc, but some) with datarace detection. [1] https://go.dev/doc/articles/race_detector https://go.dev/doc/articles/race_detector
hot_gril 2y ago
Golang should have async/await and exceptions, both of which can be implemented as syntactic sugar on top of existing features.
- throwaway894345 2y ago
  Having used both async/await and goroutines, I much prefer the latter. But yes, goroutines and channels are powerful tools that require understanding and discipline.
- randomdata 2y ago
  Go has exceptions. Always has.
  - shepherdjerred 2y ago
    Yes, but when writing idiomatic Go one would prefer error types over using panic (panic is Go's term for an exception). The parent is really complaining about idiomatic Go. Which is fair, because idiomatic Go feels like writing Java 6 (that is to say, making lots of compromises due to limitations of the language).

kirlfiend_grill 2y ago

[flagged]

onionisafruit 2y ago

I didn’t realize why channels are part of the language vs a standard library feature until I read this. Now it makes sense that it’s about compiler optimizations with goroutines.

adeptima 2y ago

Well writen. Will recommend to all newcomers from other languages. As for uncovered topics or part 2 - long running Go channels can be a nightmare. You need to implement kind of observability for them. You must find the way to stop/run again/upgrade and even version payload to handle your channels/goroutine. Common problem with channels overuse is so called goroutine leaks. Happens more often than most devs think. Especially, if lib writers initiate goroutines in init() to maintain cache or do some background cleanup job. It's good to scan all used packages for such surprises. You might also find concepts like "durable execution" or "workflow" engines down the road.

Intermernet 2y ago
One pattern for long running channels is to have another polling / heartbeat channel on the goroutine. This is often redundant as you can achieve the basic functionality with contexts, but it can be useful when you need to implement some reporting, or you need to implement more advanced load balancing or back off strategies.

parhamn 2y ago

Go channel behaviors are pretty annoying. For one I always forget the panic scenarios (e.g. writing to a closed channel), I feel like the type system could've done more here. I recently wrote a simple function that maps out tasks concurrently, can be canceled by a context.WithCancel, or if a task fails. The things that cancel the task mapper need to coordinate very carefully on both sides of the channel so that they're closed and publishers stop sending in the right sequence. The amount of switches/cancels/signals quickly explode around the coordination if you too cute on how to do it (e.g. read from the error channel to stop the work). Frankly I'm not sure I still got it right [1]. And this is probably the most unsettling part. Rereading the code I can't possibly remember the cancellation semantics and ordering of the short mess I created. Now I'm wondering if mutexes would've made for more understandable code. [1] https://gist.github.com/pnegahdar/1783f0a4e03dc9a3da43478994923956 https://gist.github.com/pnegahdar/1783f0a4e03dc9a3da43478994...

jerf 2y ago
Writing to a closed channel at all is generally a design smell. Generally this is consumers closing channels, which is not a good idea. Only producers should close channels, generally only a producer that is the sole owner of the channel, and then, being the sole owner, it should "know" that it closed the channel and by its structure never write to it again. You probably are overcomplicating matters and need just one top-level channel, which in modern Go is the one contained in a context.Context, to be the one and only stop signal in the system. I think everyone goes through a bit of a complication phase with channels, I recognize your issues in code I've written myself, this is definitely not a "only a bad person would have this problem" post. But, yes, there probably is an organization that solves this problem. There's an art to using them properly. A good 50% of that art may well be that consumers should never close channels. (Another one is the utility of sending a channel as part of the message in a different channel. It is intuitively easy to think that channels must be very expensive, but when they're not participating in a select, they're just a handful of machine words in RAM. It is perfectly sensible to send a message to a "server" process that contains the channel in it to send the reply to, because it only costs a few machine words in allocation and precisely the one sync operation it will ever participate in. Channels do not have to amortize their costs with lots of messages; a 1-message channel is practical. This also cleaned up some complicated code I had before, trying to prematurely optimize something that was already very cheap.) The other thing is, if you haven't looked at https://pkg.go.dev/golang.org/x/sync/errgroup https://pkg.go.dev/golang.org/x/sync/errgroup , you may want to. golang.org/x/ is the not-as-well-known-as-it-should-be extended standard library; things the Go team are not willing to put into the 1.0 backwards compatibility promise so they retain the ability to change things if necessary, but otherwise de facto as high quality as the standard library, modulo some reasonably well-labeled exceptions. Contra some claims that it is impossible to abstract in Go, many of these common concurrency patterns have been abstracted out and you can and should grab them off the shelf.

tapirl 2y ago

> One literal interpretation of Pike's quote is that message passing is different than sharing memory for pedantic reasons. Copying values between sender/receiver stacks is safer than sharing memory. But it is also possible to send values with pointers into channels, so that doesn't really prevent developers from abusing the model. I don't think avoiding shared memory is a top of mind consideration for developers deciding whether to use channels. I think the point of Pike's quote is that, when a goroutine gets a pointer received from a channel, it gets the ownership of the values referenced by the pointer and other goroutines give up the ownership. This is a discipline Go programmers should hold but not a rule enforced by the language.

jerf 2y ago
As I like to say, Rust may be one of the only languages that has built ownership right into its type system, but the problems that "creates" with programming in Rust are actually problems revealed by programming in Rust, not created. The ownership problems are 100% there in other languages too, it just isn't a compiler error. It can help everyone, in any threaded language, to be thinking like Rust, even if the compiler and type system do not directly help you. A simple and useful degenerate case of this is that whenever any message is sent, complete ownership of the entire transitively-reachable set of values contained in that message is by default transferred to the receiver. This is worthwhile even if you must explicitly construct some safe value to pass in order to maintain this promise. This keeps things generally easy to reason about without the full complication and power that Rust offers. At the very least, whenever I violate this, I have lots of comments about what and why on both sides of that transaction in my code base. (I do sort of wish there was a variant of a "go" statement that I could use that made me statically promise that all communication in and out of a given goroutine must be solely in the form of copied values, so I could guarantee that goroutine was an "actor". This is, admittedly, just me wishing very pie-in-the-sky. I doubt it could be turned into a practical proposal.)
- tick_tock_tick 2y ago
  To be technical Rust "creates" a lot of errors that aren't their because it doesn't, and can't, have full understanding of the control flow. Well it's less create and more complains about non issues. It why overtime code that used to be invalid Rust has become allowed as they've improved the borrow checker.

cyberax 2y ago

I must admit, I dislike Go channels. They are a hell to debug, as they are unnamed. And they're anonymous, so you can easily get a stacktrace filled with thousands of identical stacktraces that you can't correlate with logs. Golang needs to have a way to manage the channels better. Naming them and waiting on them would simplify a lot of crusty stuff. Naming is becoming possible, goroutines can already have pprof labels (that are even inherited between goroutines!), so just adding pprof labels to stacktraces will help a lot. But unfortunately, Go creators are allergic to anything that brings thread-local variables closer.

tombert 2y ago
This almost kind of feels like you're reinventing the Actor model (e.g. Erlang/Elixir).
- r14c 2y ago
  go concurrency is built on a similar concept (concurrent sequential processes), if I'm not mistaken. the languages aren't very far apart in this regard to start with!
  - tombert 2y ago
    Yeah, I'm actually pretty familiar with CSP (I'm getting a degree in a related topic using a timed variant of it), though Go breaks from the formal CSP defintion language in a few ways (if nothing else vanilla CSP doesn't officially allow buffered channels). I do find the Actor model a bit nicer at an engineering level, simply because I find being able to refer to things by name to be a bit more intuitive, and since Actors (at least in Erlang) can manage their own state it's a lot easier to kind of shape them into doing anything you want (e.g. buffered or non-buffered). I'm much less familiar with the formal definitions of Actors though.
rdtsc 2y ago
> They are a hell to debug, as they are unnamed. And they're anonymous That was one odd thing that stood out to me about Go. I am coming from Erlang so a process having process ID we can keep track of, terminate it, trace, etc, is fundamental to be able to reason about and operate a system. Advertising that they can handle millions of lightweight goroutines but then having no obvious way to identify them, and monitor their lifetimes is kind of strange. On the lower level, I sort of understand why they did it: they focused on typed channels. So they can have multiple channels of different types potentially talking to the same goroutine. Now, having both named goroutines and channels would be more complicated, and they tried to keep things "simple". Erlang's processes on the other hand, have implicit mailboxes, there is no "mailbox1" and "mailbox2" there is only one, but it's also easier there because there is no static typing. There things are simple because only processes have identities but not mailboxes.

liampulles 2y ago

As an avid Go user, I think async/await is probably a nicer construct for most usecases. But Go channels work fine as long as you keep to the basics and documented patterns. I can recommend making a utility function which accepts a set of anonymous functions and a concurrency factor. I've since extended this function with a version which accepts a rate limiter, jitter factor, and retry count. This handles most cases where I need concurrency (batches) in a simple and safe way.

adastra22 2y ago
I guess the grass is always greener on the other side. As a Rust user, I’m constantly thinking “why didn’t they just give us go channels instead of this crazy async/await nonsense?”
- steveklabnik 2y ago
  Rust has one form of channels in the standard library. More kinds available from crates.io. If you prefer channels, use them!
  - adastra22 2y ago
    Because one crate uses one channel implementation, and another crate uses a different one… this is really something that should be solved at the language / stdlib level, as Go does, but Rust falls a little bit short. There mpmc and the Send trait, but they alone are insufficient. So instead we get crate dependency hell that is only somewhat mitigated by cargo, and colored functions due to the async implementation.
brianolson 2y ago
Rust async/await is less nice than Go coroutines. There are things you can't do and weird rules around Rust async code. Every Go chan allows multiple readers and multiple writers, but Rust stdlib and tokio default to single-reader queues.
initplus 2y ago
Channels and async/await aren’t really equivalent features. Beyond the fact that they both deal with concurrency. You can do channels (message passing) on top of async await.

doawoo 2y ago

Y’all should really give Erlang/Elixir a try… this stuff is so much more trivial to deal with in that ecosystem and it pains me that it doesn’t get as much attention as Go does.

tail_exchange 2y ago
I am curious about Elixir, but I really can't bring myself to spend time mastering a language with dynamic types. You are removing one class of errors from your application, but at the same time inviting a new one that has been solved decades ago. I'm aware that this is a very controversial take, because lots of people love duck typed languages, but after working in large codebases, they are a hard pass for me. There's a reason why TypeScript was created for JavaScript, Sorbet was created for Ruby, and type hints is so popular with Python. I think I've seen something about gradual typing being introduced in Elixir, but gradual typing is still a long ways from enforcing type safety. Until then, I'll stick with Go. If I am mistaken and there is a "TypeScript for Elixir", then I would love to know about it.
- innocentoldguy 2y ago
  I've worked with strongly typed languages and duck typed languages. I've also worked a lot with Go and Elixir. In my opinion, the tradeoff between the elegance of Elixir's actor-based functional style vs. Go's warty imperative style is more compelling than static typing vs. duck typing. I love working with Elixir. Go is something I tolerate.
  - tail_exchange 2y ago
    That's a fair trade if you don't like Go, but I love working with it, so for me it has to be something more compelling than trading safe types for actor-based functional style. Go's imperativeness never bothered me.
- azurelake 2y ago
  Elixir is in the process of adding a gradual type system right now FWIW.

JaggerFoo 2y ago

Timely article for me, since I'm thinking of using channels in a real-world application for the first time.

DylanSp 2y ago

Channels can be a useful primitive, but without more structure they're tough to reason about. I really wish that the Go team would provide more implementations of common patterns and publicize them; things like x/sync/errgroup are fantastic, and I'd love to see more.