4 ms·

My experience with Goroutines is that eventually you run into the limits of working with them and your code explodes in size. Your for item := range channe

My experience with Goroutines is that eventually you run into the limits of working with them and your code explodes in size. Your for item := range channel { // code goes here } becomes for { var item Item var ok bool select { case <-ctx.Done(): break case item, ok = <-channel: if !ok { break } } // code goes here } (If you have a solution to this pattern, please do share! I would _love_ a better way.)

quadcore 3y ago
It does become a channel mess. The trick I use is to code everything using the same patterns (down to the same characters, name conventions, line breaks, etc) as much as I can so i can use the text editor to work (refactor) that mass. It's definitely the less funny part and why I'll still use clojure and it's async.io module whenever it makes sense (lots of different data). That way, I macro myself out of that problem. Now if you cant use another language, change project. Go is not perl nor python, it's a sophisticated C. Use a sophisticated C for the wrong project / product and you'll want to kill yourself.
- anacrolix 3y ago
  It's more like a toothless C. In that it knows a bunch of tricks already, you can't teach it anything new, it can't bite you, and it's mostly harmless.
  - quadcore 3y ago
    Thumb up to that.
chrsig 3y ago
Generics! func streamingApply[T any](ch channel[T], fn func(T)){ for { select { case <-ctx.Done(): break case item, ok = <-ch: if !ok { break } fn(item) } } } For everyone that's complained about generics being introduced to go...It really does help clean up this type of repetition.
PurelyApplied 3y ago
Do you typically need to check both `Done` and `ok`? The latter is only false when the channel is empty and closed, so unless multiple routines are pulling from that channel, you should only need one, should you not?
- rollulus 3y ago
  You absolutely should, because context cancellation does not imply a closed channel and vice versa.
  - PurelyApplied 3y ago
    Oh, duh. I shouldn't try to read code past midnight... Thank you!
collinvandyck76 3y ago
If your problem fits, you can sometimes keep the former version in play with the pipeline pattern, where child goroutines are stopped when the parent goroutine closes the channel in a defer when it's exiting. That being said, my experience is that anything moderately sophisticated requires the use of for/select. In my opinion, that's fine, because select gives you so much power the somewhat more complex form is worth it.
Philip-J-Fry 3y ago
What's the limit here? That code is written like that because it's the only way to write code like that. You have a channel producing items, it will produce items for as long as the channel is open. You have a context to propagate cancellation. You code gets bigger but it doesn't explode in size as this boilerplate is static in size regardless of what computation you're doing with those items. You can improve that code by the way by fixing a bug and reducing the lines. L: for { select { case <-ctx.Done(): break L case item, ok := <-channel: if !ok { break L } // code goes here } } If you think there's a way to write that code any more concise than that, then there isn't. I don't see how this is an issue. The fact that there's pretty much one way to write this code makes it so much easier to parse when reading code that uses this pattern. To me, that's great.
- deleted 3y ago
  [deleted]
- valczir 3y ago
  alternatively: var errClosed = errors.New("closed") func doAThing[T any](ctx context.Context, c chan T) (T, error) { // the generics are not necessary, but they're also not awful for // DRYing up this type of a select. select { case <-ctx.Done(): var empty T return empty, fmt.Errorf("could not doAThing: %w", ctx.Err()) case item, ok := <-c: if !ok { var empty T return empty, fmt.Errorf("channel of type %T is done: %w", empty, errClosed) } return item, nil } } // ... and later... for { item, err := doAThing(ctx, channel) if err != nil { break } // code goes here } I dislike labeled loops, naked returns, and else statements, so this is the pattern I use.
- anacrolix 3y ago
  No. Just no.
kinghajj 3y ago
If your loop body is completely CPU-bound, then either adding a select statement, or a separate goroutine that awaits cancellation and closes the channel like collinvandyck76 suggested, will work. But commonly the body will make other IO-bound calls to which the context gets passed, so you can just handle the error. for item := range channel { _, err := process(ctx, item) if err != nil { return err } }

quadcore 3y ago

It does become a channel mess. The trick I use is to code everything using the same patterns (down to the same characters, name conventions, line breaks, etc) as much as I can so i can use the text editor to work (refactor) that mass. It's definitely the less funny part and why I'll still use clojure and it's async.io module whenever it makes sense (lots of different data). That way, I macro myself out of that problem. Now if you cant use another language, change project. Go is not perl nor python, it's a sophisticated C. Use a sophisticated C for the wrong project / product and you'll want to kill yourself.

anacrolix 3y ago
It's more like a toothless C. In that it knows a bunch of tricks already, you can't teach it anything new, it can't bite you, and it's mostly harmless.
- quadcore 3y ago
  Thumb up to that.

chrsig 3y ago

Generics! func streamingApply[T any](ch channel[T], fn func(T)){ for { select { case <-ctx.Done(): break case item, ok = <-ch: if !ok { break } fn(item) } } } For everyone that's complained about generics being introduced to go...It really does help clean up this type of repetition.

PurelyApplied 3y ago

Do you typically need to check both `Done` and `ok`? The latter is only false when the channel is empty and closed, so unless multiple routines are pulling from that channel, you should only need one, should you not?

rollulus 3y ago
You absolutely should, because context cancellation does not imply a closed channel and vice versa.
- PurelyApplied 3y ago
  Oh, duh. I shouldn't try to read code past midnight... Thank you!

collinvandyck76 3y ago

If your problem fits, you can sometimes keep the former version in play with the pipeline pattern, where child goroutines are stopped when the parent goroutine closes the channel in a defer when it's exiting. That being said, my experience is that anything moderately sophisticated requires the use of for/select. In my opinion, that's fine, because select gives you so much power the somewhat more complex form is worth it.

Philip-J-Fry 3y ago

What's the limit here? That code is written like that because it's the only way to write code like that. You have a channel producing items, it will produce items for as long as the channel is open. You have a context to propagate cancellation. You code gets bigger but it doesn't explode in size as this boilerplate is static in size regardless of what computation you're doing with those items. You can improve that code by the way by fixing a bug and reducing the lines. L: for { select { case <-ctx.Done(): break L case item, ok := <-channel: if !ok { break L } // code goes here } } If you think there's a way to write that code any more concise than that, then there isn't. I don't see how this is an issue. The fact that there's pretty much one way to write this code makes it so much easier to parse when reading code that uses this pattern. To me, that's great.

deleted 3y ago
[deleted]
valczir 3y ago
alternatively: var errClosed = errors.New("closed") func doAThing[T any](ctx context.Context, c chan T) (T, error) { // the generics are not necessary, but they're also not awful for // DRYing up this type of a select. select { case <-ctx.Done(): var empty T return empty, fmt.Errorf("could not doAThing: %w", ctx.Err()) case item, ok := <-c: if !ok { var empty T return empty, fmt.Errorf("channel of type %T is done: %w", empty, errClosed) } return item, nil } } // ... and later... for { item, err := doAThing(ctx, channel) if err != nil { break } // code goes here } I dislike labeled loops, naked returns, and else statements, so this is the pattern I use.
anacrolix 3y ago
No. Just no.

kinghajj 3y ago

If your loop body is completely CPU-bound, then either adding a select statement, or a separate goroutine that awaits cancellation and closes the channel like collinvandyck76 suggested, will work. But commonly the body will make other IO-bound calls to which the context gets passed, so you can just handle the error. for item := range channel { _, err := process(ctx, item) if err != nil { return err } }