3 ms·

This feels like it's undermining channels, the feature go really wants you to use in other places too (but people tend to still use mutexes). Channels aren't qu

by _cenw 2y ago

This feels like it's undermining channels, the feature go really wants you to use in other places too (but people tend to still use mutexes). Channels aren't quite as lazy (if you use an unbuffered one you supply one element in advance), but they're close.

jakjak123 2y ago
Channels complicate error handling quite a bit, so I tend to avoid them
- tapirl 2y ago
  Could you elaborate more?
  - jakjak123 2y ago
    If you have shipped some task to a channel, or is waiting for some work to complete on a channel, there is no native way to propagate the error that your task may have failed. Also if a error did happen during processing the task you put on the channel, the stacktrace suddenly is not the whole story anymore. Channels also has no way to make sure the context.Context is reasonably propagated
    - tapirl 2y ago
      Is it possible to enhance channels to fit into the iterator need? I mean using them without always creating new goroutines.
thwarted 2y ago
Here's a rough implementation of iterators using channels that I was experimenting with. It needs some syntactic sugar to do proper cleanup of the go routine if the loop exits early; probably the iterator function needs to return a func that closes the channel called after the loop, and some error handling/closed channel checking. https://gist.github.com/thwarted/a6e5e7ca5ce552311a7d5ece13d298ac https://gist.github.com/thwarted/a6e5e7ca5ce552311a7d5ece13d...
- tapirl 2y ago
  What is meaningfulness of the two `yield` functions? I mean why not expand/inline them. To use channels as iterators efficiently, we need find a way to let the functions creating the channels return without creating new goroutines.
  - thwarted 2y ago
    The yield functions only exist as syntactic sugar to make it look like iterators in other languages and to make it clear where the value emission point is (I had mentioned this in tweets and skeets when I was originally working on this, if didn't make it into the gist). An unbuffered channel is really a scheduler abstraction. Consuming from an unbuffered channel blocks, the thread can enter and immediately begin executing the go routine that was blocking on producing. The go routine is acting like a closure around the channel state. I had some further experiments interleaving these iterators, but didn't clean it up at the time before I had sufficiently convinced myself it was possible and I got distracted with other things.
    - tapirl 2y ago
      The idea listens promising. It is more intuitive than the current design.
blixt 2y ago
Channels require a different goroutine to send values while also receiving them (which is how you’d have two loops communicating, essentially what you get from range funcs). There’s nothing stopping you from doing this but it does mean you are introducing the requirement of thread safety in your code, in the case where the iterator is stateful. I would argue anything that needs a range func beyond the simple functional things like filters is probably a stateful iterator (or generator if you’d like), and as such having range funcs is a great way to write code that doesn’t go wrong due to parallelism. Now you could add two way communication to your channel iterator (or any other locking mechanism) for safety but honestly I think range funcs perfectly solve this use case, and have already used them to keep my code more readable and correct. All this said, while I’m still a fan of Go and have used it regularly since 0.9 as well as contributed to the language, I will agree with the other comments that sometimes the language design bends over backward to be purist at the cost of having to add more footguns in user land.
- tapirl 2y ago
  > Channels require a different goroutine to send values while also receiving them ... The requirement is a limit of the current design and implementation. Channels can be enhanced to avoid the requirement.
masklinn 2y ago
For this use case, not only do channels add an insane amount of overhead, they're also broken in all sorts of way e.g. there is no way to properly clean up resources around the iteration, and they add more opportunities for race conditions since the object under iteration has to be shared with the channel's producing goroutine.
mxey 2y ago
https://research.swtch.com/coro https://research.swtch.com/coro at the very end explains that channels are too slow. Re mutexes: one of Rob Pike‘s Go Proverbs (https://go-proverbs.github.io/ https://go-proverbs.github.io/) was "channels orchestrate, mutexes serialize“. Both have their places.

by _cenw 2y ago

jakjak123 2y ago

Channels complicate error handling quite a bit, so I tend to avoid them

tapirl 2y ago
Could you elaborate more?
- jakjak123 2y ago
  If you have shipped some task to a channel, or is waiting for some work to complete on a channel, there is no native way to propagate the error that your task may have failed. Also if a error did happen during processing the task you put on the channel, the stacktrace suddenly is not the whole story anymore. Channels also has no way to make sure the context.Context is reasonably propagated
  - tapirl 2y ago
    Is it possible to enhance channels to fit into the iterator need? I mean using them without always creating new goroutines.

thwarted 2y ago

Here's a rough implementation of iterators using channels that I was experimenting with. It needs some syntactic sugar to do proper cleanup of the go routine if the loop exits early; probably the iterator function needs to return a func that closes the channel called after the loop, and some error handling/closed channel checking. https://gist.github.com/thwarted/a6e5e7ca5ce552311a7d5ece13d298ac https://gist.github.com/thwarted/a6e5e7ca5ce552311a7d5ece13d...

tapirl 2y ago
What is meaningfulness of the two `yield` functions? I mean why not expand/inline them. To use channels as iterators efficiently, we need find a way to let the functions creating the channels return without creating new goroutines.
- thwarted 2y ago
  The yield functions only exist as syntactic sugar to make it look like iterators in other languages and to make it clear where the value emission point is (I had mentioned this in tweets and skeets when I was originally working on this, if didn't make it into the gist). An unbuffered channel is really a scheduler abstraction. Consuming from an unbuffered channel blocks, the thread can enter and immediately begin executing the go routine that was blocking on producing. The go routine is acting like a closure around the channel state. I had some further experiments interleaving these iterators, but didn't clean it up at the time before I had sufficiently convinced myself it was possible and I got distracted with other things.
  - tapirl 2y ago
    The idea listens promising. It is more intuitive than the current design.

blixt 2y ago

Channels require a different goroutine to send values while also receiving them (which is how you’d have two loops communicating, essentially what you get from range funcs). There’s nothing stopping you from doing this but it does mean you are introducing the requirement of thread safety in your code, in the case where the iterator is stateful. I would argue anything that needs a range func beyond the simple functional things like filters is probably a stateful iterator (or generator if you’d like), and as such having range funcs is a great way to write code that doesn’t go wrong due to parallelism. Now you could add two way communication to your channel iterator (or any other locking mechanism) for safety but honestly I think range funcs perfectly solve this use case, and have already used them to keep my code more readable and correct. All this said, while I’m still a fan of Go and have used it regularly since 0.9 as well as contributed to the language, I will agree with the other comments that sometimes the language design bends over backward to be purist at the cost of having to add more footguns in user land.

tapirl 2y ago
> Channels require a different goroutine to send values while also receiving them ... The requirement is a limit of the current design and implementation. Channels can be enhanced to avoid the requirement.

masklinn 2y ago

For this use case, not only do channels add an insane amount of overhead, they're also broken in all sorts of way e.g. there is no way to properly clean up resources around the iteration, and they add more opportunities for race conditions since the object under iteration has to be shared with the channel's producing goroutine.

mxey 2y ago

https://research.swtch.com/coro https://research.swtch.com/coro at the very end explains that channels are too slow. Re mutexes: one of Rob Pike‘s Go Proverbs (https://go-proverbs.github.io/ https://go-proverbs.github.io/) was "channels orchestrate, mutexes serialize“. Both have their places.