5 ms·

> Go makes it harder by making a write to a closed channel kill the whole program. So if a receiver decides it needs to shut down, it can't just close the rece

> Go makes it harder by making a write to a closed channel kill the whole program. So if a receiver decides it needs to shut down, it can't just close the receive end of the channel and let the sender hit a send error. Negotiations are required to get the sender shut down. And thus Borg was born. Too hard to write processes that handle situations gracefully, so let's just let the process die and spin up a new one immediately. Literally was looking at a patch set that was proposed recently that involved several channels in Go and one of the codepaths was os.Exit(1) for lack of any meaningful mechanism to sanely clean everything up. Sadly, this was in library code, so would be totally unexpected by the caller and probably not desirable whatsoever.

Animats 6y ago
And thus Borg was born. Yes. Basic Dijkstra producer-consumer one-way queues are simple and elegant. But the error cases are hell. OK, so if the consumer wants to make the producer stop, it sets an abort flag. That's a shared variable. (Does it need a lock? Does it need a lock or fencing on ARM, which has weaker memory concurrency guarantees than x86?) So, producer sees the abort flag set and closes its end of the channel, right? Maybe not. Maybe the channel is full, and the producer is blocked on a send. OK, so, when the consumer aborts, it has to enter a drain loop, doing receives and discarding the result until it gets an EOF/channel close. That will unblock the producer, and then the consumer can exit. But what if the producer doesn't have anything to send right now? Now the consumer is stuck waiting for the producer to send more data, and the consumer can't exit. Should the consumer close the producer's end of the channel after setting the abort flag? Now there's a race condition between checking the abort flag and sending. Add a critical section that covers both actions? Now the producer can be blocked while in a critical section. This creates a potential deadlock if the consumer wants to set the abort flag while the producer is inside the critical section. Now both ends are blocked and there's a deadlock. But it all looked so simple in theory!
- codr7 6y ago
  If you insist on making it as difficult as possible, I guess. Otherwise the producer decides when the show stops, by simply closing the channel.
  - earthboundkid 6y ago
    Yes, exactly. The workers can send results to the manager, but the manager does all the management. Anything else gets too hairy, too fast to reason about. I wrote a blog post about it: https://blog.carlmjohnson.net/post/share-memory-by-communicating/ https://blog.carlmjohnson.net/post/share-memory-by-communica...
    - Animats 6y ago
      Does that scale to a pipeline, a chain of producers and consumers? What happened with Go, I suspect, is that the syntax came first. Go uses a "<-" operator to talk to channels. There's no place for an error status with that operator. Hence the problem. UNIX-type pipelines have SIGPIPE to unwind things when something downstream aborts. Abort has to propagate back up the chain. I tend to argue that the workarounds for not having exceptions are worse than the problems of exceptions.
  - fovc 6y ago
    The ECMAScript TC came to the same conclusion with promises and fetch. Instead of adding an `.abort` method to promises for the consumer to use, you need to wire in an AbortController so that the producer (fetch) can close the channel. The consumer then sees an AbortError. It's a bit more work to wire up than simply calling `p.abort()`, but it saves you from having to reason through some hairy race conditions.

Animats 6y ago

And thus Borg was born. Yes. Basic Dijkstra producer-consumer one-way queues are simple and elegant. But the error cases are hell. OK, so if the consumer wants to make the producer stop, it sets an abort flag. That's a shared variable. (Does it need a lock? Does it need a lock or fencing on ARM, which has weaker memory concurrency guarantees than x86?) So, producer sees the abort flag set and closes its end of the channel, right? Maybe not. Maybe the channel is full, and the producer is blocked on a send. OK, so, when the consumer aborts, it has to enter a drain loop, doing receives and discarding the result until it gets an EOF/channel close. That will unblock the producer, and then the consumer can exit. But what if the producer doesn't have anything to send right now? Now the consumer is stuck waiting for the producer to send more data, and the consumer can't exit. Should the consumer close the producer's end of the channel after setting the abort flag? Now there's a race condition between checking the abort flag and sending. Add a critical section that covers both actions? Now the producer can be blocked while in a critical section. This creates a potential deadlock if the consumer wants to set the abort flag while the producer is inside the critical section. Now both ends are blocked and there's a deadlock. But it all looked so simple in theory!

codr7 6y ago
If you insist on making it as difficult as possible, I guess. Otherwise the producer decides when the show stops, by simply closing the channel.
- earthboundkid 6y ago
  Yes, exactly. The workers can send results to the manager, but the manager does all the management. Anything else gets too hairy, too fast to reason about. I wrote a blog post about it: https://blog.carlmjohnson.net/post/share-memory-by-communicating/ https://blog.carlmjohnson.net/post/share-memory-by-communica...
  - Animats 6y ago
    Does that scale to a pipeline, a chain of producers and consumers? What happened with Go, I suspect, is that the syntax came first. Go uses a "<-" operator to talk to channels. There's no place for an error status with that operator. Hence the problem. UNIX-type pipelines have SIGPIPE to unwind things when something downstream aborts. Abort has to propagate back up the chain. I tend to argue that the workarounds for not having exceptions are worse than the problems of exceptions.
- fovc 6y ago
  The ECMAScript TC came to the same conclusion with promises and fetch. Instead of adding an `.abort` method to promises for the consumer to use, you need to wire in an AbortController so that the producer (fetch) can close the channel. The consumer then sees an AbortError. It's a bit more work to wire up than simply calling `p.abort()`, but it saves you from having to reason through some hairy race conditions.