4 ms·

> APIs should be designed synchronously, and the callers should orchestrate concurrency if they choose Wait, why would you say that? In general, if "orchestra

> APIs should be designed synchronously, and the callers should orchestrate concurrency if they choose Wait, why would you say that? In general, if "orchestrating concurrency" involves guarding access to shared mutable state, then you can't orchestrate it at the callers site. It would be a massive encapsulation leak, because synchronization is not composable, requires special knowledge, plus you don't necessarily know how to synchronize unless you have knowledge about internals. Furthermore, because it is after the fact, your only choice of handling it is by means of mutexes, which has really terrible performance characteristics. Even if you could do ordering by other means, you end up paying the price of LOCK XCHG or whatever mutexes translate to, not to mention that you'll have problems if you want (soft) real-time behavior, because now you can end up with both dead-locks and live-locks. And this brings us to another problem. If you end up doing such synchronization in Go, then Go's M:N multi-threading ends up doing more harm than good, because if you need such synchronization, you also need to fine tune your thread-pools and at this point 1:1 would be better. On top of 1:1 platforms you can build M:N solutions, but it doesn't work well in the other direction. > Novices to the language have a tendency to overuse channels Novices to software development have a tendency to overuse mutexes as well.

sagichmal 11y ago
> In general, if "orchestrating concurrency" involves guarding access to shared mutable state, then you can't orchestrate it at the callers site. Shared mutable state is generally behind an API boundary. I'm talking about the exported method set of that API. That is, func (f *Foo) Update(i int) (int, error) // yes func (f *Foo) Update(i int) (<-chan int, error) // no func (f *Foo) Update(i int, result chan<- int) error // no
jerf 11y ago
"In general, if "orchestrating concurrency" involves guarding access to shared mutable state, then you can't orchestrate it at the callers site." What this generally means in Go is that it is an antipattern for your library to provide something like "a method that makes an HTTP request in a goroutine". In Go, you should simply provide code that "makes an HTTP request", and it's up to the user to decide whether they want to run that in a goroutine. The rest of what you're talking about is a completely different issue. Channels are smelly in an API. IIRC in the entire standard library there's less than 10 functions/methods that return a channel. But the use case does occasionally arise.
pmarreck 11y ago
Reading this... only makes me gladder that I'm pursuing work in the Erlang/Elixir space, where messaging "just works" and concurrency "just works" and immutability "just works" (and new processes take a microsecond to spin up) and tearing anything down is basically a nonissue as failure is embraced and logged at every turn and cascading teardowns simply happen automatically depending on how the processes are linked and all this turns out to be a really amazing system of paradigms when designing apps to work in the real world
- svanderbleek 11y ago
  It "just works" in Go too, minus immutability, but congrats on your technology decision. You don't get type checking but c'est la vie.
  - raould42 11y ago
    Dialyzer
  - pmarreck 11y ago
    Restricting input based on type hierarchies can reduce a certain class of bugs, yes, but careful use of guards as well as typespecs and unit test coverage (which you should have, anyway) can accomplish much of what type restrictions can
    - pmarreck 11y ago
      Was something I said factually wrong? User im_down_w_otp put up an example of what I'm talking about (minus the unit testing) so what gives?
      - sagichmal 11y ago
        I suspect dismissing static typing whole cloth with "unit tests and certain guards can give you most of the benefits" comes off badly to some people.
pron 11y ago
> you end up paying the price of LOCK XCHG or whatever mutexes translate to But channels use locks internally. The choice of channels vs. mutexes is one of design, not implementation. Also, mutexes are blocking; LOCK XCHG isn't. Sure, mutexes also use LOCK XCHG (but so do channels, and nearly all concurrent data sctructures), but they also block (as do channels). > your only choice of handling it is by means of mutexes, which has really terrible performance characteristics That's just not true. There is a way to translate any locking algorithm to a non-blocing one (in fact, wait-free, which is the "strongest" non-blocking guarantee), yet only only a handful of wait-free algorithms are used in practice. Why? Because it's hard to make them more efficient than locks in the general case. > not to mention that you'll have problems if you want (soft) real-time behavior, because now you can end up with both dead-locks and live-locks. Again, channels are blocking data structures. > If you end up doing such synchronization in Go, then Go's M:N multi-threading ends up doing more harm than good, because if you need such synchronization, you also need to fine tune your thread-pools and at this point 1:1 would be better. I don't know where you're getting that. AFAIK, Go's mutexes don't block the underlying thread; only the goroutine. The question of which concurrency mechanism should be used is a difficult one (and in general, more than one is necessary; even Erlang has shared, synchronized mutable state with its ETS tables), but you are very misinformed about how concurrency constructs are built and about their performance behavior.

sagichmal 11y ago

> In general, if "orchestrating concurrency" involves guarding access to shared mutable state, then you can't orchestrate it at the callers site. Shared mutable state is generally behind an API boundary. I'm talking about the exported method set of that API. That is, func (f *Foo) Update(i int) (int, error) // yes func (f *Foo) Update(i int) (<-chan int, error) // no func (f *Foo) Update(i int, result chan<- int) error // no

jerf 11y ago

"In general, if "orchestrating concurrency" involves guarding access to shared mutable state, then you can't orchestrate it at the callers site." What this generally means in Go is that it is an antipattern for your library to provide something like "a method that makes an HTTP request in a goroutine". In Go, you should simply provide code that "makes an HTTP request", and it's up to the user to decide whether they want to run that in a goroutine. The rest of what you're talking about is a completely different issue. Channels are smelly in an API. IIRC in the entire standard library there's less than 10 functions/methods that return a channel. But the use case does occasionally arise.

pmarreck 11y ago

Reading this... only makes me gladder that I'm pursuing work in the Erlang/Elixir space, where messaging "just works" and concurrency "just works" and immutability "just works" (and new processes take a microsecond to spin up) and tearing anything down is basically a nonissue as failure is embraced and logged at every turn and cascading teardowns simply happen automatically depending on how the processes are linked and all this turns out to be a really amazing system of paradigms when designing apps to work in the real world

svanderbleek 11y ago
It "just works" in Go too, minus immutability, but congrats on your technology decision. You don't get type checking but c'est la vie.
- raould42 11y ago
  Dialyzer
- pmarreck 11y ago
  Restricting input based on type hierarchies can reduce a certain class of bugs, yes, but careful use of guards as well as typespecs and unit test coverage (which you should have, anyway) can accomplish much of what type restrictions can
  - pmarreck 11y ago
    Was something I said factually wrong? User im_down_w_otp put up an example of what I'm talking about (minus the unit testing) so what gives?
    - sagichmal 11y ago
      I suspect dismissing static typing whole cloth with "unit tests and certain guards can give you most of the benefits" comes off badly to some people.

pron 11y ago

> you end up paying the price of LOCK XCHG or whatever mutexes translate to But channels use locks internally. The choice of channels vs. mutexes is one of design, not implementation. Also, mutexes are blocking; LOCK XCHG isn't. Sure, mutexes also use LOCK XCHG (but so do channels, and nearly all concurrent data sctructures), but they also block (as do channels). > your only choice of handling it is by means of mutexes, which has really terrible performance characteristics That's just not true. There is a way to translate any locking algorithm to a non-blocing one (in fact, wait-free, which is the "strongest" non-blocking guarantee), yet only only a handful of wait-free algorithms are used in practice. Why? Because it's hard to make them more efficient than locks in the general case. > not to mention that you'll have problems if you want (soft) real-time behavior, because now you can end up with both dead-locks and live-locks. Again, channels are blocking data structures. > If you end up doing such synchronization in Go, then Go's M:N multi-threading ends up doing more harm than good, because if you need such synchronization, you also need to fine tune your thread-pools and at this point 1:1 would be better. I don't know where you're getting that. AFAIK, Go's mutexes don't block the underlying thread; only the goroutine. The question of which concurrency mechanism should be used is a difficult one (and in general, more than one is necessary; even Erlang has shared, synchronized mutable state with its ETS tables), but you are very misinformed about how concurrency constructs are built and about their performance behavior.