3 ms·

Could you expand on what you think is hard about the concurrency in Go? (And perhaps give an example of a language that makes it easy/easier?)

by bborud 7y ago

gen220 7y ago
Not the OP, but if you’ve ever had to write 100% fault tolerant concurrent Go code, you find yourself having to add a lot of monitor-recover-restart “boilerplate” that ends up being quite complex, depending on the task at hand. It helps if you follow certain principles like making your concurrent tasks idempotent etc., but Go doesn’t force you to write things that way, so it takes a bit of remembering each time. If the task is relatively simple, you end up writing a lot of concurrency infra for a little bit of concurrency, which can become frustrating if you’re doing it for the Nth time. It sounds like a problem that’s ripe for abstraction, but it turns out that it’s quite hard to build abstractions that get all the trade offs just right for each case. There’s some worker pool style libraries out there, but if you care about performance you can usually eke more out by writing a custom solution. As for alternatives, I have heard that erlang has a better story for this, because of the inherent restrictions of the language, but I haven’t used it much. In my experience, Go gives you enough rope to make this task “merely annoying” (rather than impossible or astoundingly challenging) without giving you enough to truly hang yourself with. You can get really tangled up though :)
masklinn 7y ago
> Could you expand on what you think is hard about the concurrency in Go? Go's concurrency is cheap[0] multithreading (shared-memory concurrency). It has a tagline that you should not share memory ("Do not communicate by sharing memory; instead, share memory by communicating"), however nothing prevents sharing memory (so it will eventually leak in) and channels are pretty slow[1] so it's not uncommon to go back to regular shared-memory primitives (with no safety net). You've got no idea what your application state really is (even less so once things start falling over), and there are no built-in tools for reliability e.g. you can't get notified that a goroutine has keeled over or anything unless you implement that yourself. > And perhaps give an example of a language that makes it easy/easier? Erlang / elixir was built specifically with that in mind. It uses lightweight shared-nothing concurrency (non-OS processes) and the system builds-in ways for processes to have oversight over others (e.g. monitors & supervisors) or get killed alongside them (links) and provides behaviours which bundle known patterns, etc… [0] as in "not expensive", as in uses quite a bit less memory & especially kernel resources than OS threads or processes[1] [1] though this point should not be too overstated, people commonly misuse "the C stack is XMB" with "creating a thread consumes XMB of physical memory", a C stack is mostly a regular allocation and thus usually just VSS until actually exercised [2] and not necessarily a huge gain in bug-freedom: https://songlh.github.io/paper/go-study.pdf https://songlh.github.io/paper/go-study.pdf
- bborud 7y ago
  Thanks for taking the time to expand on that. I haven’t really benchmarked channels since the bottlenecks in code I write tend to be I/O, but I’ll play around a bit to understand that better.

Could you expand on what you think is hard about the concurrency in Go? (And perhaps give an example of a language that makes it easy/easier?)

by bborud 7y ago

Could you expand on what you think is hard about the concurrency in Go? (And perhaps give an example of a language that makes it easy/easier?)

gen220 7y ago

Not the OP, but if you’ve ever had to write 100% fault tolerant concurrent Go code, you find yourself having to add a lot of monitor-recover-restart “boilerplate” that ends up being quite complex, depending on the task at hand. It helps if you follow certain principles like making your concurrent tasks idempotent etc., but Go doesn’t force you to write things that way, so it takes a bit of remembering each time. If the task is relatively simple, you end up writing a lot of concurrency infra for a little bit of concurrency, which can become frustrating if you’re doing it for the Nth time. It sounds like a problem that’s ripe for abstraction, but it turns out that it’s quite hard to build abstractions that get all the trade offs just right for each case. There’s some worker pool style libraries out there, but if you care about performance you can usually eke more out by writing a custom solution. As for alternatives, I have heard that erlang has a better story for this, because of the inherent restrictions of the language, but I haven’t used it much. In my experience, Go gives you enough rope to make this task “merely annoying” (rather than impossible or astoundingly challenging) without giving you enough to truly hang yourself with. You can get really tangled up though :)

masklinn 7y ago

> Could you expand on what you think is hard about the concurrency in Go? Go's concurrency is cheap[0] multithreading (shared-memory concurrency). It has a tagline that you should not share memory ("Do not communicate by sharing memory; instead, share memory by communicating"), however nothing prevents sharing memory (so it will eventually leak in) and channels are pretty slow[1] so it's not uncommon to go back to regular shared-memory primitives (with no safety net). You've got no idea what your application state really is (even less so once things start falling over), and there are no built-in tools for reliability e.g. you can't get notified that a goroutine has keeled over or anything unless you implement that yourself. > And perhaps give an example of a language that makes it easy/easier? Erlang / elixir was built specifically with that in mind. It uses lightweight shared-nothing concurrency (non-OS processes) and the system builds-in ways for processes to have oversight over others (e.g. monitors & supervisors) or get killed alongside them (links) and provides behaviours which bundle known patterns, etc… [0] as in "not expensive", as in uses quite a bit less memory & especially kernel resources than OS threads or processes[1] [1] though this point should not be too overstated, people commonly misuse "the C stack is XMB" with "creating a thread consumes XMB of physical memory", a C stack is mostly a regular allocation and thus usually just VSS until actually exercised [2] and not necessarily a huge gain in bug-freedom: https://songlh.github.io/paper/go-study.pdf https://songlh.github.io/paper/go-study.pdf

bborud 7y ago
Thanks for taking the time to expand on that. I haven’t really benchmarked channels since the bottlenecks in code I write tend to be I/O, but I’ll play around a bit to understand that better.