3 ms·

goroutines are just m:n threads with message passing queues. please correct me if I'm missing some obvious detail that makes them unique in any way. in that c

• fidotron 3y ago
  The message passing queues following CSP semantics regarding synchronization is the big deal. If you only ever use buffered queues then you're missing half of the story. Occam is probably the most important totally forgotten thread of programming language development.
  • dboreham 3y ago
    Not really forgotten because Phil Winterbottom was familiar with it and went on to work on Go.
  • packetlost 3y ago
    Can you elaborate or provide sources on what you mean by "CSP semantics" here?
    • Jtsummers 3y ago
      They mean communicating sequential processes. Here's a book on the topic by C.A.R. Hoare: http://www.usingcsp.com/ http://www.usingcsp.com/
  • dekhn 3y ago
    The message passing queues in CSP aren't any different from buffered queues, right? I've been asking people this for some time and never have heard a convincing "no, and here's why they are different". There are CSP implementations in C++ and Python and their implementations seem to be buffered queues that block if there are no readers.
    • fidotron 3y ago
      CSP doesn’t have queues, only blocking channels. Some implementations may provide them on top, as go does. I would suggest reviewing the Hoare book on CSP, it’s very readable.
      • dekhn 3y ago
        I have read it (along with lots Russ Cox wrote about CSP and Go, such as https://swtch.com/~rsc/thread/ https://swtch.com/~rsc/thread/) and McIlroy (https://www.cs.dartmouth.edu/~doug/sieve/sieve.pdf https://www.cs.dartmouth.edu/~doug/sieve/sieve.pdf) . From what I can tell, CSP channels are semantically equivalent to a particular subtype of queues and use the same OS primitives. I'm not an OS expert but I've worked with distributed systems and threads for decades, and nobody I've asked has disagreed that Go's implementation of CSP is an M:N thread model with (blocking) queues.
      • deleted 3y ago
        [deleted]
      • Jtsummers 3y ago
        CSP doesn't need queues (buffered channels) as a primitive because you can create them. Here's a Queue in CSP from the CSP book I linked elsewhere: BUFFER = P⟨⟩ where P⟨⟩ = left?x → P⟨x⟩ P⟨x⟩⌢s = (left?y → P⟨x⟩⌢s⌢⟨y⟩ | right!x → Ps ) If data can be read from left it is and it's enqueued, if data can be written to right (because a reader is available) it dequeues it. But neither the read nor the write block the entire process. Go's buffered channels act in the same way so end up offering a shortcut versus pure CSP. There are other ways that Go deviates from CSP (shared memory, among others) but with channels it's not deviating, just optimizing an existing behavior.
• hashmash 3y ago
  What are the expert-programmer aspects you're thinking of?
  • jayd16 3y ago
    Probably that it doesn't mix well when you need to pin work to a native/specific thread. This can matter when that thread is used across runtimes or with single threaded concurrency designs like GUI threads for qt or gtk.
    • arccy 3y ago
      https://pkg.go.dev/runtime#LockOSThread https://pkg.go.dev/runtime#LockOSThread
  • dekhn 3y ago
    shared memory, https://go.dev/blog/codelab-share https://go.dev/blog/codelab-share although of course go added thread primitives. it's just that the language creators explicitly wanted to steer people away from the "subtle" details of multithreading.
• klooney 3y ago
  "Just m:n threads" is serious downplaying, most operating systems and languages that had that retired it eventually. The go implementation of goroutines is really really good.
  • felixgallo 3y ago
    m:n threads is the basis of every non-embedded operating system, so I wouldn't say that model's been retired or is even under plausible threat. Loom and such are refitting Java with it, for example. The only problem with m:n is that it requires a thread-aware runtime to fully implement the abstraction.
    • klooney 3y ago
      Linux (or glibc I guess) and Solaris both rolled back their m:n threading implementations over a decade ago. It's a really hard problem, and you kind of need to own the whole stack. The only other system with a comparable implementation is Erlang.
      • felixgallo 3y ago
        linux is m:n today and always has been, so I guess I don't know what you're talking about.
  • dekhn 3y ago
    I'm not saying their implementation isn't good, it is good. And probably message-passing is the right concurrency primitive to expose at a language level for people who want to take advantage of multi-core machines running concurrent code. I'm not downplaying anything; I'm expressing merely that with a title like "Goroutines: The concurrency model we all..." makes it sounds like go invented something new, but afaict, they're just a well-engineered implemention of already well-understood concurrency principles.
    • zwieback 3y ago
      100%, as I was reading the article I was remembering the different types of concurrency mechanisms I've used and I think it's helpful to have something like goroutines as a first-class language feature but it's not new in any way. When it comes to concurrency I think it's good to go through the curriculum: - main loop + interrupt handler - processes (fork) - multi-core concerns - threads (now is a good time to learn about mutex, semaphore, critical section) - lightweight threads (fiber, coroutines, green threads): learn how they are different - async/await statemachines written by the compiler Ok, now we have the background info to make wise choices and call things by their generalized names to avoid holy wars.
• titzer 3y ago
  IMO thread pools are anti-pattern in several ways. They don't globally load-balance properly and can be exhausted. I think the Go concurrency model with many lightweight threads is really nice. The only issue I have is that race conditions in Go code can violate the type system and cause horror-crashes. I've never used Go in production but have talked to outfits who do and have confirmed that race conditions are a serious issue.
  • procone 3y ago
    Sorry, race conditions? You really have to go out of your way to implement those.
  • imiric 3y ago
    One of my biggest pet peeves are race conditions in tests. They typically only show up when run in CI, and are impossible to reproduce locally. I have a hunch this is because of resource-starved VMs running on over provisioned hardware, which messes with time in unpredictable ways, making tests flaky. Sure, tests shouldn't rely on time, and these failures sometimes point that out for us to fix, but it's very annoying to troubleshoot when you can't reproduce it locally. I've tried everything from running in resource-limited containers to similarly spec'd VMs, and wasn't able to. We could use a time-faking library, but sometimes this is unavoidable if some dependency, whether from a 3rd party or in stdlib, depends on reliable timing. I'm wondering how other Go devs approach this.
    • noisem4ker 3y ago
      Have you tried running your tests with the race detector enabled? https://go.dev/doc/articles/race_detector https://go.dev/doc/articles/race_detector https://go.dev/blog/race-detector https://go.dev/blog/race-detector It only logs when a race condition happens, and it may have a non-negligible impact on performance, but it should print useful info to aid in debugging.
      • imiric 3y ago
        Oh, of course, I always run with -race enabled. But this is what I mean, the races are only detected in CI, not locally. Sometimes the stack traces point to a simple fix in the test itself, but often the issue is much more difficult to troubleshoot, especially since it can't be reproduced locally. It's also frustrating since this behavior is intermittent, and most of the time tests run fine in CI, but there are "troublesome" days when the issues happen much more frequently, which is why I think this is so dependent on the CI infrastructure (GitHub Actions FWIW). A solution would be to run CI on self-managed nodes that behave more reliably, but that has maintenance costs that we don't want to deal with.