5 ms·
Structured Concurrency Definition (2019)
- gavinhoward 4y agoAuthor here. It's interesting to see this posted after three years. In fact, there should be "(2019)" in the Hacker News title. After looking at the definition, I think it holds up well. I would probably tweak it, but I was three years less experienced back then. It's weird to think that I posted that just two months before lockdown. It seems like I posted it long before that...
- mananaysiempre 4y agoSmall quibble with the article: It refers to Trio nurseries as “scopes”. I thought about them like that for a while, but don’t think that’s not exactly the right view now. As a subroutine, you can’t (usually) reach up into your parent static or dynamic scopes and cause new objects to be created there and outlive your own dynamic extent. (Tcl exempted.) Because nurseries are first-class objects, though, you’re free to create a root nursery at startup and stick it in a global variable, after which everyone is free to create as many Go-style unaccountable threads as they want to. It’s just that you don’t have to. Yet passing worker threads their originating nursery so that they can spawn more peer workers is a moderately useful pattern. TL;DR: Firsr-class nurseries are not quite scopes because they don’t enforce strict lifetime nesting between a spawned thread and the thread that caused it to be spawned.
- gavinhoward 4y agoOne of the tweaks I would use is to use the word "lifetimes" instead of scopes. I think this would since the problem you see because each object, even a nursery, has a lifetime, and threads are tied to those lifetimes.
- MarkMarine 4y agoMore than just I read your article about C and memory safety today and went clicking around in the archive. Good work. When I read the Go statement considered harmful post, I felt the same thing. This just makes sense, everything should be like this. A number of these ideas have been designed into Kotlin’s concurrency library, what are your thoughts about it?
- gavinhoward 4y agoThank you! I haven't seen Kotlin's library, so I can't comment. Do you have a link? I wasn't able to pull up anything good with a search.
- garrybelka 4y agohttps://elizarov.medium.com/structured-concurrency-722d765aa952 https://elizarov.medium.com/structured-concurrency-722d765aa... https://elizarov.medium.com/coroutine-context-and-scope-c8b255d59055 https://elizarov.medium.com/coroutine-context-and-scope-c8b2... https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/launch.html https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-corout...
- gavinhoward 4y agoThank you! To answer the GGP, this looks like true structured concurrency to me. Chalk one up for Kotlin!
- manmal 4y agoDo you think that what Swift implemented can be called structured concurrency by your definition?
- gavinhoward 4y agoYes. Swift beat me to it.
- rerdavies 4y agoTo the author: The disclaimer is needlessly @ssh0lish.
- gavinhoward 4y agoOn the post? Not really. I've had people go after me for such things.
- rerdavies 4y agoNow you have people like me after you for the disclaimer. There is another solution.
- gavinhoward 4y agoSelf-censoring isn't an option, but if you have another, I'd be happy to listen.
- strawpeople 4y agoWhat’s wrong with his disclaimer?
- rerdavies 4y agoKinda like std::jthread. Definitely a good thing. But does that really justify the label "structured concurrency"? I was rather hoping to see something more, especially since languages like Rust, Go, JavaScript (and even Python) go much further.
- jerf 4y agoGo goes "beyond" structured concurrency in the same sense that BASIC goes "beyond" structured programming because it has a "goto" statement. That is, it goes "beyond" it by being incapable of guaranteeing it at all. The entire point of structured concurrency or structured programming is to build limitations into the system, then build other guarantees on those limitations. If you don't have the limitations, you don't have the things you can build on it. Systems lacking those limitations do not "go beyond" the limitations, they lack the ability to build the guarantees. I have been influenced in my Go programming by this essay. A lot of my concurrency is actually "structured concurrency" if you look at it. But you have to look at it to see, because there is no way for me to guarantee it structurally. No matter how much I use the ideas, no matter how much I were to write it into an API, I can never prevent anything from blasting a "go" statement whereever they want, whenever they want, and violating the limitations, and thus breaking the guarantees. This is not the end of the world some programmers treat it as; no language can do everything and enforce everything and be perfect without the programmer having to bring some discipline to the party. But neither is it no big deal at all and something you can just wave away with a vague statement about how some programmers just need to be better programmers or something. These things matter.
- mananaysiempre 4y agoI like structured concurrency (in its Trio embodiment). I’d love to go all in on structured concurrency... except I see no way to marry it (or async/await-style syntax) to promise pipelining, which seems to be the only sane avenue to avoiding the roundtrip tax on fine-grained remote APIs[1]. Is anyone aware of any attempts in this area? [1] https://capnproto.org/rpc.html https://capnproto.org/rpc.html
- gavinhoward 4y agoI could implement something like this. First, instead of calling a function, just add that function to a list of functions. Then when you're done, spawn a thread and pass it the needed data and the list of functions. The thread will work its way through the chain. This technique is possible because "closures are just a poor man's object." (I think I saw that on the c2 wiki.)
- alwaysbeconsing 4y ago> "closures are just a poor man's object." Yes; the inverse is also true :) The "koan" is on c2 here: http://wiki.c2.com/?ClosuresAndObjectsAreEquivalent http://wiki.c2.com/?ClosuresAndObjectsAreEquivalent (with a link to its original source).
- gavinhoward 4y agoYes, that's the one!
- mananaysiempre 4y ago(Musings not that relevant to asynchronous ocap messaging, where the problems don’t really depend on whether you’re passing around objects or closures.) One thing I don’t think I’ve seen mentioned on those pages is the distinction between a “closed” object you can only send messages to and an “open” one you can inherit from / delegate to / mix in / COM-aggregate: the former is indeed a closure, that is some code and an environment, that you feed the message selector and some arguments; the latter is also some code and an environment, but you feed it the resend target, the message selector, and some arguments. So depending on what exactly you’re calling an object, an object is a closure... in one of two non-equivalent ways. Maybe not really a closure then? See also for not-quite-identical points of view: Piumarta’s note on prototypes in his COLA system[1]; Chisnall’s description of the (IIUC now-dead) new Objective-C runtime for Étoilé[2]. [1] https://www.piumarta.com/pepsi/prototypes.html https://www.piumarta.com/pepsi/prototypes.html [2] https://www.jot.fm/issues/issue_2009_01/article4.pdf https://www.jot.fm/issues/issue_2009_01/article4.pdf
- convolvatron 4y agois similar to nested dag? not exactly because it can contain loops, but children never outlive their parent?
- gavinhoward 4y agoBasically, though I've also extended the idea since.
- kzuberi 4y agoA useful introduction to the ideas is in Nathaniel's PyCon 2018 talk on trio [1], where he live-codes a solution to a concurrency-spaghetti problem. [1] https://www.youtube.com/watch?v=oLkfnc_UMcE https://www.youtube.com/watch?v=oLkfnc_UMcE
- samsquire 4y agoThank you for this. If you found this interesting then you shall like the following pages: The Vale language blog post on seamless structured concurrency https://verdagon.dev/blog/seamless-fearless-structured-concurrency https://verdagon.dev/blog/seamless-fearless-structured-concu... The rust book on concurrency https://doc.rust-lang.org/book/ch16-00-concurrency.html https://doc.rust-lang.org/book/ch16-00-concurrency.html I've been working on implementing Java async/await state machine with switch statements and a scheduling loop. If the user doesn't await the async task handle, then the task's returnvalue is never handled. This is similar to the Go problem with the go statement. https://github.com/samsquire/multiversion-concurrency-control/blob/main/src/main/java/main/MultiAwait.java https://github.com/samsquire/multiversion-concurrency-contro... If your async call returns a handle and the handle is linked to a scope then in theory the async/await could be used as primitive to structured concurrency. I journal about asynchrony and parallelism everyday in ideas4 (see my HN profile)
- TYMorningCoffee 4y agoStructured concurrency is in Java 19 early access [1]. Try it out! They reuse the try with resources to implement it. [1] https://openjdk.org/jeps/428 https://openjdk.org/jeps/428
- marssaxman 4y agoThanks! I hadn't encountered this idea before, and the essay kicked off several hours of reading and learning. This will definitely influence my approach to all concurrency problems in the future.