6 ms·

In other words futures are not as composable as one would hope. John Reppy's CML seems like a much better toolbox which gets composability without pretension.

In other words futures are not as composable as one would hope. John Reppy's CML seems like a much better toolbox which gets composability without pretension. Vesa Karvonen (whom I understand has worked on the MLTon compiler) has offered an excellent delivery in Hopac for C# and F# complete with a slew of combinators: https://github.com/Hopac/Hopac https://github.com/Hopac/Hopac I'm not aware of anyone offering an alternative superior to an informal CSP yet which seems to be the reason why Go and Clojure have picked it as well for their concurrency model. See David Nolen's excellent blog posts on the matter: http://swannodette.github.io/2013/07/12/communicating-sequential-processes http://swannodette.github.io/2013/07/12/communicating-sequen... http://swannodette.github.io/2013/08/17/comparative http://swannodette.github.io/2013/08/17/comparative

pcwalton 10y ago
I think Rust's approach here makes a lot more sense for the language than Go/CML style M:N would. You can recover most of the M:N ergonomics over time via async/await style syntax. But if your foundation is built on a (comparatively) slow "pthreads in userland" threading model, then you hit a performance ceiling that you can never break through. For a language like Rust, it makes sense to begin in the optimum place and then layer zero-cost abstractions on top to achieve the right ergonomics.
- jholman 10y ago
  I think there's little or no evidence that "you can recover most of the M:N ergonomics over time via async/await style syntax", despite a decade or so of attempts. I think there's an underlying semantic concern that seems unsugarable. Munificent's http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y... expresses the problem eloquently. None of that means you're wrong about async making "a lot more sense for the language", though.
  - pcwalton 10y ago
    > Munificent's http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y.. http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y.... expresses the problem eloquently. I have several issues with that blog post. 1. The performance concerns are glossed over. The conclusion is "threads are superior", but that ignores the reason why we want async I/O in the first place: performance. 2. "What if the new place you want to call it is blue? You’ll have to turn it red." is false. There's a way to convert blocking code to async-friendly code: use a thread pool. This is in fact what cgo does internally—see the next point. 3. You always have red functions and blue functions if you have an FFI. But this isn't a big deal because synchronous functions are easily convertible to asynchronous functions (via a thread pool) and asynchronous functions are easily convertible into synchronous functions (via just blocking). So the supposedly big semantic problem just boils down into a question of making sure you remember to switch into the other mode when necessary (which you can always easily do without restructuring your program). This is something that the language can do for you automatically—proof is that Go does it!—and I would like to see type system or other static approaches to doing this in Rust.
  - gpderetta 10y ago
    This is currently hotly debated in the C++ committee. Some people want shallow C#, python style generators, while other want proper stackful coroutines a-la Lua (full disclosure: I'm on this group). A third group is trying to mediate and trying to come up with an hybrid stackful model that can be optimized as well the async/await model at least in some cases (I.e. full cps transform and fallback to a cactus stack when invoking non transformable functions).
    - wahern 10y ago
      I've been writing async I/O networking software for about 15 years now. Early on most of that was in C, now it's split about 50/50 between C and Lua. Most of my C I/O code is still in C because I prefer my libraries to be reuseable outside of Lua or any particular event loop, and they often are. Lua's coroutines are usually higher up the stack, juggling more abstract state; and I use them for more than asynchronous I/O or asynchronous tasks. The thing about async/await is that in a language like C, I can already accomplish much of that with tricks like Duff's Device and macros. It has its limitations, but IME they're not much more onerous than the limitations of async/await, especially in the context of a language lacking GC. I have to manually keep state off the stack (or otherwise copy/restore it), but you do that anyhow when you don't have lexical closures and GC, and often even when you do. The beautiful thing about coroutines in Lua is that it's based on a threading model, but not one bound to the C stack or a kernel thread, which are completely orthogonal concerns left to the application to deal with or not deal with. And it does this while preserving functions as first-class objects. Neither callers nor callees need to know anything about coroutines. That kind of composability makes coroutines useful and convenient for many more things than simulating green threading or managing CPU parallelism. Among other things, it means I can mix-and-match functional and imperative styles according to the problem, and not whether it will be convenient to then make use of coroutines. It means that I have a single, natural call stack--not an implicit stack and an explicit stack. async/await and futures unify your data stack, but you're still manually managing the call stack through syntactic devices or otherwise formalized calling conventions. However heavily sugared, it will hinder the design of your software no less than if you had to manually manage the data stack, too. Coroutines that aren't stackful aren't nearly as powerful in terms of problem solving. Without them being stackful, it's a horribly leaky abstraction for non-trivial uses. Most people would agree that the C preprocessor is a mess, and that functions as first-class objects are powerful. So modern languages strive to create templating systems that allow you to construct _real_ functions that are indistinguishable from any another function. But then they introduce monstrosities like futures or async/await, that beautiful symmetry is broken. It's like bringing back C's macro preprocessor--now you have regular functions and these weird things with different syntactic and control follow semantics, whether you wanted it or not. The decision is no longer yours, which means you're bending to the language's deficiencies. Why even bother with such half-baked solutions? In almost every case it's utterly transparent that these solutions exist for the benefit of the compiler and runtime author, usually because of intentional or unintentional technical debt--a direct or indirect dependency on the C or kernel stack. For C++ it's understandably a difficult dilemma, but for every other language it's a total cop-out. Then these solutions are sold to the public by prettifying the implementations with fancy terminology and beguiling examples showing how they can be used to implement async I/O or parallel CPU jobs. But few, if any, language features are so narrowly tailored to such specific use cases. Why? Because languages are supposed to provide simple building blocks that compose as seamlessly as possible at a much higher level of abstraction than, e.g., a slightly nicer way to implement HTTP long-polling servers. Such contrivances are as removed from the basic simplicity of the function as C's macros are from first-class functions. In both cases you can implement solutions for a certain subset of problems that superficially look convenient and nice; but in the real world the limitations become swiftly apparent, and you realize a lot of effort was spent in design and implementation for little real-world gain. With Lua's coroutines, I can implement a futures pattern easily when it's appropriate, and it will be more powerful because the futures themselves can make use of coroutines both internally and externally. But in my use of coroutines in Lua futures are rarely the most natural design pattern. Sometime you want a full-blown CPS solution, sometimes you simply want to be able to arbitrarily swap producer/consumer control flow, for example in a lexer. Often you want a mixture of all of these. Coroutines--stackful coroutines--provide all that and more, seamlessly. Futures only look nice and elegant in contrast to event loop oriented, callback-style programming. But that's a really, really low bar. Please aim higher, people!
- soulbadguy 10y ago
  > You can recover most of the M:N ergonomics over time via async/await style syntax. While i agree with the tradeoff made by rust (although i think the approach used by C++ coroutine is better), i don't think that having async/await syntax give you "most" of the ergonomics of the Go M:N model . The main advantage of the go model is that both asynchronous and synchronous operations are identical, with async/await you still need to model the async operation and the sync operation with different types. Not having to decide(and design) upfront which part of your computation is async and which is not is what makes go so attractives
  - pcwalton 10y ago
    > although i think the approach used by C++ coroutine is better How? > The main advantage of the go model is that both asynchronous and synchronous operations are identical, with async/await you still need to model the async operation and the sync operation with different types. It's more like "everything is synchronous" in the Go model. Semantically, Go doesn't have async I/O at all. It has a userspace M:N implementation of synchronous I/O. You can get the same effect in any other language by just not using async I/O. > Not having to decide(and design) upfront which part of your computation is async and which is not is what makes go so attractives That doesn't make sense to me. If async I/O is important in your app, why not just make your entire app use async I/O?
    - skybrian 10y ago
      Defaults matter. If some people use async I/O and others don't then you get a mess when they want to share reusable libraries. It's similar to the mess you get when there is more than one string type. I think the "what color is your function" problem could be mostly solved by making async/await the default function type - that is, most callback functions should be allowed to suspend execution by waiting on a Future. Then you could have special-purpose "atomic" functions that are occasionally useful for pure functional code. (Unfortunately, the default has to be the opposite in browser-based languages due to performance concerns.)
      - 10y ago
namelezz 10y ago
> I'm not aware of anyone offering an alternative superior to an informal CSP yet which seems to be the reason why Go and Clojure have picked it as well for their concurrency model. What about Quasar [0] on the JVM? 0 - http://docs.paralleluniverse.co/quasar/ http://docs.paralleluniverse.co/quasar/
- davidgrenier 10y ago
  But Quasar does seem to offer go-like channels? I'm unclear what kind of combinators are provided but those could be implemented.
  - Matthias247 10y ago
    It does offer channels, see the linked Docu. For combinators I don't think those are really needed in the CSP environments because you can do most transformations with normal function calls and normal control flow contructs (loops, if, ...)

pcwalton 10y ago

I think Rust's approach here makes a lot more sense for the language than Go/CML style M:N would. You can recover most of the M:N ergonomics over time via async/await style syntax. But if your foundation is built on a (comparatively) slow "pthreads in userland" threading model, then you hit a performance ceiling that you can never break through. For a language like Rust, it makes sense to begin in the optimum place and then layer zero-cost abstractions on top to achieve the right ergonomics.

jholman 10y ago
I think there's little or no evidence that "you can recover most of the M:N ergonomics over time via async/await style syntax", despite a decade or so of attempts. I think there's an underlying semantic concern that seems unsugarable. Munificent's http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y... expresses the problem eloquently. None of that means you're wrong about async making "a lot more sense for the language", though.
- pcwalton 10y ago
  > Munificent's http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y.. http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y.... expresses the problem eloquently. I have several issues with that blog post. 1. The performance concerns are glossed over. The conclusion is "threads are superior", but that ignores the reason why we want async I/O in the first place: performance. 2. "What if the new place you want to call it is blue? You’ll have to turn it red." is false. There's a way to convert blocking code to async-friendly code: use a thread pool. This is in fact what cgo does internally—see the next point. 3. You always have red functions and blue functions if you have an FFI. But this isn't a big deal because synchronous functions are easily convertible to asynchronous functions (via a thread pool) and asynchronous functions are easily convertible into synchronous functions (via just blocking). So the supposedly big semantic problem just boils down into a question of making sure you remember to switch into the other mode when necessary (which you can always easily do without restructuring your program). This is something that the language can do for you automatically—proof is that Go does it!—and I would like to see type system or other static approaches to doing this in Rust.
- gpderetta 10y ago
  This is currently hotly debated in the C++ committee. Some people want shallow C#, python style generators, while other want proper stackful coroutines a-la Lua (full disclosure: I'm on this group). A third group is trying to mediate and trying to come up with an hybrid stackful model that can be optimized as well the async/await model at least in some cases (I.e. full cps transform and fallback to a cactus stack when invoking non transformable functions).
  - wahern 10y ago
    I've been writing async I/O networking software for about 15 years now. Early on most of that was in C, now it's split about 50/50 between C and Lua. Most of my C I/O code is still in C because I prefer my libraries to be reuseable outside of Lua or any particular event loop, and they often are. Lua's coroutines are usually higher up the stack, juggling more abstract state; and I use them for more than asynchronous I/O or asynchronous tasks. The thing about async/await is that in a language like C, I can already accomplish much of that with tricks like Duff's Device and macros. It has its limitations, but IME they're not much more onerous than the limitations of async/await, especially in the context of a language lacking GC. I have to manually keep state off the stack (or otherwise copy/restore it), but you do that anyhow when you don't have lexical closures and GC, and often even when you do. The beautiful thing about coroutines in Lua is that it's based on a threading model, but not one bound to the C stack or a kernel thread, which are completely orthogonal concerns left to the application to deal with or not deal with. And it does this while preserving functions as first-class objects. Neither callers nor callees need to know anything about coroutines. That kind of composability makes coroutines useful and convenient for many more things than simulating green threading or managing CPU parallelism. Among other things, it means I can mix-and-match functional and imperative styles according to the problem, and not whether it will be convenient to then make use of coroutines. It means that I have a single, natural call stack--not an implicit stack and an explicit stack. async/await and futures unify your data stack, but you're still manually managing the call stack through syntactic devices or otherwise formalized calling conventions. However heavily sugared, it will hinder the design of your software no less than if you had to manually manage the data stack, too. Coroutines that aren't stackful aren't nearly as powerful in terms of problem solving. Without them being stackful, it's a horribly leaky abstraction for non-trivial uses. Most people would agree that the C preprocessor is a mess, and that functions as first-class objects are powerful. So modern languages strive to create templating systems that allow you to construct _real_ functions that are indistinguishable from any another function. But then they introduce monstrosities like futures or async/await, that beautiful symmetry is broken. It's like bringing back C's macro preprocessor--now you have regular functions and these weird things with different syntactic and control follow semantics, whether you wanted it or not. The decision is no longer yours, which means you're bending to the language's deficiencies. Why even bother with such half-baked solutions? In almost every case it's utterly transparent that these solutions exist for the benefit of the compiler and runtime author, usually because of intentional or unintentional technical debt--a direct or indirect dependency on the C or kernel stack. For C++ it's understandably a difficult dilemma, but for every other language it's a total cop-out. Then these solutions are sold to the public by prettifying the implementations with fancy terminology and beguiling examples showing how they can be used to implement async I/O or parallel CPU jobs. But few, if any, language features are so narrowly tailored to such specific use cases. Why? Because languages are supposed to provide simple building blocks that compose as seamlessly as possible at a much higher level of abstraction than, e.g., a slightly nicer way to implement HTTP long-polling servers. Such contrivances are as removed from the basic simplicity of the function as C's macros are from first-class functions. In both cases you can implement solutions for a certain subset of problems that superficially look convenient and nice; but in the real world the limitations become swiftly apparent, and you realize a lot of effort was spent in design and implementation for little real-world gain. With Lua's coroutines, I can implement a futures pattern easily when it's appropriate, and it will be more powerful because the futures themselves can make use of coroutines both internally and externally. But in my use of coroutines in Lua futures are rarely the most natural design pattern. Sometime you want a full-blown CPS solution, sometimes you simply want to be able to arbitrarily swap producer/consumer control flow, for example in a lexer. Often you want a mixture of all of these. Coroutines--stackful coroutines--provide all that and more, seamlessly. Futures only look nice and elegant in contrast to event loop oriented, callback-style programming. But that's a really, really low bar. Please aim higher, people!
soulbadguy 10y ago
> You can recover most of the M:N ergonomics over time via async/await style syntax. While i agree with the tradeoff made by rust (although i think the approach used by C++ coroutine is better), i don't think that having async/await syntax give you "most" of the ergonomics of the Go M:N model . The main advantage of the go model is that both asynchronous and synchronous operations are identical, with async/await you still need to model the async operation and the sync operation with different types. Not having to decide(and design) upfront which part of your computation is async and which is not is what makes go so attractives
- pcwalton 10y ago
  > although i think the approach used by C++ coroutine is better How? > The main advantage of the go model is that both asynchronous and synchronous operations are identical, with async/await you still need to model the async operation and the sync operation with different types. It's more like "everything is synchronous" in the Go model. Semantically, Go doesn't have async I/O at all. It has a userspace M:N implementation of synchronous I/O. You can get the same effect in any other language by just not using async I/O. > Not having to decide(and design) upfront which part of your computation is async and which is not is what makes go so attractives That doesn't make sense to me. If async I/O is important in your app, why not just make your entire app use async I/O?
  - skybrian 10y ago
    Defaults matter. If some people use async I/O and others don't then you get a mess when they want to share reusable libraries. It's similar to the mess you get when there is more than one string type. I think the "what color is your function" problem could be mostly solved by making async/await the default function type - that is, most callback functions should be allowed to suspend execution by waiting on a Future. Then you could have special-purpose "atomic" functions that are occasionally useful for pure functional code. (Unfortunately, the default has to be the opposite in browser-based languages due to performance concerns.)
    - 10y ago

namelezz 10y ago

> I'm not aware of anyone offering an alternative superior to an informal CSP yet which seems to be the reason why Go and Clojure have picked it as well for their concurrency model. What about Quasar [0] on the JVM? 0 - http://docs.paralleluniverse.co/quasar/ http://docs.paralleluniverse.co/quasar/

davidgrenier 10y ago
But Quasar does seem to offer go-like channels? I'm unclear what kind of combinators are provided but those could be implemented.
- Matthias247 10y ago
  It does offer channels, see the linked Docu. For combinators I don't think those are really needed in the CSP environments because you can do most transformations with normal function calls and normal control flow contructs (loops, if, ...)