4 ms·

How do you feel about the example in the article? Is it unnecessary performance optimization to parallelize multiple adjacent asynchronous functions or should w

How do you feel about the example in the article? Is it unnecessary performance optimization to parallelize multiple adjacent asynchronous functions or should we just hope that the runtime/compiler gets smart enough to know what is safe to parallelize?

spinningslate 6y ago
Not GP but: > Is it unnecessary performance optimization to parallelize multiple adjacent asynchronous functions? The important word in your question is "async". Take that out and the answer is "no" - it's not unnecessary to parallelise. But the example serves well to illustrate at least one aspect of why red/green is wrong in my opinion: who decides what should be able to run concurrently? Let's modify the example slightly, and assume we have a function `size(url) -> Int` that returns the size in bytes of a given URL. It's making network calls, so it'll have latency. Should the function be async? As the developer of that function, I can't reasonably make that choice beyond "probably". But it comes with a cost. The caller can't now just call it: they have to await, and perhaps unbox the answer. If they're only calling it once then you've just added unnecessary complexity to their code. The _callee_ is the one who's appropriately placed to decide. Using the example, we're calling it 3 times. So, as the developer writing the _calling_ function, I can decide that it's appropriate to paralellise. To use GP's example, Erlang gets that the right way round. Functions are functions are functions. If I want to call a function synchronously, I just call it. If I want to parallelise, then I simply spawn the function instead. > or should we just hope that the runtime/compiler gets smart enough to know what is safe to parallelize? That would be a step on further still. It's definitely doable - compilers already do a buch of pretty sophisticated dataflow analysis. But even today, the red/green async model is a very limited thing. With significant constraints. Other than perhaps rust - with its focus on zero-cost abstractions - it feels at best an interim step that asks programmers to compensate for a lack of good fine-grained concurrency support in language runtimes.
- gpderetta 6y ago
  A Cilk-style depth-first work stealing executor would block by default, but if any executor has free cycles it can steal the continuation and generate more work. You would need to wait only when work is actually needed or when returning from the function. Cilk requires annotating async functions calls at the call point (with the spawn keyword to make them async), but not the funciton themselves, so there is no red/blue coloring. Synchronization is explicit via the sync keyword, but in principle I think a compiler could insert the sync point on first use of the value. I don't think a cilk style runtime would be heavier than an executor behind async/await.
  - deleted 6y ago
    [deleted]
- lmm 6y ago
  > As the developer of that function, I can't reasonably make that choice beyond "probably". But it comes with a cost. The caller can't now just call it: they have to await, and perhaps unbox the answer. If they're only calling it once then you've just added unnecessary complexity to their code. > The _callee_ is the one who's appropriately placed to decide. Using the example, we're calling it 3 times. So, as the developer writing the _calling_ function, I can decide that it's appropriate to paralellise. The right answer is polymorphism, and it's easy in a language with HKT (I do it in Scala all the time). If you make every function implicitly async (or possibly-async at the runtime's discretion) then you lose the ability to know where the yield points are, to get higher performance from blocking where appropriate, but more than that you just don't get consistent performance behaviour.
  - spinningslate 6y ago
    that's interesting - can you expand? How do higher-kinded types address the sync/async question?
    - lmm 6y ago
      You can write code that's polymorphically async or not - in its simplest form the same function returns either a value or a future/promise for that value depending on which type parameter you invoke it with. Obviously at low level there has to be two separate code paths (e.g. different system calls for a blocking vs async socket read), which you handle via typeclasses or similar, but you can write all of the composition logic (make this request, then based on the result of that make that request or...) generically in terms of an interface that both futures/promises and direct values can implement (e.g. monad).
- singularity2001 6y ago
  Functions are functions are functions. If I want to call a function synchronously, I just call it. If I want to parallelise, then I simply spawn the function instead. I don't understand why this isn't the universally accepted correct answer. Why not put the async idea on the caller side alone, as suggested and as seen in go? fetch(url) // synchronous go fetch(url) // asynchronous if the compiled code of synchronous and async functions really has to look different for fundamental reasons, the compiler can still silently emit both versions (and gc unused variants) Someone please help me understand, or rid the world of this async-await mess.
  - dnautics 6y ago
    This is what zig does, and I can attest it works fantastically. I use zig to dispatch C ABI FFI in erlang vm, which uses a unfriendly tail-call system to yield from cooperative C ABI code into it's preemptive scheduler. With zig async you can write traditional looping structures and yield back to the erlang vm inside your loop. Because zig has uncolored async you can call the same function using a different strategy (for example in a spawned os thread), and the yields will no-op without having to rewrite the function. > the compiler can still silently emit both versions I think that is zig's strategy under the hood (not 100% sure)
    - AndyKelley 6y ago
      Zig only emits one version. Either: 1. The function does not have a suspend point, so it is emitted as a normal function, and it just gets called like a normal function at the async callsite. Await is a no-op. or... 2. It has a suspend point. It is emitted with the async calling convention, and any normal callsites to it also become suspend points.
      - dnautics 6y ago
        Thanks for clarifying!! I had my understanding inverted!
  - kortex 6y ago
    `fetch (url) // synchronous` `go fetch(url) // asynchronous` This is not something you can readily abstract over. That depends on whether language is compiled vs interpreted, has a runtime or not, is scheduled or not, stack model, threading model, ownership model, and so on. `go myfunc()` implies something to dispatch, execute and reap that function and its metadata, and there's no one-size-fits-all solution. You can design a language that has this feature, but it's not without tradeoffs, and it's certainly not something you could just introduce to a language unless it already lends itself to it.
simiones 6y ago
If you want to parallelize X requests, you should spawn X threads to handle those X requests, right? Depending on how large X is and how long you expect the requests to take, this may only be worth it if your threads are actually stack-less coroutines, of course, but why not be explicit about that fact that you want parallelism here? Thread-based code (CSP) is, after all, the most well studied model of parallel computation, and for a reason. Async code which achieves real parallelism may get much harder to interpret, especially if it's non-trivial (e.g. requestA.then(requestB).then(requestC); in parallel, requestD.then(requestE); and mix this up with mild processing between A and B and C, and between D and E - you can easily get to race conditions in this type of code even in single-threaded runtimes like NodeJS).
- lmm 6y ago
  > Thread-based code (CSP) is, after all, the most well studied model of parallel computation, and for a reason. Async code which achieves real parallelism may get much harder to interpret, especially if it's non-trivial (e.g. requestA.then(requestB).then(requestC); in parallel, requestD.then(requestE); and mix this up with mild processing between A and B and C, and between D and E - you can easily get to race conditions in this type of code even in single-threaded runtimes like NodeJS). Huh? Traditional threading is not CSP; threads are generally defined as sharing memory. Async is much closer to the CSP model.
  - simiones 6y ago
    Threads + some kind of channel (a la Erlang processes or Go's goroutines) are pretty much direct programmatic representations of CSP. Shared memory does break the model to a great extent, but async code is significantly more different. The main difference is that in CSP (and thread-based models) the thread of execution is explicitly represented as an entity in the code. In async models, the threads of execution are mostly abstracted away and implemented by a runtime scheduler, while communication is done purely by function return values. Of course, you can typically implement any of these models in terms of each other (e.g. you can easily implement CSP channels in a language which offers mutexes and shared data; or you can implement them with async/await primitives). But the base support in the language and the patterns you'll see emerge in idiomatic code will be different between goroutine/CSP, shared-memory parallelism, and async/await.

spinningslate 6y ago

Not GP but: > Is it unnecessary performance optimization to parallelize multiple adjacent asynchronous functions? The important word in your question is "async". Take that out and the answer is "no" - it's not unnecessary to parallelise. But the example serves well to illustrate at least one aspect of why red/green is wrong in my opinion: who decides what should be able to run concurrently? Let's modify the example slightly, and assume we have a function `size(url) -> Int` that returns the size in bytes of a given URL. It's making network calls, so it'll have latency. Should the function be async? As the developer of that function, I can't reasonably make that choice beyond "probably". But it comes with a cost. The caller can't now just call it: they have to await, and perhaps unbox the answer. If they're only calling it once then you've just added unnecessary complexity to their code. The _callee_ is the one who's appropriately placed to decide. Using the example, we're calling it 3 times. So, as the developer writing the _calling_ function, I can decide that it's appropriate to paralellise. To use GP's example, Erlang gets that the right way round. Functions are functions are functions. If I want to call a function synchronously, I just call it. If I want to parallelise, then I simply spawn the function instead. > or should we just hope that the runtime/compiler gets smart enough to know what is safe to parallelize? That would be a step on further still. It's definitely doable - compilers already do a buch of pretty sophisticated dataflow analysis. But even today, the red/green async model is a very limited thing. With significant constraints. Other than perhaps rust - with its focus on zero-cost abstractions - it feels at best an interim step that asks programmers to compensate for a lack of good fine-grained concurrency support in language runtimes.

gpderetta 6y ago
A Cilk-style depth-first work stealing executor would block by default, but if any executor has free cycles it can steal the continuation and generate more work. You would need to wait only when work is actually needed or when returning from the function. Cilk requires annotating async functions calls at the call point (with the spawn keyword to make them async), but not the funciton themselves, so there is no red/blue coloring. Synchronization is explicit via the sync keyword, but in principle I think a compiler could insert the sync point on first use of the value. I don't think a cilk style runtime would be heavier than an executor behind async/await.
- deleted 6y ago
  [deleted]
lmm 6y ago
> As the developer of that function, I can't reasonably make that choice beyond "probably". But it comes with a cost. The caller can't now just call it: they have to await, and perhaps unbox the answer. If they're only calling it once then you've just added unnecessary complexity to their code. > The _callee_ is the one who's appropriately placed to decide. Using the example, we're calling it 3 times. So, as the developer writing the _calling_ function, I can decide that it's appropriate to paralellise. The right answer is polymorphism, and it's easy in a language with HKT (I do it in Scala all the time). If you make every function implicitly async (or possibly-async at the runtime's discretion) then you lose the ability to know where the yield points are, to get higher performance from blocking where appropriate, but more than that you just don't get consistent performance behaviour.
- spinningslate 6y ago
  that's interesting - can you expand? How do higher-kinded types address the sync/async question?
  - lmm 6y ago
    You can write code that's polymorphically async or not - in its simplest form the same function returns either a value or a future/promise for that value depending on which type parameter you invoke it with. Obviously at low level there has to be two separate code paths (e.g. different system calls for a blocking vs async socket read), which you handle via typeclasses or similar, but you can write all of the composition logic (make this request, then based on the result of that make that request or...) generically in terms of an interface that both futures/promises and direct values can implement (e.g. monad).
singularity2001 6y ago
Functions are functions are functions. If I want to call a function synchronously, I just call it. If I want to parallelise, then I simply spawn the function instead. I don't understand why this isn't the universally accepted correct answer. Why not put the async idea on the caller side alone, as suggested and as seen in go? fetch(url) // synchronous go fetch(url) // asynchronous if the compiled code of synchronous and async functions really has to look different for fundamental reasons, the compiler can still silently emit both versions (and gc unused variants) Someone please help me understand, or rid the world of this async-await mess.
- dnautics 6y ago
  This is what zig does, and I can attest it works fantastically. I use zig to dispatch C ABI FFI in erlang vm, which uses a unfriendly tail-call system to yield from cooperative C ABI code into it's preemptive scheduler. With zig async you can write traditional looping structures and yield back to the erlang vm inside your loop. Because zig has uncolored async you can call the same function using a different strategy (for example in a spawned os thread), and the yields will no-op without having to rewrite the function. > the compiler can still silently emit both versions I think that is zig's strategy under the hood (not 100% sure)
  - AndyKelley 6y ago
    Zig only emits one version. Either: 1. The function does not have a suspend point, so it is emitted as a normal function, and it just gets called like a normal function at the async callsite. Await is a no-op. or... 2. It has a suspend point. It is emitted with the async calling convention, and any normal callsites to it also become suspend points.
    - dnautics 6y ago
      Thanks for clarifying!! I had my understanding inverted!
- kortex 6y ago
  `fetch (url) // synchronous` `go fetch(url) // asynchronous` This is not something you can readily abstract over. That depends on whether language is compiled vs interpreted, has a runtime or not, is scheduled or not, stack model, threading model, ownership model, and so on. `go myfunc()` implies something to dispatch, execute and reap that function and its metadata, and there's no one-size-fits-all solution. You can design a language that has this feature, but it's not without tradeoffs, and it's certainly not something you could just introduce to a language unless it already lends itself to it.

simiones 6y ago

If you want to parallelize X requests, you should spawn X threads to handle those X requests, right? Depending on how large X is and how long you expect the requests to take, this may only be worth it if your threads are actually stack-less coroutines, of course, but why not be explicit about that fact that you want parallelism here? Thread-based code (CSP) is, after all, the most well studied model of parallel computation, and for a reason. Async code which achieves real parallelism may get much harder to interpret, especially if it's non-trivial (e.g. requestA.then(requestB).then(requestC); in parallel, requestD.then(requestE); and mix this up with mild processing between A and B and C, and between D and E - you can easily get to race conditions in this type of code even in single-threaded runtimes like NodeJS).

lmm 6y ago
> Thread-based code (CSP) is, after all, the most well studied model of parallel computation, and for a reason. Async code which achieves real parallelism may get much harder to interpret, especially if it's non-trivial (e.g. requestA.then(requestB).then(requestC); in parallel, requestD.then(requestE); and mix this up with mild processing between A and B and C, and between D and E - you can easily get to race conditions in this type of code even in single-threaded runtimes like NodeJS). Huh? Traditional threading is not CSP; threads are generally defined as sharing memory. Async is much closer to the CSP model.
- simiones 6y ago
  Threads + some kind of channel (a la Erlang processes or Go's goroutines) are pretty much direct programmatic representations of CSP. Shared memory does break the model to a great extent, but async code is significantly more different. The main difference is that in CSP (and thread-based models) the thread of execution is explicitly represented as an entity in the code. In async models, the threads of execution are mostly abstracted away and implemented by a runtime scheduler, while communication is done purely by function return values. Of course, you can typically implement any of these models in terms of each other (e.g. you can easily implement CSP channels in a language which offers mutexes and shared data; or you can implement them with async/await primitives). But the base support in the language and the patterns you'll see emerge in idiomatic code will be different between goroutine/CSP, shared-memory parallelism, and async/await.