3 ms·

> This is correct, it's for increasing _throughput_ in concurrent scenarios. I believe you mean the exact opposite. It decreases latency (because task B isn't

> This is correct, it's for increasing _throughput_ in concurrent scenarios. I believe you mean the exact opposite. It decreases latency (because task B isn't blocked waiting for task A to complete) but it does so at the expense of decreased throughput. The context switches add overhead. If you just synchronously run A then B, the overall time would be shorter (higher throughput) because of less context switching overhead.

reubenbond 5y ago
If task A & B perform IO (eg, a DB call) and the alternatives are running them sequentially on one thread or running them concurrently (via async/await) on one thread, then running them concurrently can both decrease end-to-end latency and increase throughput. > If you just synchronously run A then B, the overall time would be shorter (higher throughput) because of less context switching overhead. There are no context switches: async/await isn't threads. The compiler generates state machines which are scheduled on a thread pool. Basically, each time an event happens (eg, database request completes or times out, or a new request arrives), that state machine is scheduled again so that it can observe that event. This doesn't involve context switching: you can have 1 thread or N threads happily working away on many concurrent tasks without needing to context switch between them.
- munificent 5y ago
  > There are no context switches: async/await isn't threads. By "context switch", I didn't meant to imply "hardware thread context switch", just the general sense of "spend some CPU time messing about with scheduling". There is overhead to async in that you're unwinding the stack, bouncing to the thread pool scheduler, loading variables from the heap (since your async code was compiled to closures) back onto the stack, etc. As far as I know, it's always possible to complete some given set of work in less total time (i.e. highest throughput) using a carefully hand-written multithreaded program than it is using async. Of course, most people don't have the luxury of writing and maintaining that program, so async code can often be a net win to both throughput and latency, but the overhead is there. It's analogous to going from a manually-memory language to a language with GC. The GC makes your life easier and makes it much easier to write programs that are generally efficient, but it does incur some level of runtime overhead when compared to a program with optimally written manual alloc and free.
  - deleted 5y ago
    [deleted]
SideburnsOfDoom 5y ago
No, I mean that yielding allows more requests to be executed at the same time on the same number of threads, increasing throughput. Overhead of context switches is not that relevant, this is are small fry compared to e.g. waiting 100s of milliseconds (or more) for a DB or API. Yielding instead of busy-waiting, as I said above, allow another request that is ready to execute to do so sooner. This leads to higher throughput. The other reply from reubenbond ( https://twitter.com/reubenbond https://twitter.com/reubenbond ) is correct. Also the implication that async does sometimes decrease end-to-end latency because you don't have wait for request A to complete before starting request B. async/await is not the same thing as threading, it is about using a fraction of a thread: when awaiting, "there is no thread" being used. https://blog.stephencleary.com/2013/11/there-is-no-thread.html https://blog.stephencleary.com/2013/11/there-is-no-thread.ht... > I believe you mean the exact opposite. As an aside, how about you say what you mean, and I'll work on what I mean.

reubenbond 5y ago

If task A & B perform IO (eg, a DB call) and the alternatives are running them sequentially on one thread or running them concurrently (via async/await) on one thread, then running them concurrently can both decrease end-to-end latency and increase throughput. > If you just synchronously run A then B, the overall time would be shorter (higher throughput) because of less context switching overhead. There are no context switches: async/await isn't threads. The compiler generates state machines which are scheduled on a thread pool. Basically, each time an event happens (eg, database request completes or times out, or a new request arrives), that state machine is scheduled again so that it can observe that event. This doesn't involve context switching: you can have 1 thread or N threads happily working away on many concurrent tasks without needing to context switch between them.

munificent 5y ago
> There are no context switches: async/await isn't threads. By "context switch", I didn't meant to imply "hardware thread context switch", just the general sense of "spend some CPU time messing about with scheduling". There is overhead to async in that you're unwinding the stack, bouncing to the thread pool scheduler, loading variables from the heap (since your async code was compiled to closures) back onto the stack, etc. As far as I know, it's always possible to complete some given set of work in less total time (i.e. highest throughput) using a carefully hand-written multithreaded program than it is using async. Of course, most people don't have the luxury of writing and maintaining that program, so async code can often be a net win to both throughput and latency, but the overhead is there. It's analogous to going from a manually-memory language to a language with GC. The GC makes your life easier and makes it much easier to write programs that are generally efficient, but it does incur some level of runtime overhead when compared to a program with optimally written manual alloc and free.
- deleted 5y ago
  [deleted]

SideburnsOfDoom 5y ago

No, I mean that yielding allows more requests to be executed at the same time on the same number of threads, increasing throughput. Overhead of context switches is not that relevant, this is are small fry compared to e.g. waiting 100s of milliseconds (or more) for a DB or API. Yielding instead of busy-waiting, as I said above, allow another request that is ready to execute to do so sooner. This leads to higher throughput. The other reply from reubenbond ( https://twitter.com/reubenbond https://twitter.com/reubenbond ) is correct. Also the implication that async does sometimes decrease end-to-end latency because you don't have wait for request A to complete before starting request B. async/await is not the same thing as threading, it is about using a fraction of a thread: when awaiting, "there is no thread" being used. https://blog.stephencleary.com/2013/11/there-is-no-thread.html https://blog.stephencleary.com/2013/11/there-is-no-thread.ht... > I believe you mean the exact opposite. As an aside, how about you say what you mean, and I'll work on what I mean.