10 ms·

> Asynchronous code (e.g. callbacks) is more efficient (letting you do other work while waiting for things) but is also more complicated (manually saving and re

by jmull 4y ago

> Asynchronous code (e.g. callbacks) is more efficient (letting you do other work while waiting for things) but is also more complicated (manually saving and restoring state). This is a persistent misunderstanding about async... it is not generally more efficient than non-async code. In fact, the overhead of async may slow things down. (There can be other downsides as well. E.g. debugging may be more difficult.) For it to help, you need to have something better for the current thread to be doing than waiting for whatever it is you're waiting for. Whatever that is, another worker thread could be doing it, so what we're really talking about is that async may let you reduce the number of threads you're using. (Async is so useful in JS because it lets you reduce the number of threads to one, which lets you sidestep all kinds of hairy issues.) That might be good enough, depending on your situation, to overcome the downsides of async worthwhile. But it may very well not. Anyway: don't use async because it's "more efficient".

KMag 4y ago
Agreed in general, that async code is generally less efficient than epoll/kqueue/poll/select/etc. non-blocking I/O. However, note that async code is often quite a bit more efficient than naive Java 1.3-style (pre-NIO) tons-of-blocked-threads code. Now, io_uring async code is yet another story. I was working at LimeWire when Apple finally shipped a 1.4 JVM as their system JVM. The whole thing got much lighter weight when we could use non-blocking I/O.
- junon 4y ago
  Eh? Async code almost always uses the APIs you describe.
  - KMag 4y ago
    The async runtime is almost always built on top of non-blocking I/O APIs, and the extra layer of abstraction adds overhead. You're rarely calling epoll in your async code, but rather your runtime is handling all of that for you. That is, unless you're using io_uring, POSIX AIO, etc., in which case your async code might not have the extra layer of abstraction on top of the system APIs.
    - junon 4y ago
      io_uring isn't typically used directly, you're most likely using liburing which is already an abstraction. I don't really see your point.
mr_00ff00 4y ago
But a single thread with async can, and for many tasks, is more efficient that a single thread without async, because you aren’t waiting on requests. One point also is that until recently most personal computers did not have multiple cores, aka you could think at the lowest level that even multiple threads were just more dangerous async lol Even making new threads in the JVM isn’t guaranteed to result in more than a single OS thread.
- jcelerier 4y ago
  > One point also is that until recently most personal computers did not have multiple cores, core 2 duo has been released 17 years ago
  - jcranmer 4y ago
    Dual core started coming out ~2005. It took a few more years for it to dominate the product lineup. Computers tend to have a replacement cycle of ~5 years. Consequently, it's not until about 2012-ish that you could reasonably expect a personal computer to have a multicore computer. (Yeah, that's still a decade ago, but that's a good deal more recent than 17 years.)
    - dymk 4y ago
      A decade is a long time for computers
      - mr_00ff00 4y ago
        There was plenty of async JavaScript code and multithreaded Java code out there in 2013, I promise
the_duke 4y ago
You are leaving out an important aspect: context switches are expensive, especially with all the mitigations introduced after Spectre et al. Doing more work in a single thread saves a lot of context switches, especially if IO makes up a large portion of the workload. And even more so with other mechanisms to minimize syscalls, like io_uring on Linux. Async overhead is highly dependent on the way the async runtime is implemented. It's true that async requires scheduling and task switching to be implemented in userspace, but assuming that async is used as an alternative to threads, then the OS would be doing a similar amount of work anyway. The OS may be able to do that more efficiently because it has more information about the system. Or it may be the opposite, because the application has more information about the workloads. "async = overhead" as a general statement is not correct.
- otabdeveloper4 4y ago
  Context switches happen regardless of whether you're using kernel mode ("threads") or user mode ("async") scheduling. And kernel mode actually has much more efficient and performant context switches, because the kernel can access hardware directly. (It is sometimes useful to be able to do custom user mode scheduling, but certainly not because of "context switches".)
  - deleted 4y ago
    [deleted]
  - otherjason 4y ago
    Do you have a citation for kernel mode having more efficient context switches? What kind of direct hardware access are you referring to that would be better than pushing the register context onto the stack? In my experience, the exact opposite is true, particularly in the era of CPU mitigations that require TLB flushes upon every kernel-mode context switch.
    - messe 4y ago
      You're right, kernel-level context switching is much slower than user-level context switching. User-level can also have the advantage of having more actual context about the task that is running, meaning that it's often able to avoid saving/restoring as much data as a kernel-level switch would. See Go's green threads for a great example of this kind of cooperation between runtime and language. > Do you have a citation for kernel mode having more efficient context switches? What kind of direct hardware access are you referring to that would be better than pushing the register context onto the stack? The closest thing to this that I can think of is on 32-bit x86 which did have hardware assisted context switching via TSRs. As it happens, everybody stopped using it because it was too slow, and a bit painful unless you fully bought into x86's awful segmentation model. Early Linux kernels use it if you want to see it in action.
naasking 4y ago
> Anyway: don't use async because it's "more efficient". I don't think efficiency has only one meaning. For instance, you can create millions of coroutines while you're limited to only thousands of threads. Doesn't that mean your program is making better use of your hardware and thus is more efficient?
- nmilo 4y ago
  No, your processor can only do n things at a time, where n is the number of cores (probably like 8) (simplifying hugely of course). It's pretty easy to fill out 8 cores with 1000s of threads, the coroutines only become useful when almost all of them are blocking on a socket or something, periodically switching on and off to handle it.
  - naasking 4y ago
    Yes, but contexts where you're creating a million coroutines are almost certainly I/O bound (like a web server).
  - deleted 4y ago
    [deleted]
- otabdeveloper4 4y ago
  Coroutines and threads are not that different. Threads usually preallocate some stack space, while coroutines usually don't. But that isn't such a big deal.
  - naasking 4y ago
    That's not the only difference. Coroutines also typically don't have nearly as many concurrency hazards because you can control when you yield.
    - dymk 4y ago
      coroutines can be either cooperative or not, depending on implementation, so it depends
duped 4y ago
Asynchronous execution is more efficient if and only if the time spent waiting for a call to complete is longer than the time spent checking if the call has completed plus the time spent dispatching to another asynchronous call (and waiting vs checking if it is completed, and so on). Remember that concurrency is not parallelism, it's not about the number of threads being used.
branko_d 4y ago
There is another use case for async in client-side JavaScript that often gets overlooked: modeling the flow of UI. So, let's say you need to show a "dialog" which, upon exit, should execute a piece of code (e.g. to show another piece of UI). Maybe that dialog can be invoked from multiple places, and the code to execute (after the dialog) changes accordingly. You could do that via callbacks, but that is not very composable, and is generally messy. Better solution: a Promise which gets resolved when the user exits the dialog! Composing multiple pieces of UI then becomes equivalent to composing multiple (async) function calls, you can pass output from one as the input to another, use exceptions to "circuit break" the UI flow, etc.
- 8n4vidtmkvmk 4y ago
  I'm not convinced of the dialog use-case. Seems rather sketchy to await a user to click OK. What happens when you add another button or two to the dialog?
  - branko_d 4y ago
    You return a different result (i.e. the Promise resolves to a different result). The calling code can then decide what to do with it.
  - El_RIDO 4y ago
    In this model, you wouldn't await the user clicking OK, but exiting the modal. The promise only resolves "user took a decision", not the actual choice they took. The result then contains if the user clicked OK, Cancel, [X] to close the modal or any other state transition that you may have added to that dialog. The recipient (not necessarily the caller that initiated the dialog, in case you chain the responses) can then act accordingly and presumably triggers a different action on OK then the other state transitions.

by jmull 4y ago

KMag 4y ago

Agreed in general, that async code is generally less efficient than epoll/kqueue/poll/select/etc. non-blocking I/O. However, note that async code is often quite a bit more efficient than naive Java 1.3-style (pre-NIO) tons-of-blocked-threads code. Now, io_uring async code is yet another story. I was working at LimeWire when Apple finally shipped a 1.4 JVM as their system JVM. The whole thing got much lighter weight when we could use non-blocking I/O.

junon 4y ago
Eh? Async code almost always uses the APIs you describe.
- KMag 4y ago
  The async runtime is almost always built on top of non-blocking I/O APIs, and the extra layer of abstraction adds overhead. You're rarely calling epoll in your async code, but rather your runtime is handling all of that for you. That is, unless you're using io_uring, POSIX AIO, etc., in which case your async code might not have the extra layer of abstraction on top of the system APIs.
  - junon 4y ago
    io_uring isn't typically used directly, you're most likely using liburing which is already an abstraction. I don't really see your point.

mr_00ff00 4y ago

But a single thread with async can, and for many tasks, is more efficient that a single thread without async, because you aren’t waiting on requests. One point also is that until recently most personal computers did not have multiple cores, aka you could think at the lowest level that even multiple threads were just more dangerous async lol Even making new threads in the JVM isn’t guaranteed to result in more than a single OS thread.

jcelerier 4y ago
> One point also is that until recently most personal computers did not have multiple cores, core 2 duo has been released 17 years ago
- jcranmer 4y ago
  Dual core started coming out ~2005. It took a few more years for it to dominate the product lineup. Computers tend to have a replacement cycle of ~5 years. Consequently, it's not until about 2012-ish that you could reasonably expect a personal computer to have a multicore computer. (Yeah, that's still a decade ago, but that's a good deal more recent than 17 years.)
  - dymk 4y ago
    A decade is a long time for computers
    - mr_00ff00 4y ago
      There was plenty of async JavaScript code and multithreaded Java code out there in 2013, I promise

the_duke 4y ago

You are leaving out an important aspect: context switches are expensive, especially with all the mitigations introduced after Spectre et al. Doing more work in a single thread saves a lot of context switches, especially if IO makes up a large portion of the workload. And even more so with other mechanisms to minimize syscalls, like io_uring on Linux. Async overhead is highly dependent on the way the async runtime is implemented. It's true that async requires scheduling and task switching to be implemented in userspace, but assuming that async is used as an alternative to threads, then the OS would be doing a similar amount of work anyway. The OS may be able to do that more efficiently because it has more information about the system. Or it may be the opposite, because the application has more information about the workloads. "async = overhead" as a general statement is not correct.

otabdeveloper4 4y ago
Context switches happen regardless of whether you're using kernel mode ("threads") or user mode ("async") scheduling. And kernel mode actually has much more efficient and performant context switches, because the kernel can access hardware directly. (It is sometimes useful to be able to do custom user mode scheduling, but certainly not because of "context switches".)
- deleted 4y ago
  [deleted]
- otherjason 4y ago
  Do you have a citation for kernel mode having more efficient context switches? What kind of direct hardware access are you referring to that would be better than pushing the register context onto the stack? In my experience, the exact opposite is true, particularly in the era of CPU mitigations that require TLB flushes upon every kernel-mode context switch.
  - messe 4y ago
    You're right, kernel-level context switching is much slower than user-level context switching. User-level can also have the advantage of having more actual context about the task that is running, meaning that it's often able to avoid saving/restoring as much data as a kernel-level switch would. See Go's green threads for a great example of this kind of cooperation between runtime and language. > Do you have a citation for kernel mode having more efficient context switches? What kind of direct hardware access are you referring to that would be better than pushing the register context onto the stack? The closest thing to this that I can think of is on 32-bit x86 which did have hardware assisted context switching via TSRs. As it happens, everybody stopped using it because it was too slow, and a bit painful unless you fully bought into x86's awful segmentation model. Early Linux kernels use it if you want to see it in action.

naasking 4y ago

> Anyway: don't use async because it's "more efficient". I don't think efficiency has only one meaning. For instance, you can create millions of coroutines while you're limited to only thousands of threads. Doesn't that mean your program is making better use of your hardware and thus is more efficient?

nmilo 4y ago
No, your processor can only do n things at a time, where n is the number of cores (probably like 8) (simplifying hugely of course). It's pretty easy to fill out 8 cores with 1000s of threads, the coroutines only become useful when almost all of them are blocking on a socket or something, periodically switching on and off to handle it.
- naasking 4y ago
  Yes, but contexts where you're creating a million coroutines are almost certainly I/O bound (like a web server).
- deleted 4y ago
  [deleted]
otabdeveloper4 4y ago
Coroutines and threads are not that different. Threads usually preallocate some stack space, while coroutines usually don't. But that isn't such a big deal.
- naasking 4y ago
  That's not the only difference. Coroutines also typically don't have nearly as many concurrency hazards because you can control when you yield.
  - dymk 4y ago
    coroutines can be either cooperative or not, depending on implementation, so it depends

duped 4y ago

Asynchronous execution is more efficient if and only if the time spent waiting for a call to complete is longer than the time spent checking if the call has completed plus the time spent dispatching to another asynchronous call (and waiting vs checking if it is completed, and so on). Remember that concurrency is not parallelism, it's not about the number of threads being used.

branko_d 4y ago

There is another use case for async in client-side JavaScript that often gets overlooked: modeling the flow of UI. So, let's say you need to show a "dialog" which, upon exit, should execute a piece of code (e.g. to show another piece of UI). Maybe that dialog can be invoked from multiple places, and the code to execute (after the dialog) changes accordingly. You could do that via callbacks, but that is not very composable, and is generally messy. Better solution: a Promise which gets resolved when the user exits the dialog! Composing multiple pieces of UI then becomes equivalent to composing multiple (async) function calls, you can pass output from one as the input to another, use exceptions to "circuit break" the UI flow, etc.

8n4vidtmkvmk 4y ago
I'm not convinced of the dialog use-case. Seems rather sketchy to await a user to click OK. What happens when you add another button or two to the dialog?
- branko_d 4y ago
  You return a different result (i.e. the Promise resolves to a different result). The calling code can then decide what to do with it.
- El_RIDO 4y ago
  In this model, you wouldn't await the user clicking OK, but exiting the modal. The promise only resolves "user took a decision", not the actual choice they took. The result then contains if the user clicked OK, Cancel, [X] to close the modal or any other state transition that you may have added to that dialog. The recipient (not necessarily the caller that initiated the dialog, in case you chain the responses) can then act accordingly and presumably triggers a different action on OK then the other state transitions.