5 ms·

The entire unix operating system is designed so that you can write sequential code. It abstracts concurrency away. It’s incredible.

deleted 3y ago
[deleted]
gwbas1c 3y ago
The point of async is to move concurrency from the OS into the process. The specific issue is the context switch. The compiler, with async is able to be much more performant than the context switch that the OS provides. Depending on the kind of application you're writing, this is either splitting hairs or very, very important. Applications that handle many (hundreds, 1000s,) of concurrent IO will have a noticeable performance improvement using async versus the OS's context switching. But, there's a more important thing to consider: "async" in a programming language communicates that a method can block. It allows the caller to start the call and do something while it's waiting for the result, without needing to get into the weeds of threading. Purely relying on the OS for context switching means that it's hard to know what methods block.
- starcraft2wol 3y ago
  I agree with most of this. > without needing to get into the weeds of threading A thread is the exact concept needed to describe that and preserve “if else then” sequential code. > 1000s of threads Linux handles thousands of threads just fine. If you’re using a scripting language like python, context switching is the least of your performance concerns.
  - gwbas1c 3y ago
    > A thread is the exact concept needed to describe that and preserve “if else then” sequential code. I suggest looking at C# and Javascript that implement async very, very well. The difference is that with conventional threading semantics, when you join a thread, it doesn't return a result. You still need to write some kind of "thing" to get your result from the subthread to whatever's waiting on it. (C# also provides a less-well-known BeginInvoke mechanism which is somewhat cleaner than join.) In contrast, the promise (Javascript) or task (C#) has a result. Instead of joining a thread, the await keyword gets the result, just like calling a method. > Linux handles thousands of threads just fine. Yes... And no... It doesn't matter what OS you're on, each thread needs its own allocated stack space and has the overhead of context switching. "async" optimizes that by putting data that would normally go into many different stack spaces into the heap and jumping around among concurrent operations without the context switch. Again, depending on what you're doing, that's either splitting hairs, or really making a tangible improvement. But you can't argue that more allocated stacks, and more context switches, is faster than doing it in process. At that point you're arguing with fact.
    - gwbas1c 3y ago
      Too late to edit: I should also point out that Rust has the same issue with joining a thread: It doesn't give you the result of the function; unlike awaiting on a promise. (I really struggled with async rust, so I'll admit that I don't remember the name of the type that represents the promise.)
    - starcraft2wol 3y ago
      > C#/JavaScript I am familiar with how these are implemented. My opinion is the same. Yes you can implement slightly lighter weight threads in a language itself. > It doesn't matter what OS you're on, each thread needs its own allocated stack space and has the overhead of context switching This is a quantitative question. I know what it does, the question is how much slower it is than whatever async construct you want to use.
- returningfory2 3y ago
  Just a note that it seems many people think the main problem isn't the context switch. Rather, Linux by default allocates a very large stack to every thread, and thus many threads lead to high memory usage. Async makes this better. See e.g. this recent clip from one of the engineers on Project Loom in which they argue that the context switch is relatively low overhead: https://youtu.be/07V08SB1l8c?si=i0v9w90Kb_M0I4gP&t=966 https://youtu.be/07V08SB1l8c?si=i0v9w90Kb_M0I4gP&t=966
  - gwbas1c 3y ago
    At least on Windows I've reconfigured the stack space. (I had to run an experiment that required a lot of threads.) Is this something that's hard to do on Linux?
    - steveklabnik 3y ago
      It's the same, but this issue gets to one of the hearts here: Both of these APIs are set by you, the user. You can choose how big to set your stack size, it's true. However, what value do you actually set? Too high, and you're still using too much memory, though admittedly less. Too low, and you either need to accept death by stack overflow, or runtime detect this case and fix things up. With async/await in Rust, the compiler can statically see how large the stack size needs to be. Each "thread" will have a perfectly sized stack. No user intervention required, no fiddling with settings.
      - 10000truths 3y ago
        > Both of these APIs are set by you, the user. You can choose how big to set your stack size, it's true. However, what value do you actually set? Too high, and you're still using too much memory, though admittedly less. Too low, and you either need to accept death by stack overflow, or runtime detect this case and fix things up. This is a solved problem - it's "just" the longest path in a call graph where each node is weighted by its function's stack frame size. > With async/await in Rust, the compiler can statically see how large the stack size needs to be. Each "thread" will have a perfectly sized stack. No user intervention required, no fiddling with settings. There is nothing stopping a compiler from doing the exact same analysis for a sync function at compile time (barring exceptions like varargs or deliberate recursion). They just... haven't, for some reason. It's a shame that it took until Zig for it to be addressed.

deleted 3y ago

[deleted]

gwbas1c 3y ago

The point of async is to move concurrency from the OS into the process. The specific issue is the context switch. The compiler, with async is able to be much more performant than the context switch that the OS provides. Depending on the kind of application you're writing, this is either splitting hairs or very, very important. Applications that handle many (hundreds, 1000s,) of concurrent IO will have a noticeable performance improvement using async versus the OS's context switching. But, there's a more important thing to consider: "async" in a programming language communicates that a method can block. It allows the caller to start the call and do something while it's waiting for the result, without needing to get into the weeds of threading. Purely relying on the OS for context switching means that it's hard to know what methods block.

starcraft2wol 3y ago
I agree with most of this. > without needing to get into the weeds of threading A thread is the exact concept needed to describe that and preserve “if else then” sequential code. > 1000s of threads Linux handles thousands of threads just fine. If you’re using a scripting language like python, context switching is the least of your performance concerns.
- gwbas1c 3y ago
  > A thread is the exact concept needed to describe that and preserve “if else then” sequential code. I suggest looking at C# and Javascript that implement async very, very well. The difference is that with conventional threading semantics, when you join a thread, it doesn't return a result. You still need to write some kind of "thing" to get your result from the subthread to whatever's waiting on it. (C# also provides a less-well-known BeginInvoke mechanism which is somewhat cleaner than join.) In contrast, the promise (Javascript) or task (C#) has a result. Instead of joining a thread, the await keyword gets the result, just like calling a method. > Linux handles thousands of threads just fine. Yes... And no... It doesn't matter what OS you're on, each thread needs its own allocated stack space and has the overhead of context switching. "async" optimizes that by putting data that would normally go into many different stack spaces into the heap and jumping around among concurrent operations without the context switch. Again, depending on what you're doing, that's either splitting hairs, or really making a tangible improvement. But you can't argue that more allocated stacks, and more context switches, is faster than doing it in process. At that point you're arguing with fact.
  - gwbas1c 3y ago
    Too late to edit: I should also point out that Rust has the same issue with joining a thread: It doesn't give you the result of the function; unlike awaiting on a promise. (I really struggled with async rust, so I'll admit that I don't remember the name of the type that represents the promise.)
  - starcraft2wol 3y ago
    > C#/JavaScript I am familiar with how these are implemented. My opinion is the same. Yes you can implement slightly lighter weight threads in a language itself. > It doesn't matter what OS you're on, each thread needs its own allocated stack space and has the overhead of context switching This is a quantitative question. I know what it does, the question is how much slower it is than whatever async construct you want to use.
returningfory2 3y ago
Just a note that it seems many people think the main problem isn't the context switch. Rather, Linux by default allocates a very large stack to every thread, and thus many threads lead to high memory usage. Async makes this better. See e.g. this recent clip from one of the engineers on Project Loom in which they argue that the context switch is relatively low overhead: https://youtu.be/07V08SB1l8c?si=i0v9w90Kb_M0I4gP&t=966 https://youtu.be/07V08SB1l8c?si=i0v9w90Kb_M0I4gP&t=966
- gwbas1c 3y ago
  At least on Windows I've reconfigured the stack space. (I had to run an experiment that required a lot of threads.) Is this something that's hard to do on Linux?
  - steveklabnik 3y ago
    It's the same, but this issue gets to one of the hearts here: Both of these APIs are set by you, the user. You can choose how big to set your stack size, it's true. However, what value do you actually set? Too high, and you're still using too much memory, though admittedly less. Too low, and you either need to accept death by stack overflow, or runtime detect this case and fix things up. With async/await in Rust, the compiler can statically see how large the stack size needs to be. Each "thread" will have a perfectly sized stack. No user intervention required, no fiddling with settings.
    - 10000truths 3y ago
      > Both of these APIs are set by you, the user. You can choose how big to set your stack size, it's true. However, what value do you actually set? Too high, and you're still using too much memory, though admittedly less. Too low, and you either need to accept death by stack overflow, or runtime detect this case and fix things up. This is a solved problem - it's "just" the longest path in a call graph where each node is weighted by its function's stack frame size. > With async/await in Rust, the compiler can statically see how large the stack size needs to be. Each "thread" will have a perfectly sized stack. No user intervention required, no fiddling with settings. There is nothing stopping a compiler from doing the exact same analysis for a sync function at compile time (barring exceptions like varargs or deliberate recursion). They just... haven't, for some reason. It's a shame that it took until Zig for it to be addressed.