5 ms·

> Now imagine a parallel universe where instead of focusing on making asynchronous IO work, we focused on improving the performance of OS threads such that one

> Now imagine a parallel universe where instead of focusing on making asynchronous IO work, we focused on improving the performance of OS threads such that one can easily use hundreds of thousands of OS threads without negatively impacting performance I actually can't imagine how that would ever be accomplished at the OS level. The fact that each thread needs its own stack is an inherent limiter for efficiency, as switching stacks leads to cache misses. Asynchronous I/O has an edge because it only stores exactly as much state as it needs for its continuation, and multiple tasks can have their state in the same CPU cache line. The OS doesn't know nearly enough about your program to optimize the stack contents to only contain the state you need for the remainder of the thread. But at the programming language level the compiler does have insight into the dependencies of your continuation, so it can build a closure that has only what it needs to have. You still have asynchronous I/O at the core but the language creates an abstraction that behaves like a synchronous threaded model, as seen in C#, Kotlin, etc. This doesn't come without challenges. For example, in Kotlin the debugger is unable to show contents of variables that are not needed further down in the code because they have already been removed from the underlying closure. But I'm sure they are solvable.

dist1ll 2y ago
> But at the programming language level the compiler does have insight into the dependencies of your continuation This is really the key point - coupled with the fact that certain I/O operations are just inherently asynchronous. The TX/RX queues in NICs are an async, message passing interface - regardless of whether you're polling descriptors or receiving completion interrupts. So really, async I/O is the natural abstraction for networking.
- dietr1ch 2y ago
  Anything that happens far enough from the CPU is async, and here far probably means 10cm thanks to the speed of light not being fast enough, even in vacuum. So, any computation that spans a machine the size of our hands needs async unless you are willing to drop the clocks to push "far" a bit further away (and bring in power, heat and noise with it).
  - BenoitP 2y ago
    Thanks for putting words into this. Another cut off could be 3 cm away: the RAM. If data needs to go on the heap, be shared, one can consider the truth lives farther away than 3cm and thus has async/impure effects.
    - kaba0 2y ago
      But the same way the CPU works very hard to hide that away from you (reordering, caches, etc), green threads could do the same with barely any performance hit.
      - FridgeSeal 2y ago
        Which is fine until we decide that actually, we _would_ like that performance back, and now we re-implement async flows at language level again, realise what an advantage it’s bets us, and then that trickles into other languages as they also seek those advantages.
      - dietr1ch 2y ago
        With async, things happening far away can completely prevent you from making progress, in which case you are better off doing something else in the meantime, like simply blocking and scheduling a thread from a different process as even the cost of swapping threads, destroying the caches and other cpu state is minimal and nothing else can be done. The optimization begins instead of a single thing prevents you from making progress, it's a set of things, so you can parallelize the tasks and handle them in the seemingly random order they happen to finish. Now the question is how is this not an event loop running async tasks?
- ffsm8 2y ago
  > certain I/O operations are just inherently asynchronous. That's technically not true. The fact that its inherently async is an implementation detail. You either have blocking sync or non-blocking async. the implementation could be synchronous if the blocking didn't cause overhead and that was the proposed idea here - at least as far as I interpreted it.
  - Veserv 2y ago
    No, the hardware is frequently inherently asynchronous. You write some memory and then the hardware consumes the prepared data asynchronously, in parallel, until it informs you in some manner that the operation is complete (usually either a asynchronous interrupt, or asynchronous write to a location you are polling). You can do whatever you want after preparing the data without waiting for completion. That is a inherently asynchronous hardware interface. The software interfaces built on top of the inherently asynchronous hardware interface can either preserve or change that nature. That is a implementation detail.
    - afiori 2y ago
      Hardware is even-driven not asynchronous (the event-driven paradigm is an asynchronous paradigm, but I assume here you mean asynchronous as in async/await)
      - deleted 2y ago
        [deleted]
      - arghwhat 2y ago
        async/await is just a syntax built on top of an event driven architecture. Even at the highest level, this is backed by an event loop. Use of an event loop is the original asynchronous application design.
    - ffsm8 2y ago
      While that is technically true, it's also missing the point entirely. That's why I said you can have either blocking sync or non-blocking async. The article explicitly talks about the API provided by the OS. As a matter of fact, they're even more specific talking about spawning os threads vs async non-blocking file access. At this level, the async is an implementation detail. I guess your comment confirms that you didn't read the article, and neither did the people down voting me. As usual on HN. lots of people suffering from the Dunning Kruger complex
- vbezhenar 2y ago
  > So really, async I/O is the natural abstraction for networking. Special pointer values are also natural abstraction.
  - zarzavat 2y ago
    They’re a natural implementation detail. They could be exposed by the programming language as some other type than SomeObject* or SomeReference.
- inopinatus 2y ago
  It’s been said that the only sync I/O behaviour in software is the interrupt handler in the top half of your kernel (or equivalent). Everything else, at every other layer, however you contextualise it or write the API, is in actuality some variant of polling.
lossolo 2y ago
Not to mention the security/mitigation overhead of context switches.

dist1ll 2y ago

> But at the programming language level the compiler does have insight into the dependencies of your continuation This is really the key point - coupled with the fact that certain I/O operations are just inherently asynchronous. The TX/RX queues in NICs are an async, message passing interface - regardless of whether you're polling descriptors or receiving completion interrupts. So really, async I/O is the natural abstraction for networking.

dietr1ch 2y ago
Anything that happens far enough from the CPU is async, and here far probably means 10cm thanks to the speed of light not being fast enough, even in vacuum. So, any computation that spans a machine the size of our hands needs async unless you are willing to drop the clocks to push "far" a bit further away (and bring in power, heat and noise with it).
- BenoitP 2y ago
  Thanks for putting words into this. Another cut off could be 3 cm away: the RAM. If data needs to go on the heap, be shared, one can consider the truth lives farther away than 3cm and thus has async/impure effects.
  - kaba0 2y ago
    But the same way the CPU works very hard to hide that away from you (reordering, caches, etc), green threads could do the same with barely any performance hit.
    - FridgeSeal 2y ago
      Which is fine until we decide that actually, we _would_ like that performance back, and now we re-implement async flows at language level again, realise what an advantage it’s bets us, and then that trickles into other languages as they also seek those advantages.
    - dietr1ch 2y ago
      With async, things happening far away can completely prevent you from making progress, in which case you are better off doing something else in the meantime, like simply blocking and scheduling a thread from a different process as even the cost of swapping threads, destroying the caches and other cpu state is minimal and nothing else can be done. The optimization begins instead of a single thing prevents you from making progress, it's a set of things, so you can parallelize the tasks and handle them in the seemingly random order they happen to finish. Now the question is how is this not an event loop running async tasks?
ffsm8 2y ago
> certain I/O operations are just inherently asynchronous. That's technically not true. The fact that its inherently async is an implementation detail. You either have blocking sync or non-blocking async. the implementation could be synchronous if the blocking didn't cause overhead and that was the proposed idea here - at least as far as I interpreted it.
- Veserv 2y ago
  No, the hardware is frequently inherently asynchronous. You write some memory and then the hardware consumes the prepared data asynchronously, in parallel, until it informs you in some manner that the operation is complete (usually either a asynchronous interrupt, or asynchronous write to a location you are polling). You can do whatever you want after preparing the data without waiting for completion. That is a inherently asynchronous hardware interface. The software interfaces built on top of the inherently asynchronous hardware interface can either preserve or change that nature. That is a implementation detail.
  - afiori 2y ago
    Hardware is even-driven not asynchronous (the event-driven paradigm is an asynchronous paradigm, but I assume here you mean asynchronous as in async/await)
    - deleted 2y ago
      [deleted]
    - arghwhat 2y ago
      async/await is just a syntax built on top of an event driven architecture. Even at the highest level, this is backed by an event loop. Use of an event loop is the original asynchronous application design.
  - ffsm8 2y ago
    While that is technically true, it's also missing the point entirely. That's why I said you can have either blocking sync or non-blocking async. The article explicitly talks about the API provided by the OS. As a matter of fact, they're even more specific talking about spawning os threads vs async non-blocking file access. At this level, the async is an implementation detail. I guess your comment confirms that you didn't read the article, and neither did the people down voting me. As usual on HN. lots of people suffering from the Dunning Kruger complex
vbezhenar 2y ago
> So really, async I/O is the natural abstraction for networking. Special pointer values are also natural abstraction.
- zarzavat 2y ago
  They’re a natural implementation detail. They could be exposed by the programming language as some other type than SomeObject* or SomeReference.
inopinatus 2y ago
It’s been said that the only sync I/O behaviour in software is the interrupt handler in the top half of your kernel (or equivalent). Everything else, at every other layer, however you contextualise it or write the API, is in actuality some variant of polling.

lossolo 2y ago

Not to mention the security/mitigation overhead of context switches.