5 ms·

Asynchronous IO: the next billion-dollar mistake?

157 points by signa11 2y ago

nullindividual 2y ago
We do live in the universe of high performance threads with asynchronous I/O. The author is looking for Windows NT.
- wtallis 2y ago
  The same Windows NT whose decades-old async IO capabilities are so good they immediately cloned io_uring?
  - lmz 2y ago
    Sorry, which one came first between io_uring and windows RIO? https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/hh997032(v=ws.11) https://learn.microsoft.com/en-us/previous-versions/windows/...
    - wtallis 2y ago
      I think you missed the point: io_uring is the newer API. Microsoft saw the new async IO API developed for Linux, and made a version for Windows. If what Windows has had all along is so great, why did they bother cloning io_uring when io_uring came along? It definitely isn't because there's any significant body of software relying on io_uring yet. My theory: people like to extol the virtues of NT's IOCP as "checking the box" for being async, but seldom bother to assess whether the decades-old async API actually offers good performance in comparison to more recent alternatives. But some parts of Microsoft understand quite well that they have pervasive issues with IO performance: replacing WSL1 with WSL2, introducing DirectStorage and a partial copy of io_uring, the Dev Drive feature in Windows 11—all changes that are driven by the understanding that the status quo isn't all that great.
      - nullindividual 2y ago
        > I think you missed the point: io_uring is the newer API. GP was pointing out that io_uring came after RIO; RIO has similar capabilities to io_uring for the network space. GP did not "miss the point". > Microsoft understand quite well that they have pervasive issues with IO performance This has nothing to do with IOCP but rather the file system filters. This is why removal of file system filters from "DevDrive" (a ReFS volume) improves performance vs a ReFS volume with file system filters in place. > If what Windows has had all along is so great, why did they bother cloning io_uring when io_uring came along? IoRing eliminates the need to copy buffers between kernel and user space [0][1]. With regards to performance, this is a bench between a traditional multithreaded UDP server and an RIO IOCP UDP server. The gains are significant. [2] And further comparisons... [3] > seldom bother to assess whether the decades-old async API actually offers good performance in comparison to more recent alternatives There are no complete alternatives [r]. Nor is Linux fully asynchronous throughout the kernel. > introducing DirectStorage Not really relevant to the context. DirectStorage is a method to deliver data from nVME storage with minimal CPU overhead and/or compressed [4]. Such a system would improve performance on any OS and indeed, we can see that on PS OS (PS5, FreeBSD-based (probably)). Again, limited in scope. I'm honestly not sure what you're attempting to argue. IoRing brings improvements, yes, but isn't a replacement of IOCP. We should see improvements in any mainstream kernel as a good thing. It's progress. And hopefully the Linux kernel gets to full async I/O at some point in the future. [0] https://windows-internals.com/i-o-rings-when-one-i-o-operation-is-not-enough/ https://windows-internals.com/i-o-rings-when-one-i-o-operati... (this is from a co-author of Windows Internals) [1] https://windows-internals.com/ioring-vs-io_uring-a-comparison-of-windows-and-linux-implementations/ https://windows-internals.com/ioring-vs-io_uring-a-compariso... [2] https://serverframework.com/asynchronousevents/2012/08/winsock-registered-io-io-completion-port-performance.html https://serverframework.com/asynchronousevents/2012/08/winso... [3] https://speakerdeck.com/trent/pyparallel-how-we-removed-the-gil-and-exploited-all-cores https://speakerdeck.com/trent/pyparallel-how-we-removed-the-... [4] https://stackoverflow.com/questions/73878597/what-are-the-differences-between-windows-overlapped-io-registered-io-rio-and https://stackoverflow.com/questions/73878597/what-are-the-di... [5] https://learn.microsoft.com/en-us/gaming/gdk/_content/gc/system/overviews/directstorage/directstorage-overview https://learn.microsoft.com/en-us/gaming/gdk/_content/gc/sys...
  - nullindividual 2y ago
    Yes, it is nearly a 1:1 copy of io_uring, but unlike io_uring which applies to a single function, all I/O in the NT kernel is asynchronous. IOCP acts on file, network, mail slot, pipes, etc. IoRing/io_uring is for files only. RegisteredIO is network only. While Windows userland itself may be a 'mess' and archaic in it's own way (among the other anti-consumer bits), the NT kernel is technically quite advanced, not only for it's time, but at the present time. David Cutler's team got the kernel architecture correct the first time. Linux still has a ways to catch up in certain aspects.
    - lmz 2y ago
      A free sample chapter from Windows Internals about the I/O system for the curious: https://www.microsoftpressstore.com/articles/article.aspx?p=2201309&seqNum=3 https://www.microsoftpressstore.com/articles/article.aspx?p=...
      - nullindividual 2y ago
        Thanks, but I've been referring people like you who lack an understanding of Windows I/O (and VMM) to this book since at least the 4th edition. I've had my copy of the 7th since it came out.
        lmz 2y ago
        It wasn't for your benefit somce you obviously are familiar with the topic, but for passers-by who may be interested in the topic.
deleted 2y ago
[deleted]
NinoScript 2y ago
Interesting take. I’d like to see such an OS, I wonder what requirements/limitations it would have.
CJefferson 2y ago
I generally agree with this article. There are programs where async IO is great, but in my experience it stops being useful as your code “does more stuff”. The few large scale async systems I’ve worked with end up with functions taking too long, so you use ability to spin off functions into threadpools, then async wait for their return, at which point you often end up with the worst of both threads and async.
jiggawatts 2y ago
There are fundamental reasons for OS threads being slow, mostly to do with processor design. Changing the silicon would be hundreds of times more expensive than solving the problem in software in user mode. This is a billion-dollar solution to a hundred-billion dollar problem.
- Animats 2y ago
  There is a history of trying to support context switching in CPU hardware. That was a hot idea decades ago. Intel had call gates and some other stuff. Some of the RISC machines had hardware to spill all the registers to memory in background. Some early machines, from the days where CPUs were slower than memory, just had a context pointer in hardware. Change that and you're running a different thread. None of this helped all that much. Vanilla hardware won out. This is to some extent a consequence of C winning. C likes a single flat address space.
  - harry8 2y ago
    > a consequence of C winning. C likes a single flat address space. Interesting. What other language did C beat? How was their address space handling different?
    - Animats 2y ago
      C allows casting to (void *). That implies all pointers are in the same address space. Segmented architectures, where that's not true, have mostly died out. Burroughs used to build segmented address space machines where an address was a path, something like programname/scopename/variablename[subscript]. IBM's System 38 had an "object" architecture. Both worked fine, but were not very C-compatible.
      - harry8 2y ago
        Thanks.
  - yencabulator 2y ago
hinkley 2y ago
> More specifically, what if instead of spending 20 years developing various approaches to dealing with asynchronous IO (e.g. async/await), we had instead spent that time making OS threads more efficient, such that one wouldn't need asynchronous IO in the first place? This is still living in an antiquated world where IO was infrequent and contained enough that one blocking call per thread still made you reasonable forward progress. When you’re making three separate calls and correlating the data between them having the entire thread blocked for each call is still problematic. Linux can handle far more threads than Windows and it still employs io_uring. Why do you suppose that is? One little yellow box about it is not enough to defend the thesis of this article.
- nullindividual 2y ago
  Linux and Windows are limited by the commit limit when it comes to threads. Linux has a default thread stack size of 8Mb vs. NT of 1Mb, in theory meaning Linux will run out of allocation space much quicker. But in the end, both are limited by available memory.
_davide_ 2y ago
What about memory? the real price of threads is the stack. Even when perfectly optimized, it wouldn't be enough to handle serious workloads.
- kaba0 2y ago
  Well, the runtime (instead of the OS) knows better and can e.g. allocate part of the call stack on the heap itself, like how Java’s virtual threads do.
a-dub 2y ago
you should be able to reason synchronously and a good computer system would handle the rest.
- AndrewDucker 2y ago
  You cannot reason synchronously about things that are asynchronous. Unless you want to just block on IO. Which is clearly not great as a user experience or very efficient.
haileys 2y ago
Asynchronous IO isn't about efficiency. The approach the author takes with their language is just threads, but scheduled in userland. This model allows a decoupling of the performance characteristics of runtime threads from OS threads - which can sometimes be beneficial - but essentially, the programming model is fundamentally still synchronous. Asynchronous programming with async/await is about revealing the time dimension of execution as a first class concept. This allows more opportunities for composition. Take cancellation for example: cancelling tasks under the synchronous programming model requires passing a context object through every part of your code that might call down into an IO operation. This context object is checked for cancellation at each point a task might block, and checked when a blocking operation is interrupted. Timeouts are even trickier to do in this model, especially if your underlying IO only allows you to set per-operation timeouts and you're trying to expose a deadline-style interface instead. Under the asynchronous model, both timeouts and cancellation simply compose. You take a future representing the work you're doing, and spawn a new future that completes after sleeping for some duration, or spawn a new future that waits on a cancel channel. Then you just race these futures. Take whichever completes first and cancel the other. Having done a lot of programming under both paradigms, the synchronous model is so much more clunky and error-prone to work with and involves a lot of tedious manual work, like passing context objects around, that simply disappears under the asynchronous model.
- fulafel 2y ago
  I'd argue that few usages of async are motivated this way. In Rust land, it's efficiency and in JS land it's the browser scripting language legacy. > cancelling tasks under the synchronous programming model requires passing a context object through every part of your code that might call down into an IO operation. This is true for some but not all implementations. See eg Erlang or Unix processes (and maybe cancellation in pthreads?).
  - Yoric 2y ago
    To be more precise, in JS land, we introduced async not directly because of scripting but because of backwards compatibility - prior to async/Promise, the JS + DOM semantics were specified with a single thread of execution in mind, with complex dependencies in both directions (e.g. some DOM operations can cause sync reflow while handling an event, which is... bad) and run-to-completion. Promise made it easier to: - cut monolithic chunks of code-that-needs-to-be-executed-to-completion-before-updating-the-screen into something that didn't cause jank; - introduce background I/O. async/await made Promise more readable. (yes, that's for Promise and async/await on the browser, async callbacks have a different history on Node)
BenoitP 2y ago
> Now imagine a parallel universe where instead of focusing on making asynchronous IO work Funny choice of words. In the JVM world, Ron Pressler's first foray into fibers -quasar- was named "parallel universe". It worked with a java agent manipulating bytecode. Then Ron went to Oracle and now we have Loom, aka a virtual thread unmounted at each async IO request. Java's Loom is not even mentioned in the article. I wonder for a cofounder: does the "parallel universe" appear in a other foundational paper, calling for a lightweight thread abstraction? https://docs.paralleluniverse.co/quasar/ https://docs.paralleluniverse.co/quasar/ Anyway, yes we need sound abstractions for async IO
- junon 2y ago
  Quasar was awesome when it came out. Still remember trying to make it work when I was at Uber but it was really finicky.
- yas_hmaheshwari 2y ago
  1. I liked the way Java did not implement the async await style, and waited for the right abstraction with Project loom. 2. Though for a single threaded language like Javascript, I like the whole async await style because the alternative was worse (promises, callbacks ) Did not knew this history of how Project Loom came to be (though Quasar)
spullara 2y ago
Synchronous IO has always been more efficient. Anyone that thought otherwise doesn't understand how complicated context switches are in CPUs. The benefit of async io has always been handling tons of idle connections.
- wruza 2y ago
  Anyone that thought otherwise doesn't understand how complicated context switches are in CPUs That’s not true. I understand how context switches work down to tss records, but can’t immediately see why nonblock should be less efficient. Is it due to for-rw vs for-poll-rw? Doesn’t kqueue/iocp ought to solve that?
  - nullindividual 2y ago
    kqueue, like all non-io_uring implementations on Linux, are synchronous under the hood.
- dist1ll 2y ago
  Yielding in a cooperatively multitasked runtime is not comparable to OS context switches between user threads. The former is very, very efficient.
- anonymoushn 2y ago
  You can perform all the I/O you want with 0 context switches today in Linux. Why would performing more context switches (e.g. to call read(2)) speed things up?
p1necone 2y ago
Async/await is a language semantics thing. It's not really relevant whether there's a "real" OS thread under the hood, some language level green thread system, or just the current process blocking on something - the syntax exists because sometimes you don't want to block on things that take a long time semantically - I.e. you want the next line of code to run immediately. You could absolutely write a language where the blocking on long running tasks was implicit and instead there was a keyword for when you don't want to block, but the programmer doesn't really need to care about the underlying threading system.
- Animats 2y ago
  That's something like what Go does. Goroutines are "green threads" - they can be preempted. There's a CPU scheduler in user space. Go tries to provide "async" performance, and goroutines have minimal state. This seems to work well for the web server case. Pure "Async" means your application is now in the CPU dispatching business. This works well only if your application is totally I/O bound and has no substantial compute sections. Outside of that use case, it may be a huge mismatch to the problem. Worst case tends to be programs where almost all the time, something is fast, but sometimes it takes a lot of compute. Then all those quick async tasks get stuck behind the compute-bound operation. Web browsers struggle with that class of problems. Async I/O tends to be over-used today because Javascript works that way. Many programmers came up from Javascript land. That's the only way they can conceive concurrency. It's a simple, clean model - no need for locks. Threading is hard. Especially in languages that don't provide much help with locking. Even then, you have lock order problems, deadlocks, starvation, futex congestion...
  - Yoric 2y ago
    > It's a simple, clean model - no need for locks. Nit: You can very easily have race conditions in async JS. There are all sorts of Mutex-style structures for async.
    - ayewo 2y ago
      That's really interesting. Care to share a link to 1 or 2 real world examples of this that you've seen? Or even better, examples of how one would write such locks in JS that would be effective against these type of race conditions?
GolDDranks 2y ago
Then again, threads are a primitive API for concurrency. I hope that structured concurrency becomes more common. How they are inplemented should be more or less an implementation detail. https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ https://vorpus.org/blog/notes-on-structured-concurrency-or-g...
HippoBaro 2y ago
I am not sure I buy the underlying idea behind this piece, that somehow a lot of money/time has been invested into asynchronous IO at the expense of thread performance (creation time, context switch time, scheduler efficiency, etc.). First, significant work has been done in the kernel in that area simply because any gains there massively impact application performance and energy efficiency, two things the big kernel sponsors deeply care about. Second, asynchronous IO in the kernel has actually been underinvested for years. Async disk IO did not exist at all for years until AIO came to be. And even that was a half-backed, awful API no one wanted to use except for some database people who needed it badly enough to be willing to put up with it. It's a somewhat recent development that really fast, genuinely async IO has taken center stage through io_uring and the likes of AF_XDP.
- mmis1000 2y ago
  Make os thread runs more efficient is like `faking async IOs (disk/network/whatever goes out from the computer shell) into the sync operations in a more efficient way`. But why would you do it at first place if the program can handle async operations at first place? Just let userland program do their business would be a better decision though.
deleted 2y ago
[deleted]
ruduhudi 2y ago
With async IO you can do stuff concurrently without doing it in parallel which is a desirable thing for many workloads. With only threads you‘d need sync primitives all over the place.
winternewt 2y ago
> Now imagine a parallel universe where instead of focusing on making asynchronous IO work, we focused on improving the performance of OS threads such that one can easily use hundreds of thousands of OS threads without negatively impacting performance I actually can't imagine how that would ever be accomplished at the OS level. The fact that each thread needs its own stack is an inherent limiter for efficiency, as switching stacks leads to cache misses. Asynchronous I/O has an edge because it only stores exactly as much state as it needs for its continuation, and multiple tasks can have their state in the same CPU cache line. The OS doesn't know nearly enough about your program to optimize the stack contents to only contain the state you need for the remainder of the thread. But at the programming language level the compiler does have insight into the dependencies of your continuation, so it can build a closure that has only what it needs to have. You still have asynchronous I/O at the core but the language creates an abstraction that behaves like a synchronous threaded model, as seen in C#, Kotlin, etc. This doesn't come without challenges. For example, in Kotlin the debugger is unable to show contents of variables that are not needed further down in the code because they have already been removed from the underlying closure. But I'm sure they are solvable.
- dist1ll 2y ago
  > But at the programming language level the compiler does have insight into the dependencies of your continuation This is really the key point - coupled with the fact that certain I/O operations are just inherently asynchronous. The TX/RX queues in NICs are an async, message passing interface - regardless of whether you're polling descriptors or receiving completion interrupts. So really, async I/O is the natural abstraction for networking.
  - dietr1ch 2y ago
    Anything that happens far enough from the CPU is async, and here far probably means 10cm thanks to the speed of light not being fast enough, even in vacuum. So, any computation that spans a machine the size of our hands needs async unless you are willing to drop the clocks to push "far" a bit further away (and bring in power, heat and noise with it).
josefrichter 2y ago
Isn’t that parallel universe actually the Erlang BEAM?
- BenoitP 2y ago
  Or Go. Or Java' virtual threads. It must happen at the language level; When it comes to execution context knowledge: what context to compile out (stackless), or what context to serialize to the heap (stackful). Programming language will always know much more about the program than the OS. If I'm not mistaken in Erlang the programmer will provide the exact context to serialize : the actor.
  - josefrichter 2y ago
    I mentioned the BEAM as “universe” especially because it provides so much more than Go or Java in this respect.
  - yencabulator 2y ago
    This abstraction that a heap is somehow different than stack is hurting understanding; they're just regions of memory. Go stacks are allocated areas of memory like anything else. When a green thread is suspended, it's stack is not "serialized to the heap". Switching green threads is mostly a question of setting the stack pointer to point to the new green thread's stack. What makes threads "green" is that they are not supposed to call blocking syscalls (they can queue work in a separate blocking threadpool and call the scheduler to suspend themselves). Go has a signal-based mechanism for non-cooperative scheduling of green threads, so green threads are not even required to cooperatively yield to the scheduler.
stuaxo 2y ago
20 years ago we had the world of threads and it was very easy to get into a mess.
torginus 2y ago
Asynchronous IO is just simply how the world works. Instead, the idea that changes happen only during CPU computation is the mistake. Your disk drive/network card exists in parallel to your CPU and can process stuff concurrently. Your CPU very likely has a DMA engine that works in parallel without consuming a hardware thread.
- yencabulator 2y ago
  The "world" these days works by ringbuffers and interrupts, neither of which looks like async/await. Async/await is a chosen abstraction over what hardware/kernel interfaces really look like.
  - torginus 2y ago
    Well, async/await in languages is just promises/futures made to look like regular flow control via state machines. And promises are controlled via callbacks, which are called from some event. It's essentially the same thing as when you get an interrupt from your microcontroller's DMA engine that the ring buffer needs to be refilled, just with 2 layers of abstractions on top to make it look nicer.
    - yencabulator 2y ago
      So you agree this to be wrong, then.. > Asynchronous IO is just simply how the world works. How the world works looks more like message passing. What abstraction you construct on top of that is a choice. Async/await is not necessarily the global maxima.
jeffreygoesto 2y ago
Some answer was already posted here: https://utcc.utoronto.ca/~cks/space/blog/tech/OSThreadsAlwaysExpensive https://utcc.utoronto.ca/~cks/space/blog/tech/OSThreadsAlway... https://news.ycombinator.com/item?id=41472027 https://news.ycombinator.com/item?id=41472027 To me the article reads as if the programming language author wants to push a difficult problem out of his language without deeper analysis. As if it would be easier if it was somebody else's problem.
csomar 2y ago
> Now imagine a parallel universe where instead of focusing on making asynchronous IO work, we focused on improving the performance of OS threads such that one can easily use hundreds of thousands of OS threads without negatively impacting performance Isn't that why async I/O was created in the first place? > Just use 100 000 threads and let the OS handle it. How does the OS handle it? How does the OS know whether to give it CPU time or not? I was expecting something from the OP (like a new networking or multi-threading primitive) but I have a feeling he lacks an understanding of how networking and async I/O works.
pizza 2y ago
Correct me if I'm wrong, but Microsoft's DirectStorage seems to me something like what the author is writing about. It lets you do eg massively parallel NVME file io ops from the GPU itself of lots of small files. This avoids the delay of the path through the CPU, any extra threads/saturation of the CPU, and even lets you do eg decompression of game assets on the GPU itself thereby saving even more CPU. This demo benchmark shows DEFLATE going from 1 GB/s on CPU to 7 GBs/ on GPU https://github.com/microsoft/DirectStorage/tree/main/Samples/GpuDecompressionBenchmark https://github.com/microsoft/DirectStorage/tree/main/Samples...
- dudeinjapan 2y ago
  I interpreted it as mainly network I/O, but the core point in the article is less about the I/O itself and more about thread-based async I/O handling.
  - pizza 2y ago
    I see, I had this notion that DirectStorage worked by somehow allowing the GPU cores to host multiple parallel threads each doing full requests on their own, but on further research I was mistaken - requests are still CPU-submitted
    - compressedgas 2y ago
      I recently sought to check if DirectStorage used PCIe P2P to transfer data from the SSD to the GPU memory. It definitely could use it but I didn't see any evidence that it did.
alexgartrell 2y ago
> File IO is perhaps the best example of this (at least on Linux). To handle such cases, languages must provide some sort of alternative strategy such as performing the work in a dedicated pool of OS threads. AIO has existed for a long time. A lot longer than io_uring. I think the thing that the author misses here is that the majority of IO that happens is actually interrupt driven in the first place, so async io is always going to be the more efficient approach. The author also misses that scheduling threads efficiently from a kernel context is really hard. Async io also confers a benefit in terms of “data scheduling.” This is more relevant for workloads like memcached.
Someone 2y ago
FTA: “Need to call a C function that may block the calling thread? Just run it on a separate thread, instead of having to rely on some sort of mechanism provided by the IO runtime/language to deal with blocking C function calls.” And then? How do you know when your call completed without “some sort of mechanism provided by the IO runtime/language”? Yes, you periodically ask the OS whether that thread completed, but that doesn’t come for free and is far from elegant. There are solutions. The cheapest, resource-wise, are I/O completion callbacks. That’s what ”System” had on the original Mac in 1984, and there likely were even smaller systems before that had them. Easier for programmers would be something like what we now have with async/await. It might not be the best option, but AFAICT, this article doesn’t propose a better one. Yes, firing off threads is easy, but getting the parts together the moment they’re all available isn’t.
sedatk 2y ago
Many synchronous I/O operations under the hood are just async I/O + blocking waits, at least that's the case with Windows. Why? Because all I/O is inherently async. Even polling I/O requires timed waits which also makes it async. That said, I like async programming model in general, not just for I/O. It makes modeling your software as separetely flowing operations that need to be synchronized occasionally quite easy. Some tasks need to run in parallel? Then, you just wait for them later. I also like the channel concept of Golang and D in the same manner, but I heard it brought up some problems that async/await model didn't have. Can't remember what it was now. Maybe they are more susceptible to race conditions? Not sure.
snickerbockers 2y ago
This is extremely myopic. there is not a 1:1 correspondence between using asynchronous io and using one thread per file. Asynchronous io lets you dodge thread safety mechanisms like semaphores. Not all of us are trying to write a webapp or whatever, some of us just need to load a lot of data from several descriptors without serializing all the blocking operations. >Not every IO operation can be performed asynchronously though. File IO is perhaps the best example of this (at least on Linux). To handle such cases, languages must provide some sort of alternative strategy such as performing the work in a dedicated pool of OS threads. Uhhh this is just wrong file io can definitely be done asynchronously, on Linux, and without language support.
dwaite 2y ago
I don't quite agree with this piece, as it is comparing apples and oranges. What you want is patterns for having safety, efficiency and maintainability for concurrent and parallelized processing. One early pattern for doing that was codified as POSIX threads - continue the blocking processing patterns of POSIX so that you can have multiple parallelizable streams of execution with primitives to protect against simultaneous use of shared resources and data. IO_URING is not such a pattern. It is a kernel API. You can try to use it directly, but you can also use it as one component in a userland thread systems, in actor systems, in structured concurrency systems, etc. So the author is seemingly comparing the shipped pattern (threads) vs direct manipulation, and complaining that the direct manipulation isn't as safe or maintainable. It wasn't meant to be.
anonymoushn 2y ago
Do Mac and Windows really fail to expose some file I/O operations with kqueue and IOCP?
anonymoushn 2y ago
It's not clear that context switches can be made sufficiently cheap on fast CPUs without disabling mitigations for side-channel attacks. So the idea of making OS threads comparably performant to goroutines, rust async, or any implementation of cooperative multithreading seems impractical.
lima 2y ago
Google's SwitchTo kernel patches are similar to what you describe: https://lore.kernel.org/lkml/20200722234538.166697-1-posk@posk.io/ https://lore.kernel.org/lkml/20200722234538.166697-1-posk@po... Slides: https://web.archive.org/web/20200802205544/https://pdxplumbers.osuosl.org/2013/ocw//system/presentations/1653/original/LPC%20-%20User%20Threading.pdf https://web.archive.org/web/20200802205544/https://pdxplumbe...
weinzierl 2y ago
"Not every IO operation can be performed asynchronously though. File IO is perhaps the best example of this (at least on Linux). To handle such cases, languages must provide some sort of alternative strategy such as performing the work in a dedicated pool of OS threads." Can someone explain, why this would be the case? - Why can't every IO op be async? - Why is file IO on Linux not async? - What does iouring have to do with it?
- seanhunter 2y ago
  File IO can absolutely be async on linux[1] and this has been supported since version 2.5 or something provided that the device supports it (which they all have since about 2000). io_uring was the syscall interface that was introduced in 5.1 to improve async file i/o performance so it is relevant in that the previous way to do it wasn't great. [1] https://kkourt.io/blog/2017/10-14-linux-aio.html https://kkourt.io/blog/2017/10-14-linux-aio.html <- note the 2017 date.
- yencabulator 2y ago
  In-kernel file or block I/O is async if the driver/file implemented the async operations. If not, it's blocking. io_uring is a relatively new userspace syscall API to minimize the number of syscalls used for I/O, and to provide a common mechanism for doing many kinds of operations. Previously, large numbers of syscalls could become a bottleneck for busy applications, and some kinds of operations did not have a usable non-blocking interface.
sph 2y ago
Async IO is hilarious in a universe where Hewitt's actors or even Hoare's CSP exist. They are a subpar technique that works well with languages with semantics from the 1970s that do not have communication primitives, in the age of multicore and the Internet. The saddest thing is the most hyped language of the decade went all in with this miserable idea, and turned me completely off the ecosystem.
- fire_lake 2y ago
  How is CSP different from async/await? One is a semantics and the other is a syntax, no?
- wruza 2y ago
  What problems do you have with async io? I try to imagine and can’t see how it is bad. It’s completely transparent and non-problematic to me. Also, what’s the most hyped lang of the decade? Go, Kotlin?
- Paradigma11 2y ago
  I think CSP and Actors are overkill for nearly all programs. A simpler async system can do the job and introduce much less cognitive overhead. If really necessary there are still special libraries in most ecosystems like RX, Akka, Zio, Hopac, lwt......
praptak 2y ago
There's an effort by Google to address this with userspace threads. I think it has stalled though. HN discussion thereof: https://news.ycombinator.com/item?id=23964633 https://news.ycombinator.com/item?id=23964633
nasretdinov 2y ago
Somewhat controversial take: the current threads implementation is usually already performant enough for most use cases. The actual reason why we don't use them to handle more than a few thousand concurrent operations is that, at least in Linux, threads are scheduled and treated very similarly to processes. E.g. if a single process with 3000 threads gets bottlenecked on some syscall, etc, your system load average will become 3000, and it will essentially lead to no other processes being able to run well on the same machine. Another issue with threads performance is that they are visible to most system tools like `ps`, and thus having too mamy threads starts to affect operations _outside_ the kernel, e.g. many monitoring tools, etc. So that's the main reason why user-space scheduling became so popular: it hides the "threads" from the system, allowing for processes to be scheduled more fairly (preventing stuff like reaching LA 3000 when writing to 3000 parallel connections), and not affecting performance of the system infrastructure around the kernel. BTW the threads stacks, as well as everything else in Linux are allocated lazily, so if you only use like 4Kb of stack in the thread it wouldn't lead to RSS of full 8M. It will contribute to VMEM, but not RSS
- viraptor 2y ago
  > your system load average will become 3000, and it will essentially lead to no other processes being able to run well on the same machine. That doesn't sound right. I mean, there are many schedulers available, but I was under the impression that most have a separate blocked queue. (Or more specifically, anything blocked will not be in the runnable pool - those will be dequeued) I.e. anything waiting on a syscall will be mostly ignited. (Please correct me if I'm misunderstanding the usual behaviour here)
  - nasretdinov 2y ago
    Yeah I think you are right, it depends on the syscalls, etc, in question. When that happened I didn't check which exact syscalls were being used
- mike_hearn 2y ago
  Linux ps doesn't show threads (by default). That was true decades ago before NPTL but hasn't been the case for a long time. The reason not to use kernel threads for everything is RAM utilization.
throwaway81523 2y ago
File io can now be done asynchronously with io_uring.
seanhunter 2y ago
> "Not only would this offer an easier mental model for developers..." Translation: "I find async i/o confusing and all developers are like me". This argument has been going on for over 20 years at this point. There are some people who think having pools of threads polling is a natural way of thinking about IO. They keep waiting for the day this becomes an efficient way to do IO.
- tankenmate 2y ago
  I would concur, the author also mentions that file IO isn't async; this makes me conclude that the author hasn't grasped how much the Linux kernel has moved on since in the mid 2000s. I suspect that the author has an incomplete view of the current kernel / userland interface as well as the inner workings of how a kernel actually "does what it does".
  - baq 2y ago
    > the author also mentions that file IO isn't async And the best part is that NT kernel was always async… Linux is actually kind of special in that it was primarily sync.
    - tankenmate 2y ago
      The Linux kernel isn't "primarily" sync, the kernel itself has been almost entirely async except for some small parts, like the bottom half of interrupt handlers which can't be pre-empted. Even in the early days of Linux the scheduler was pre-emptive and used "wait channels" create a synchronous interface for the asynchronous kernel. In the early days of Linux, if a process was in kernel mode then it couldn't be pre-empted until it returned to user mode, but this is no longer the case, and with PREEMPT_RT this is even the case for RT processes /kernel tasks. Now, the POSIX API is largely synchronous, but this is mainly because of the history of UNIX (and Multics before it).
  - YorickPeterse 2y ago
    That particular paragraph can be phrased better so I'll adjust that. What I meant to say is that you can't handle it asynchronously like you can with sockets, i.e. polling it for readiness. This is because for file IO, reads and writes are always reported as being available, making epoll/kqueue/etc effectively useless.
pyrolistical 2y ago
What if the author’s proposed solution is the billion dollar mistake? IMO the best programming paradigms are when the abstractions are close to the hardware. Instead of pretending to have unlimited cores, what if as part of the runtime of we are given the exactly one thread per core. As the programmer we are responsible for utilizing all the cores and passing data around. It is then up to the operating system to switch entire sets of cores over different processes. This removes the footgun of a process overloading a computer with too many threads. Programmers need to consider how to best distribute work over a finite number of cores.
mmis1000 2y ago
I think compare programing pattern(threads) to kernel async apis is a questionable comparation. The point of kernel async apis is not about letting programmers write system calls directly. It's about expose the actual async operations under the hook (it could be disk, be network, be anything outside of the computer case). Those actions are never mean to be interleaved with cpu computation, because they are usually with ms level delay (which could be millions of cpu ticks). The kernel fakes these into sync calls by pause everything. But it isn't always the best idea to do these. Let userland program decide what they want to do with the delay will be a way better idea. Even they eventually just invent blocking io calls again. They can still decide what operations are more relevant to itself instead of let the kernel guessing it.
dudeinjapan 2y ago
This article is a billion dollar mistake.
necovek 2y ago
To put a different spin on what others are saying, asynchronous IO is a different programming model for concurrency that's actually more ergonomic and easier to get right for an average developer (which includes great developers on their not-highly-focused days). Dealing with raciness, deadlocks and starvation is simply hard, especially when you are focused on solving a different but also hard business problem. That's also why RDBMSes had and continue to have such a success: they hide this complexity behind a few common patterns and a simple language. Now, I do agree that languages that suffer from the "color of your functions" problems didn't get it right (Python, for instance). But ultimately, this is an easier mental model, and it's been present since the dawn of purely functional languages (nothing stops a Lisp implementation from doing async IO, and it might only be non-obvious how to do "cancellation" while "gather" is natural too)
ivanjermakov 2y ago
Async IO is not only about creating sockets and spawning threads. Idea of async IO is that the world is not controlled by your CPU. There are network, storage, sound devices that might and will take time to produce the result and the CPU has to wait for it. I feel like there is a big misunderstanding about what async IO is and what problem it solves.
- scotty79 2y ago
  Doesn't message passing capture this essence better?
  - ivanjermakov 2y ago
    Message passing, channels, thread pool, callbacks, polling - there are many ways to implement async IO and async computation in general.
dfgdfg34545456 2y ago
The post seems to be assuming that multi threaded code is easy to build and maintain. From my experience it is horrible, every new thread means going from n bugs to nn bugs. As a programmer I prefer* async constructs in languages, and do not want to spin up and manage threads and all the state synchronisation that involves.
algobro 2y ago
The 1990s called. They want their threads vs events debates back. Todays processors are fast enough to serve many useful workloads with a single core. The benefit of the async abstraction outweighs the performance benefit in the majority of the cases. And debugging multithreaded code is way harder than async code, mainly if its the kind of program that needs stepping into.
deleted 2y ago
[deleted]
bob1029 2y ago
> the cost to start threads is lower, context switches are cheaper, etc. Physics would have a word with this one. We are already pushing limits of what is possible with latency between cores vs overall system performance. There isn't an order of magnitude improvement hiding in there anywhere without some FTL communication breakthrough. In theory, yes we could sweep this problem under the rug of magical, almost-free threads. But these don't really exist. I think the best case for performance is to accumulate mini batches of pending IO somewhere and then handle them all at once. IO models based upon ring buffers are probably getting close to the theoretical ideal when considering how our CPUs work internally (cache coherency, pipelining, etc).
- YorickPeterse 2y ago
  Is this actually proven though? I've seen other people argue that threads are already as fast as they can be, yet nobody is able to actually substantiate that claim.
neonsunset 2y ago
No, it isn’t, the author is just confused, as it usually is. Worked great in C# since its introduction for task interleaving, composition, cancellation (worse languages call it structured concurrency) and switching to privileged context (UI thread), and will work even better in .NET 10.
forty 2y ago
Maybe just use Nodejs? Everything is async by default and the API is not annoying or difficult to use
Szpadel 2y ago
I think async vs threads is about completely different trade-off. Nowadays all operating systems so preemptive scheduling, but green threads (and all async by the extent) use cooperative scheduling. I believe most of discussion here is actually about pros and cons of those scheduling models. one exception is I think cancellation model, but I'm only aware about rust that does it that way, all other runtimes will happily run your green thread until it finishes or cancel by itself similarly that you do with synchronous code.
scarnie 2y ago
The author appears to contradict the very issue they argue, by presenting languages such as Go, Erlang or their own toy language. These languages hide the async / await constructs that are present in languages like Rust, Swift or Typescript. The former languages and runtimes have no function colouring problems, when working within their own SDKs. There are trade offs, and these “async” languages tend to be a bit more awkward when interacting with OS frameworks.
chucke 2y ago
My interpretation of what the author wants, is essentially lightweight threads in the kernel, standardised a lá POSIX , that every proglang could use as a primitive. That'd be sweet if this were a well understood problem. Unfortunately, we're still finding the sweet spot between I/O Cs CPU bound tasks, "everything is a file" clashing with async network APIs and mostly sync file APIs, and sending that research to the kernel would mean having improvements widely distributed in 5 years or more, and would set back the industry decades, if not centuries. We learned this much already with the history of TCP and the decision of keeping QUIC in userspace.
Brian_K_White 2y ago
Take the title at face value. If it were a sincere question the sincere answer is simply no. The world actually is concurrent and asynchronous regardless how inconvenient that is for a programmer.
rurban 2y ago
On the contrary blocking IO (and POSIX) is the old billion dollar mistake. It's inefficient and blocks concurrency safety.
orf 2y ago
Computers are inherently asynchronous, we just plaster over that with synchronous interfaces. We put a lot of effort into maintaining these synchronous facades - from superscalar CPUs translating assembly instructions into “actual” instructions and speculatively executing them in parallel to prevent stalls, to the kernel with preemptive scheduling, threads and their IO interfaces, right up to user-space and the APIs they provide on top of all this. Surely there has to be a better way? It seems ridiculous.
- yencabulator 2y ago
  https://en.wikipedia.org/wiki/Dataflow_architecture https://en.wikipedia.org/wiki/Dataflow_architecture https://stackoverflow.com/questions/530180/what-happened-to-clockless-computer-chips https://stackoverflow.com/questions/530180/what-happened-to-...
nurettin 2y ago
In several projects I switched from async code to threads and mpsc queues. Timers, data streams, external communications all run in their threads. They pass their messages to the main thread's queue. The entire thing suddenly became much easier to reason about and read.
OutOfHere 2y ago
> Languages such as Go and Erlang bake support for asynchronous IO directly into the language, while others such as Rust rely on third-party libraries such as Tokio. This is so wrong. Go and Erlang have message passing, not async. Message passing is its own thing; it should not be mixed with threading or async.
- jclulow 2y ago
  Go offers channels in the standard library, which you can use to do message passing, certainly (so does Rust). But I/O through the stdlib is absolutely asynchronous. I/O routines that would block, like sending and receiving from the network, are all magic async switch points where your goroutine can be replaced on the current OS thread with another. When epoll (or whatever) signals readiness on the blocked descriptor you get switched back on. As described in the previous post, the asynchronous I/O is hidden from you as an implementation detail of the language but it's absolutely there.
  - OutOfHere 2y ago
    That's in practically every nontrivial language.
quercus 2y ago
I thought this article was going to talk about async IO's potential to cause insidious bugs that you don't notice until they're crashing jet planes. That's definitely a topic worth of a blog post.
lincpa 2y ago
[dead]

157 points by signa11 2y ago

nullindividual 2y ago

We do live in the universe of high performance threads with asynchronous I/O. The author is looking for Windows NT.

wtallis 2y ago
The same Windows NT whose decades-old async IO capabilities are so good they immediately cloned io_uring?
- lmz 2y ago
  Sorry, which one came first between io_uring and windows RIO? https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/hh997032(v=ws.11) https://learn.microsoft.com/en-us/previous-versions/windows/...
  - wtallis 2y ago
    I think you missed the point: io_uring is the newer API. Microsoft saw the new async IO API developed for Linux, and made a version for Windows. If what Windows has had all along is so great, why did they bother cloning io_uring when io_uring came along? It definitely isn't because there's any significant body of software relying on io_uring yet. My theory: people like to extol the virtues of NT's IOCP as "checking the box" for being async, but seldom bother to assess whether the decades-old async API actually offers good performance in comparison to more recent alternatives. But some parts of Microsoft understand quite well that they have pervasive issues with IO performance: replacing WSL1 with WSL2, introducing DirectStorage and a partial copy of io_uring, the Dev Drive feature in Windows 11—all changes that are driven by the understanding that the status quo isn't all that great.
    - nullindividual 2y ago
      > I think you missed the point: io_uring is the newer API. GP was pointing out that io_uring came after RIO; RIO has similar capabilities to io_uring for the network space. GP did not "miss the point". > Microsoft understand quite well that they have pervasive issues with IO performance This has nothing to do with IOCP but rather the file system filters. This is why removal of file system filters from "DevDrive" (a ReFS volume) improves performance vs a ReFS volume with file system filters in place. > If what Windows has had all along is so great, why did they bother cloning io_uring when io_uring came along? IoRing eliminates the need to copy buffers between kernel and user space [0][1]. With regards to performance, this is a bench between a traditional multithreaded UDP server and an RIO IOCP UDP server. The gains are significant. [2] And further comparisons... [3] > seldom bother to assess whether the decades-old async API actually offers good performance in comparison to more recent alternatives There are no complete alternatives [r]. Nor is Linux fully asynchronous throughout the kernel. > introducing DirectStorage Not really relevant to the context. DirectStorage is a method to deliver data from nVME storage with minimal CPU overhead and/or compressed [4]. Such a system would improve performance on any OS and indeed, we can see that on PS OS (PS5, FreeBSD-based (probably)). Again, limited in scope. I'm honestly not sure what you're attempting to argue. IoRing brings improvements, yes, but isn't a replacement of IOCP. We should see improvements in any mainstream kernel as a good thing. It's progress. And hopefully the Linux kernel gets to full async I/O at some point in the future. [0] https://windows-internals.com/i-o-rings-when-one-i-o-operation-is-not-enough/ https://windows-internals.com/i-o-rings-when-one-i-o-operati... (this is from a co-author of Windows Internals) [1] https://windows-internals.com/ioring-vs-io_uring-a-comparison-of-windows-and-linux-implementations/ https://windows-internals.com/ioring-vs-io_uring-a-compariso... [2] https://serverframework.com/asynchronousevents/2012/08/winsock-registered-io-io-completion-port-performance.html https://serverframework.com/asynchronousevents/2012/08/winso... [3] https://speakerdeck.com/trent/pyparallel-how-we-removed-the-gil-and-exploited-all-cores https://speakerdeck.com/trent/pyparallel-how-we-removed-the-... [4] https://stackoverflow.com/questions/73878597/what-are-the-differences-between-windows-overlapped-io-registered-io-rio-and https://stackoverflow.com/questions/73878597/what-are-the-di... [5] https://learn.microsoft.com/en-us/gaming/gdk/_content/gc/system/overviews/directstorage/directstorage-overview https://learn.microsoft.com/en-us/gaming/gdk/_content/gc/sys...
- nullindividual 2y ago
  Yes, it is nearly a 1:1 copy of io_uring, but unlike io_uring which applies to a single function, all I/O in the NT kernel is asynchronous. IOCP acts on file, network, mail slot, pipes, etc. IoRing/io_uring is for files only. RegisteredIO is network only. While Windows userland itself may be a 'mess' and archaic in it's own way (among the other anti-consumer bits), the NT kernel is technically quite advanced, not only for it's time, but at the present time. David Cutler's team got the kernel architecture correct the first time. Linux still has a ways to catch up in certain aspects.
  - lmz 2y ago
    A free sample chapter from Windows Internals about the I/O system for the curious: https://www.microsoftpressstore.com/articles/article.aspx?p=2201309&seqNum=3 https://www.microsoftpressstore.com/articles/article.aspx?p=...
    - nullindividual 2y ago
      Thanks, but I've been referring people like you who lack an understanding of Windows I/O (and VMM) to this book since at least the 4th edition. I've had my copy of the 7th since it came out.
      - lmz 2y ago
        It wasn't for your benefit somce you obviously are familiar with the topic, but for passers-by who may be interested in the topic.

deleted 2y ago

[deleted]

NinoScript 2y ago

Interesting take. I’d like to see such an OS, I wonder what requirements/limitations it would have.

CJefferson 2y ago

I generally agree with this article. There are programs where async IO is great, but in my experience it stops being useful as your code “does more stuff”. The few large scale async systems I’ve worked with end up with functions taking too long, so you use ability to spin off functions into threadpools, then async wait for their return, at which point you often end up with the worst of both threads and async.

jiggawatts 2y ago

There are fundamental reasons for OS threads being slow, mostly to do with processor design. Changing the silicon would be hundreds of times more expensive than solving the problem in software in user mode. This is a billion-dollar solution to a hundred-billion dollar problem.

Animats 2y ago
There is a history of trying to support context switching in CPU hardware. That was a hot idea decades ago. Intel had call gates and some other stuff. Some of the RISC machines had hardware to spill all the registers to memory in background. Some early machines, from the days where CPUs were slower than memory, just had a context pointer in hardware. Change that and you're running a different thread. None of this helped all that much. Vanilla hardware won out. This is to some extent a consequence of C winning. C likes a single flat address space.
- harry8 2y ago
  > a consequence of C winning. C likes a single flat address space. Interesting. What other language did C beat? How was their address space handling different?
  - Animats 2y ago
    C allows casting to (void *). That implies all pointers are in the same address space. Segmented architectures, where that's not true, have mostly died out. Burroughs used to build segmented address space machines where an address was a path, something like programname/scopename/variablename[subscript]. IBM's System 38 had an "object" architecture. Both worked fine, but were not very C-compatible.
    - harry8 2y ago
      Thanks.
- yencabulator 2y ago

hinkley 2y ago

> More specifically, what if instead of spending 20 years developing various approaches to dealing with asynchronous IO (e.g. async/await), we had instead spent that time making OS threads more efficient, such that one wouldn't need asynchronous IO in the first place? This is still living in an antiquated world where IO was infrequent and contained enough that one blocking call per thread still made you reasonable forward progress. When you’re making three separate calls and correlating the data between them having the entire thread blocked for each call is still problematic. Linux can handle far more threads than Windows and it still employs io_uring. Why do you suppose that is? One little yellow box about it is not enough to defend the thesis of this article.

nullindividual 2y ago
Linux and Windows are limited by the commit limit when it comes to threads. Linux has a default thread stack size of 8Mb vs. NT of 1Mb, in theory meaning Linux will run out of allocation space much quicker. But in the end, both are limited by available memory.

_davide_ 2y ago

What about memory? the real price of threads is the stack. Even when perfectly optimized, it wouldn't be enough to handle serious workloads.

kaba0 2y ago
Well, the runtime (instead of the OS) knows better and can e.g. allocate part of the call stack on the heap itself, like how Java’s virtual threads do.

a-dub 2y ago

you should be able to reason synchronously and a good computer system would handle the rest.

AndrewDucker 2y ago
You cannot reason synchronously about things that are asynchronous. Unless you want to just block on IO. Which is clearly not great as a user experience or very efficient.

haileys 2y ago

Asynchronous IO isn't about efficiency. The approach the author takes with their language is just threads, but scheduled in userland. This model allows a decoupling of the performance characteristics of runtime threads from OS threads - which can sometimes be beneficial - but essentially, the programming model is fundamentally still synchronous. Asynchronous programming with async/await is about revealing the time dimension of execution as a first class concept. This allows more opportunities for composition. Take cancellation for example: cancelling tasks under the synchronous programming model requires passing a context object through every part of your code that might call down into an IO operation. This context object is checked for cancellation at each point a task might block, and checked when a blocking operation is interrupted. Timeouts are even trickier to do in this model, especially if your underlying IO only allows you to set per-operation timeouts and you're trying to expose a deadline-style interface instead. Under the asynchronous model, both timeouts and cancellation simply compose. You take a future representing the work you're doing, and spawn a new future that completes after sleeping for some duration, or spawn a new future that waits on a cancel channel. Then you just race these futures. Take whichever completes first and cancel the other. Having done a lot of programming under both paradigms, the synchronous model is so much more clunky and error-prone to work with and involves a lot of tedious manual work, like passing context objects around, that simply disappears under the asynchronous model.

fulafel 2y ago
I'd argue that few usages of async are motivated this way. In Rust land, it's efficiency and in JS land it's the browser scripting language legacy. > cancelling tasks under the synchronous programming model requires passing a context object through every part of your code that might call down into an IO operation. This is true for some but not all implementations. See eg Erlang or Unix processes (and maybe cancellation in pthreads?).
- Yoric 2y ago
  To be more precise, in JS land, we introduced async not directly because of scripting but because of backwards compatibility - prior to async/Promise, the JS + DOM semantics were specified with a single thread of execution in mind, with complex dependencies in both directions (e.g. some DOM operations can cause sync reflow while handling an event, which is... bad) and run-to-completion. Promise made it easier to: - cut monolithic chunks of code-that-needs-to-be-executed-to-completion-before-updating-the-screen into something that didn't cause jank; - introduce background I/O. async/await made Promise more readable. (yes, that's for Promise and async/await on the browser, async callbacks have a different history on Node)

BenoitP 2y ago

> Now imagine a parallel universe where instead of focusing on making asynchronous IO work Funny choice of words. In the JVM world, Ron Pressler's first foray into fibers -quasar- was named "parallel universe". It worked with a java agent manipulating bytecode. Then Ron went to Oracle and now we have Loom, aka a virtual thread unmounted at each async IO request. Java's Loom is not even mentioned in the article. I wonder for a cofounder: does the "parallel universe" appear in a other foundational paper, calling for a lightweight thread abstraction? https://docs.paralleluniverse.co/quasar/ https://docs.paralleluniverse.co/quasar/ Anyway, yes we need sound abstractions for async IO

junon 2y ago
Quasar was awesome when it came out. Still remember trying to make it work when I was at Uber but it was really finicky.
yas_hmaheshwari 2y ago
1. I liked the way Java did not implement the async await style, and waited for the right abstraction with Project loom. 2. Though for a single threaded language like Javascript, I like the whole async await style because the alternative was worse (promises, callbacks ) Did not knew this history of how Project Loom came to be (though Quasar)

spullara 2y ago

Synchronous IO has always been more efficient. Anyone that thought otherwise doesn't understand how complicated context switches are in CPUs. The benefit of async io has always been handling tons of idle connections.

wruza 2y ago
Anyone that thought otherwise doesn't understand how complicated context switches are in CPUs That’s not true. I understand how context switches work down to tss records, but can’t immediately see why nonblock should be less efficient. Is it due to for-rw vs for-poll-rw? Doesn’t kqueue/iocp ought to solve that?
- nullindividual 2y ago
  kqueue, like all non-io_uring implementations on Linux, are synchronous under the hood.
dist1ll 2y ago
Yielding in a cooperatively multitasked runtime is not comparable to OS context switches between user threads. The former is very, very efficient.
anonymoushn 2y ago
You can perform all the I/O you want with 0 context switches today in Linux. Why would performing more context switches (e.g. to call read(2)) speed things up?

p1necone 2y ago

Async/await is a language semantics thing. It's not really relevant whether there's a "real" OS thread under the hood, some language level green thread system, or just the current process blocking on something - the syntax exists because sometimes you don't want to block on things that take a long time semantically - I.e. you want the next line of code to run immediately. You could absolutely write a language where the blocking on long running tasks was implicit and instead there was a keyword for when you don't want to block, but the programmer doesn't really need to care about the underlying threading system.

Animats 2y ago
That's something like what Go does. Goroutines are "green threads" - they can be preempted. There's a CPU scheduler in user space. Go tries to provide "async" performance, and goroutines have minimal state. This seems to work well for the web server case. Pure "Async" means your application is now in the CPU dispatching business. This works well only if your application is totally I/O bound and has no substantial compute sections. Outside of that use case, it may be a huge mismatch to the problem. Worst case tends to be programs where almost all the time, something is fast, but sometimes it takes a lot of compute. Then all those quick async tasks get stuck behind the compute-bound operation. Web browsers struggle with that class of problems. Async I/O tends to be over-used today because Javascript works that way. Many programmers came up from Javascript land. That's the only way they can conceive concurrency. It's a simple, clean model - no need for locks. Threading is hard. Especially in languages that don't provide much help with locking. Even then, you have lock order problems, deadlocks, starvation, futex congestion...
- Yoric 2y ago
  > It's a simple, clean model - no need for locks. Nit: You can very easily have race conditions in async JS. There are all sorts of Mutex-style structures for async.
  - ayewo 2y ago
    That's really interesting. Care to share a link to 1 or 2 real world examples of this that you've seen? Or even better, examples of how one would write such locks in JS that would be effective against these type of race conditions?

GolDDranks 2y ago

Then again, threads are a primitive API for concurrency. I hope that structured concurrency becomes more common. How they are inplemented should be more or less an implementation detail. https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ https://vorpus.org/blog/notes-on-structured-concurrency-or-g...

HippoBaro 2y ago

I am not sure I buy the underlying idea behind this piece, that somehow a lot of money/time has been invested into asynchronous IO at the expense of thread performance (creation time, context switch time, scheduler efficiency, etc.). First, significant work has been done in the kernel in that area simply because any gains there massively impact application performance and energy efficiency, two things the big kernel sponsors deeply care about. Second, asynchronous IO in the kernel has actually been underinvested for years. Async disk IO did not exist at all for years until AIO came to be. And even that was a half-backed, awful API no one wanted to use except for some database people who needed it badly enough to be willing to put up with it. It's a somewhat recent development that really fast, genuinely async IO has taken center stage through io_uring and the likes of AF_XDP.

mmis1000 2y ago
Make os thread runs more efficient is like `faking async IOs (disk/network/whatever goes out from the computer shell) into the sync operations in a more efficient way`. But why would you do it at first place if the program can handle async operations at first place? Just let userland program do their business would be a better decision though.

deleted 2y ago

[deleted]

ruduhudi 2y ago

With async IO you can do stuff concurrently without doing it in parallel which is a desirable thing for many workloads. With only threads you‘d need sync primitives all over the place.

winternewt 2y ago

> Now imagine a parallel universe where instead of focusing on making asynchronous IO work, we focused on improving the performance of OS threads such that one can easily use hundreds of thousands of OS threads without negatively impacting performance I actually can't imagine how that would ever be accomplished at the OS level. The fact that each thread needs its own stack is an inherent limiter for efficiency, as switching stacks leads to cache misses. Asynchronous I/O has an edge because it only stores exactly as much state as it needs for its continuation, and multiple tasks can have their state in the same CPU cache line. The OS doesn't know nearly enough about your program to optimize the stack contents to only contain the state you need for the remainder of the thread. But at the programming language level the compiler does have insight into the dependencies of your continuation, so it can build a closure that has only what it needs to have. You still have asynchronous I/O at the core but the language creates an abstraction that behaves like a synchronous threaded model, as seen in C#, Kotlin, etc. This doesn't come without challenges. For example, in Kotlin the debugger is unable to show contents of variables that are not needed further down in the code because they have already been removed from the underlying closure. But I'm sure they are solvable.

dist1ll 2y ago
> But at the programming language level the compiler does have insight into the dependencies of your continuation This is really the key point - coupled with the fact that certain I/O operations are just inherently asynchronous. The TX/RX queues in NICs are an async, message passing interface - regardless of whether you're polling descriptors or receiving completion interrupts. So really, async I/O is the natural abstraction for networking.
- dietr1ch 2y ago
  Anything that happens far enough from the CPU is async, and here far probably means 10cm thanks to the speed of light not being fast enough, even in vacuum. So, any computation that spans a machine the size of our hands needs async unless you are willing to drop the clocks to push "far" a bit further away (and bring in power, heat and noise with it).

josefrichter 2y ago

Isn’t that parallel universe actually the Erlang BEAM?

BenoitP 2y ago
Or Go. Or Java' virtual threads. It must happen at the language level; When it comes to execution context knowledge: what context to compile out (stackless), or what context to serialize to the heap (stackful). Programming language will always know much more about the program than the OS. If I'm not mistaken in Erlang the programmer will provide the exact context to serialize : the actor.
- josefrichter 2y ago
  I mentioned the BEAM as “universe” especially because it provides so much more than Go or Java in this respect.
- yencabulator 2y ago
  This abstraction that a heap is somehow different than stack is hurting understanding; they're just regions of memory. Go stacks are allocated areas of memory like anything else. When a green thread is suspended, it's stack is not "serialized to the heap". Switching green threads is mostly a question of setting the stack pointer to point to the new green thread's stack. What makes threads "green" is that they are not supposed to call blocking syscalls (they can queue work in a separate blocking threadpool and call the scheduler to suspend themselves). Go has a signal-based mechanism for non-cooperative scheduling of green threads, so green threads are not even required to cooperatively yield to the scheduler.

stuaxo 2y ago

20 years ago we had the world of threads and it was very easy to get into a mess.

torginus 2y ago

Asynchronous IO is just simply how the world works. Instead, the idea that changes happen only during CPU computation is the mistake. Your disk drive/network card exists in parallel to your CPU and can process stuff concurrently. Your CPU very likely has a DMA engine that works in parallel without consuming a hardware thread.

yencabulator 2y ago
The "world" these days works by ringbuffers and interrupts, neither of which looks like async/await. Async/await is a chosen abstraction over what hardware/kernel interfaces really look like.
- torginus 2y ago
  Well, async/await in languages is just promises/futures made to look like regular flow control via state machines. And promises are controlled via callbacks, which are called from some event. It's essentially the same thing as when you get an interrupt from your microcontroller's DMA engine that the ring buffer needs to be refilled, just with 2 layers of abstractions on top to make it look nicer.
  - yencabulator 2y ago
    So you agree this to be wrong, then.. > Asynchronous IO is just simply how the world works. How the world works looks more like message passing. What abstraction you construct on top of that is a choice. Async/await is not necessarily the global maxima.

jeffreygoesto 2y ago

Some answer was already posted here: https://utcc.utoronto.ca/~cks/space/blog/tech/OSThreadsAlwaysExpensive https://utcc.utoronto.ca/~cks/space/blog/tech/OSThreadsAlway... https://news.ycombinator.com/item?id=41472027 https://news.ycombinator.com/item?id=41472027 To me the article reads as if the programming language author wants to push a difficult problem out of his language without deeper analysis. As if it would be easier if it was somebody else's problem.

csomar 2y ago

> Now imagine a parallel universe where instead of focusing on making asynchronous IO work, we focused on improving the performance of OS threads such that one can easily use hundreds of thousands of OS threads without negatively impacting performance Isn't that why async I/O was created in the first place? > Just use 100 000 threads and let the OS handle it. How does the OS handle it? How does the OS know whether to give it CPU time or not? I was expecting something from the OP (like a new networking or multi-threading primitive) but I have a feeling he lacks an understanding of how networking and async I/O works.

pizza 2y ago

Correct me if I'm wrong, but Microsoft's DirectStorage seems to me something like what the author is writing about. It lets you do eg massively parallel NVME file io ops from the GPU itself of lots of small files. This avoids the delay of the path through the CPU, any extra threads/saturation of the CPU, and even lets you do eg decompression of game assets on the GPU itself thereby saving even more CPU. This demo benchmark shows DEFLATE going from 1 GB/s on CPU to 7 GBs/ on GPU https://github.com/microsoft/DirectStorage/tree/main/Samples/GpuDecompressionBenchmark https://github.com/microsoft/DirectStorage/tree/main/Samples...

dudeinjapan 2y ago
I interpreted it as mainly network I/O, but the core point in the article is less about the I/O itself and more about thread-based async I/O handling.
- pizza 2y ago
  I see, I had this notion that DirectStorage worked by somehow allowing the GPU cores to host multiple parallel threads each doing full requests on their own, but on further research I was mistaken - requests are still CPU-submitted
  - compressedgas 2y ago
    I recently sought to check if DirectStorage used PCIe P2P to transfer data from the SSD to the GPU memory. It definitely could use it but I didn't see any evidence that it did.

alexgartrell 2y ago

> File IO is perhaps the best example of this (at least on Linux). To handle such cases, languages must provide some sort of alternative strategy such as performing the work in a dedicated pool of OS threads. AIO has existed for a long time. A lot longer than io_uring. I think the thing that the author misses here is that the majority of IO that happens is actually interrupt driven in the first place, so async io is always going to be the more efficient approach. The author also misses that scheduling threads efficiently from a kernel context is really hard. Async io also confers a benefit in terms of “data scheduling.” This is more relevant for workloads like memcached.

Someone 2y ago

FTA: “Need to call a C function that may block the calling thread? Just run it on a separate thread, instead of having to rely on some sort of mechanism provided by the IO runtime/language to deal with blocking C function calls.” And then? How do you know when your call completed without “some sort of mechanism provided by the IO runtime/language”? Yes, you periodically ask the OS whether that thread completed, but that doesn’t come for free and is far from elegant. There are solutions. The cheapest, resource-wise, are I/O completion callbacks. That’s what ”System” had on the original Mac in 1984, and there likely were even smaller systems before that had them. Easier for programmers would be something like what we now have with async/await. It might not be the best option, but AFAICT, this article doesn’t propose a better one. Yes, firing off threads is easy, but getting the parts together the moment they’re all available isn’t.

sedatk 2y ago

Many synchronous I/O operations under the hood are just async I/O + blocking waits, at least that's the case with Windows. Why? Because all I/O is inherently async. Even polling I/O requires timed waits which also makes it async. That said, I like async programming model in general, not just for I/O. It makes modeling your software as separetely flowing operations that need to be synchronized occasionally quite easy. Some tasks need to run in parallel? Then, you just wait for them later. I also like the channel concept of Golang and D in the same manner, but I heard it brought up some problems that async/await model didn't have. Can't remember what it was now. Maybe they are more susceptible to race conditions? Not sure.

snickerbockers 2y ago

This is extremely myopic. there is not a 1:1 correspondence between using asynchronous io and using one thread per file. Asynchronous io lets you dodge thread safety mechanisms like semaphores. Not all of us are trying to write a webapp or whatever, some of us just need to load a lot of data from several descriptors without serializing all the blocking operations. >Not every IO operation can be performed asynchronously though. File IO is perhaps the best example of this (at least on Linux). To handle such cases, languages must provide some sort of alternative strategy such as performing the work in a dedicated pool of OS threads. Uhhh this is just wrong file io can definitely be done asynchronously, on Linux, and without language support.

dwaite 2y ago

I don't quite agree with this piece, as it is comparing apples and oranges. What you want is patterns for having safety, efficiency and maintainability for concurrent and parallelized processing. One early pattern for doing that was codified as POSIX threads - continue the blocking processing patterns of POSIX so that you can have multiple parallelizable streams of execution with primitives to protect against simultaneous use of shared resources and data. IO_URING is not such a pattern. It is a kernel API. You can try to use it directly, but you can also use it as one component in a userland thread systems, in actor systems, in structured concurrency systems, etc. So the author is seemingly comparing the shipped pattern (threads) vs direct manipulation, and complaining that the direct manipulation isn't as safe or maintainable. It wasn't meant to be.

anonymoushn 2y ago

Do Mac and Windows really fail to expose some file I/O operations with kqueue and IOCP?

anonymoushn 2y ago

It's not clear that context switches can be made sufficiently cheap on fast CPUs without disabling mitigations for side-channel attacks. So the idea of making OS threads comparably performant to goroutines, rust async, or any implementation of cooperative multithreading seems impractical.

lima 2y ago

Google's SwitchTo kernel patches are similar to what you describe: https://lore.kernel.org/lkml/20200722234538.166697-1-posk@posk.io/ https://lore.kernel.org/lkml/20200722234538.166697-1-posk@po... Slides: https://web.archive.org/web/20200802205544/https://pdxplumbers.osuosl.org/2013/ocw//system/presentations/1653/original/LPC%20-%20User%20Threading.pdf https://web.archive.org/web/20200802205544/https://pdxplumbe...

weinzierl 2y ago

"Not every IO operation can be performed asynchronously though. File IO is perhaps the best example of this (at least on Linux). To handle such cases, languages must provide some sort of alternative strategy such as performing the work in a dedicated pool of OS threads." Can someone explain, why this would be the case? - Why can't every IO op be async? - Why is file IO on Linux not async? - What does iouring have to do with it?

seanhunter 2y ago
File IO can absolutely be async on linux[1] and this has been supported since version 2.5 or something provided that the device supports it (which they all have since about 2000). io_uring was the syscall interface that was introduced in 5.1 to improve async file i/o performance so it is relevant in that the previous way to do it wasn't great. [1] https://kkourt.io/blog/2017/10-14-linux-aio.html https://kkourt.io/blog/2017/10-14-linux-aio.html <- note the 2017 date.
yencabulator 2y ago
In-kernel file or block I/O is async if the driver/file implemented the async operations. If not, it's blocking. io_uring is a relatively new userspace syscall API to minimize the number of syscalls used for I/O, and to provide a common mechanism for doing many kinds of operations. Previously, large numbers of syscalls could become a bottleneck for busy applications, and some kinds of operations did not have a usable non-blocking interface.

sph 2y ago

Async IO is hilarious in a universe where Hewitt's actors or even Hoare's CSP exist. They are a subpar technique that works well with languages with semantics from the 1970s that do not have communication primitives, in the age of multicore and the Internet. The saddest thing is the most hyped language of the decade went all in with this miserable idea, and turned me completely off the ecosystem.

fire_lake 2y ago
How is CSP different from async/await? One is a semantics and the other is a syntax, no?
wruza 2y ago
What problems do you have with async io? I try to imagine and can’t see how it is bad. It’s completely transparent and non-problematic to me. Also, what’s the most hyped lang of the decade? Go, Kotlin?
Paradigma11 2y ago
I think CSP and Actors are overkill for nearly all programs. A simpler async system can do the job and introduce much less cognitive overhead. If really necessary there are still special libraries in most ecosystems like RX, Akka, Zio, Hopac, lwt......

praptak 2y ago

There's an effort by Google to address this with userspace threads. I think it has stalled though. HN discussion thereof: https://news.ycombinator.com/item?id=23964633 https://news.ycombinator.com/item?id=23964633

nasretdinov 2y ago

Somewhat controversial take: the current threads implementation is usually already performant enough for most use cases. The actual reason why we don't use them to handle more than a few thousand concurrent operations is that, at least in Linux, threads are scheduled and treated very similarly to processes. E.g. if a single process with 3000 threads gets bottlenecked on some syscall, etc, your system load average will become 3000, and it will essentially lead to no other processes being able to run well on the same machine. Another issue with threads performance is that they are visible to most system tools like `ps`, and thus having too mamy threads starts to affect operations _outside_ the kernel, e.g. many monitoring tools, etc. So that's the main reason why user-space scheduling became so popular: it hides the "threads" from the system, allowing for processes to be scheduled more fairly (preventing stuff like reaching LA 3000 when writing to 3000 parallel connections), and not affecting performance of the system infrastructure around the kernel. BTW the threads stacks, as well as everything else in Linux are allocated lazily, so if you only use like 4Kb of stack in the thread it wouldn't lead to RSS of full 8M. It will contribute to VMEM, but not RSS

viraptor 2y ago
> your system load average will become 3000, and it will essentially lead to no other processes being able to run well on the same machine. That doesn't sound right. I mean, there are many schedulers available, but I was under the impression that most have a separate blocked queue. (Or more specifically, anything blocked will not be in the runnable pool - those will be dequeued) I.e. anything waiting on a syscall will be mostly ignited. (Please correct me if I'm misunderstanding the usual behaviour here)
- nasretdinov 2y ago
  Yeah I think you are right, it depends on the syscalls, etc, in question. When that happened I didn't check which exact syscalls were being used
mike_hearn 2y ago
Linux ps doesn't show threads (by default). That was true decades ago before NPTL but hasn't been the case for a long time. The reason not to use kernel threads for everything is RAM utilization.

throwaway81523 2y ago

File io can now be done asynchronously with io_uring.

seanhunter 2y ago

> "Not only would this offer an easier mental model for developers..." Translation: "I find async i/o confusing and all developers are like me". This argument has been going on for over 20 years at this point. There are some people who think having pools of threads polling is a natural way of thinking about IO. They keep waiting for the day this becomes an efficient way to do IO.

tankenmate 2y ago
I would concur, the author also mentions that file IO isn't async; this makes me conclude that the author hasn't grasped how much the Linux kernel has moved on since in the mid 2000s. I suspect that the author has an incomplete view of the current kernel / userland interface as well as the inner workings of how a kernel actually "does what it does".
- baq 2y ago
  > the author also mentions that file IO isn't async And the best part is that NT kernel was always async… Linux is actually kind of special in that it was primarily sync.
  - tankenmate 2y ago
    The Linux kernel isn't "primarily" sync, the kernel itself has been almost entirely async except for some small parts, like the bottom half of interrupt handlers which can't be pre-empted. Even in the early days of Linux the scheduler was pre-emptive and used "wait channels" create a synchronous interface for the asynchronous kernel. In the early days of Linux, if a process was in kernel mode then it couldn't be pre-empted until it returned to user mode, but this is no longer the case, and with PREEMPT_RT this is even the case for RT processes /kernel tasks. Now, the POSIX API is largely synchronous, but this is mainly because of the history of UNIX (and Multics before it).
- YorickPeterse 2y ago
  That particular paragraph can be phrased better so I'll adjust that. What I meant to say is that you can't handle it asynchronously like you can with sockets, i.e. polling it for readiness. This is because for file IO, reads and writes are always reported as being available, making epoll/kqueue/etc effectively useless.

pyrolistical 2y ago

What if the author’s proposed solution is the billion dollar mistake? IMO the best programming paradigms are when the abstractions are close to the hardware. Instead of pretending to have unlimited cores, what if as part of the runtime of we are given the exactly one thread per core. As the programmer we are responsible for utilizing all the cores and passing data around. It is then up to the operating system to switch entire sets of cores over different processes. This removes the footgun of a process overloading a computer with too many threads. Programmers need to consider how to best distribute work over a finite number of cores.

mmis1000 2y ago

I think compare programing pattern(threads) to kernel async apis is a questionable comparation. The point of kernel async apis is not about letting programmers write system calls directly. It's about expose the actual async operations under the hook (it could be disk, be network, be anything outside of the computer case). Those actions are never mean to be interleaved with cpu computation, because they are usually with ms level delay (which could be millions of cpu ticks). The kernel fakes these into sync calls by pause everything. But it isn't always the best idea to do these. Let userland program decide what they want to do with the delay will be a way better idea. Even they eventually just invent blocking io calls again. They can still decide what operations are more relevant to itself instead of let the kernel guessing it.

dudeinjapan 2y ago

This article is a billion dollar mistake.

necovek 2y ago

To put a different spin on what others are saying, asynchronous IO is a different programming model for concurrency that's actually more ergonomic and easier to get right for an average developer (which includes great developers on their not-highly-focused days). Dealing with raciness, deadlocks and starvation is simply hard, especially when you are focused on solving a different but also hard business problem. That's also why RDBMSes had and continue to have such a success: they hide this complexity behind a few common patterns and a simple language. Now, I do agree that languages that suffer from the "color of your functions" problems didn't get it right (Python, for instance). But ultimately, this is an easier mental model, and it's been present since the dawn of purely functional languages (nothing stops a Lisp implementation from doing async IO, and it might only be non-obvious how to do "cancellation" while "gather" is natural too)

ivanjermakov 2y ago

Async IO is not only about creating sockets and spawning threads. Idea of async IO is that the world is not controlled by your CPU. There are network, storage, sound devices that might and will take time to produce the result and the CPU has to wait for it. I feel like there is a big misunderstanding about what async IO is and what problem it solves.

scotty79 2y ago
Doesn't message passing capture this essence better?
- ivanjermakov 2y ago
  Message passing, channels, thread pool, callbacks, polling - there are many ways to implement async IO and async computation in general.

dfgdfg34545456 2y ago

The post seems to be assuming that multi threaded code is easy to build and maintain. From my experience it is horrible, every new thread means going from n bugs to nn bugs. As a programmer I prefer* async constructs in languages, and do not want to spin up and manage threads and all the state synchronisation that involves.

algobro 2y ago

The 1990s called. They want their threads vs events debates back. Todays processors are fast enough to serve many useful workloads with a single core. The benefit of the async abstraction outweighs the performance benefit in the majority of the cases. And debugging multithreaded code is way harder than async code, mainly if its the kind of program that needs stepping into.

deleted 2y ago

[deleted]

bob1029 2y ago

> the cost to start threads is lower, context switches are cheaper, etc. Physics would have a word with this one. We are already pushing limits of what is possible with latency between cores vs overall system performance. There isn't an order of magnitude improvement hiding in there anywhere without some FTL communication breakthrough. In theory, yes we could sweep this problem under the rug of magical, almost-free threads. But these don't really exist. I think the best case for performance is to accumulate mini batches of pending IO somewhere and then handle them all at once. IO models based upon ring buffers are probably getting close to the theoretical ideal when considering how our CPUs work internally (cache coherency, pipelining, etc).

YorickPeterse 2y ago
Is this actually proven though? I've seen other people argue that threads are already as fast as they can be, yet nobody is able to actually substantiate that claim.

neonsunset 2y ago

No, it isn’t, the author is just confused, as it usually is. Worked great in C# since its introduction for task interleaving, composition, cancellation (worse languages call it structured concurrency) and switching to privileged context (UI thread), and will work even better in .NET 10.

forty 2y ago

Maybe just use Nodejs? Everything is async by default and the API is not annoying or difficult to use

Szpadel 2y ago

I think async vs threads is about completely different trade-off. Nowadays all operating systems so preemptive scheduling, but green threads (and all async by the extent) use cooperative scheduling. I believe most of discussion here is actually about pros and cons of those scheduling models. one exception is I think cancellation model, but I'm only aware about rust that does it that way, all other runtimes will happily run your green thread until it finishes or cancel by itself similarly that you do with synchronous code.

scarnie 2y ago

The author appears to contradict the very issue they argue, by presenting languages such as Go, Erlang or their own toy language. These languages hide the async / await constructs that are present in languages like Rust, Swift or Typescript. The former languages and runtimes have no function colouring problems, when working within their own SDKs. There are trade offs, and these “async” languages tend to be a bit more awkward when interacting with OS frameworks.

chucke 2y ago

My interpretation of what the author wants, is essentially lightweight threads in the kernel, standardised a lá POSIX , that every proglang could use as a primitive. That'd be sweet if this were a well understood problem. Unfortunately, we're still finding the sweet spot between I/O Cs CPU bound tasks, "everything is a file" clashing with async network APIs and mostly sync file APIs, and sending that research to the kernel would mean having improvements widely distributed in 5 years or more, and would set back the industry decades, if not centuries. We learned this much already with the history of TCP and the decision of keeping QUIC in userspace.

Brian_K_White 2y ago

Take the title at face value. If it were a sincere question the sincere answer is simply no. The world actually is concurrent and asynchronous regardless how inconvenient that is for a programmer.

rurban 2y ago

On the contrary blocking IO (and POSIX) is the old billion dollar mistake. It's inefficient and blocks concurrency safety.

orf 2y ago

Computers are inherently asynchronous, we just plaster over that with synchronous interfaces. We put a lot of effort into maintaining these synchronous facades - from superscalar CPUs translating assembly instructions into “actual” instructions and speculatively executing them in parallel to prevent stalls, to the kernel with preemptive scheduling, threads and their IO interfaces, right up to user-space and the APIs they provide on top of all this. Surely there has to be a better way? It seems ridiculous.

yencabulator 2y ago
https://en.wikipedia.org/wiki/Dataflow_architecture https://en.wikipedia.org/wiki/Dataflow_architecture https://stackoverflow.com/questions/530180/what-happened-to-clockless-computer-chips https://stackoverflow.com/questions/530180/what-happened-to-...

nurettin 2y ago

In several projects I switched from async code to threads and mpsc queues. Timers, data streams, external communications all run in their threads. They pass their messages to the main thread's queue. The entire thing suddenly became much easier to reason about and read.

OutOfHere 2y ago

> Languages such as Go and Erlang bake support for asynchronous IO directly into the language, while others such as Rust rely on third-party libraries such as Tokio. This is so wrong. Go and Erlang have message passing, not async. Message passing is its own thing; it should not be mixed with threading or async.

jclulow 2y ago
Go offers channels in the standard library, which you can use to do message passing, certainly (so does Rust). But I/O through the stdlib is absolutely asynchronous. I/O routines that would block, like sending and receiving from the network, are all magic async switch points where your goroutine can be replaced on the current OS thread with another. When epoll (or whatever) signals readiness on the blocked descriptor you get switched back on. As described in the previous post, the asynchronous I/O is hidden from you as an implementation detail of the language but it's absolutely there.
- OutOfHere 2y ago
  That's in practically every nontrivial language.

quercus 2y ago

I thought this article was going to talk about async IO's potential to cause insidious bugs that you don't notice until they're crashing jet planes. That's definitely a topic worth of a blog post.

lincpa 2y ago

[dead]