18 ms·
Why choose async/await over threads?
- newpavlov 3y agoI think a better question is "why choose async/await over fibers?". Yes, I know that Rust had green threads in the pre-1.0 days and it was intentionally removed, but there are different approaches for implementing fiber-based concurrency, including those which do not require a fat runtime built-in into the language. If I understand the article correctly, it mostly lauds the ability to drop futures at any moment. Yes, you can not do a similar thing with threads for obvious reasons (well, technically, you can, but it's extremely unsafe). But this ability comes at a HUGE cost. Not only you can not use stack-based arrays with completion-based executors like io-uring and execute sub-tasks on different executor threads, but it also introduces certain subtle footguns and reliability issues (e.g. see [0]), which become very unpleasant surprises after writing sync Rust. My opinion is that cancellation of tasks fundamentally should be cooperative and uncooperative cancellation is more of a misfeature, which is convenient at the surface level, but has deep issues underneath. Also, praising composability of async/await sounds... strange. Its viral nature makes it anything but composable (with the current version of Rust without a proper effect system). For example, try to use async closure with map methods from std. What about using the standard io::Read/Write traits? [0]: https://smallcultfollowing.com/babysteps/blog/2022/06/13/async-cancellation-a-case-study-of-pub-sub-in-mini-redis/ https://smallcultfollowing.com/babysteps/blog/2022/06/13/asy...
- lmm 3y agoHow do fibers solve your cancellation problem? Aren't they more or less equivalent? (I find fiber-based code hard to follow because you're effectively forced to reason operationally. Keeping track of in-progress threads in your head is much harder than keeping track of to-be-completed values, at least for me)
- bheadmaster 3y ago> Keeping track of in-progress threads in your head is much harder than keeping track of to-be-completed values, at least for me I think that's true for everybody. Our minds barely handle state for sequential code - the explosion of complexity of multiple state-modifying threads is almost impossible to follow. There are ways to convert "keeping track of in-progress threads" to "keeping track or to-be-completed" values - in particular, Go uses channels as a communication mechanism which explicitly does the latter, while abstracting away the former.
- lmm 3y ago> There are ways to convert "keeping track of in-progress threads" to "keeping track or to-be-completed" values - in particular, Go uses channels as a communication mechanism which explicitly does the latter, while abstracting away the former. I find the Go style pretty impossible to follow - you have to keep track of which in-progress threads are waiting on which lines, because what will happen when you send to a given channel depends on what was waiting to receive from that channel, no? The only way of doing this stuff that I've ever found comprehensible is iteratees, where you reify the continuation step as a regular value that runs when you call it explicitly.
- newpavlov 3y agoWith fibers you send cancellation signal to a task, then on next IO operation (or more generally yield) it will get cancellation error code with ability to get true result of IO operation, if there is any. Note that it does not mean that the task will sleep until the IO operation gets completed, cancellation signal causes any ongoing IO to "complete" immediately if it's possible (e.g. IIRC disk IO can not be cancelled). It then becomes responsibility of the task to handle this signal. It may either finish immediately (e.g. by bubbling the "cancellation" error), finish some critical section before that and do some cleanup IO, or it may even outright ignore the signal. With futures you just drop the task's future (i.e. its persistent stack) maybe with some synchronous cleanup and that's it, you don't give the task a chance to say a word in its cancellation. Hypothetical async Drop could help here (though you would have to rely on async drop guards extensively instead of processing "cancellation errors"), but adding it to Rust is far from easy and AFAIK there are certain fundamental issues with it. With io-uring sending cancellation signals is quite straightforward (though you need to account for different possibilities, such as task being currently executed on a separate executor thread, or its CQE being already in completion queue), but with epoll, unfortunately, it's... less pleasant.
- lmm 3y ago> Hypothetical async Drop could help here (though you would have to rely on async drop guards extensively instead of processing "cancellation errors"), but adding it to Rust is far from easy and AFAIK there are certain fundamental issues with it. Wouldn't fiber cancellation be equivalent and have equivalent implementation difficulties? You say you just send a signal to the task, but in practice picking up and running the task to trigger its cancellation error handling is going to look the same as running a future's async drop, isn't it?
- newpavlov 3y agoFirstly, Rust does not have async Drop and it's unlikely to be added in the foreseeable future. Secondly, cancellation signals is a more general technique than async Drop, i.e. you can implement the latter on top of the former, but not the other way around. For example, with async Drop you can not ignore cancellation event (unless you copy code of your whole task into Drop impl). Some may say that it's a good thing, but it's just an obvious example of cancellation signals being more powerful than hypothetical async Drop. As for implementation difficulties, I don't think so. For async Drop you need to mess with some fundamental parts of the Rust language (since Futures are "just types"), while fiber-based concurrency, in a certain sense, is transparent for compiler and implementation complexity is moved to executors. If you are asking about how it would look in user code, then, yes, they would be somewhat similar. With cancellation signals you would call something like `let res = task_handle.cancell_join();`, while with async Drop you would use `drop(task_future)`. Note that the former also allows to get result from a cancelled task, another example of greater flexibility.
- lloeki 3y ago> I think a better question is "why choose async/await over fibers? > there are different approaches for implementing fiber-based concurrency, including those which do not require a fat runtime built-in into the language This keynote below is Ruby so the thread/GVL situation is different than for Rust, but is that the kind of thing you mean? https://m.youtube.com/watch?v=qKQcUDEo-ZI https://m.youtube.com/watch?v=qKQcUDEo-ZI I think it makes a good case that async/await is infectious and awkward, and fibers (at least as implemented in Ruby) is quite simply a better paradigm.
- newpavlov 3y agoI only scrolled the video, but it sounds similar, yes. Though, implementation details would probably vary significantly, since Rust is a lower-level language.
- dwaite 3y ago> I think it makes a good case that async/await is infectious and awkward, and fibers (at least as implemented in Ruby) is quite simply a better paradigm. As someone who has done fibers development in Ruby, I disagree. CRuby has the disadvantage of a global interpreter lock. This means that parallelism can only be achieved via multiple processes. This is not the case in Rust, where have access to do true parallelism in a single process. Second, this talk is not arguing for use of fibers as much as it is arguing as using fibers to rig up a bespoke green threads-like system for a specific web application server, and advocating for ruby runtime features to make the code burden on them of doing this lighter. Ruby has a global interpreter lock, so even though it uses native threads only one of them can be executing ruby code at a time. Fibers have native stacks, so they have all the resource requirements of a thread sans context switching - but the limitations from the GIL actually mean you aren't _saving_ context switching by structuring your code to use fibers in typical (non "hello world" web server) usage.
- lloeki 3y agoI'm sorry but you seem to have missed the point I'm referring to, even though you have directly quoted it. None of your paragraphs are relevant to async await vs fiber: async await requires you to put keywords in all sorts of unexpected places, fibers do not. > CRuby has the disadvantage of a global interpreter lock. This means that parallelism can only be achieved via multiple processes. I am very well cognisant of this fact, but this bears absolutely no relationship with async await vs fibers: one is sweeping callbacks on an event loop under leaky syntactic sugar, the other is cooperative coroutines; they're all on the same Ruby thread, the GVL simply does not intervene. > this talk is not arguing for use of fibers as much as it is arguing as using fibers to rig up a bespoke green threads-like system for a specific web application server, and advocating for ruby runtime features to make the code burden on them of doing this lighter. That sounds like a very cynical view of it. I believe there are arguments made in this talk that are entirely orthogonal to how Falcon benefits from fibers. > Fibers have native stacks, so they have all the resource requirements of a thread For starters fibers don't have the resource requirements of a thread because they're not backed by native OS threads, while CRuby creates an OS thread for each Ruby VM thread (until MaNy lands, but even with M:N they'd still be subject to the GVL). Are you arguing for a stackless design like Stackless Python? Goroutines are stack-based too and the benefit of coroutines (and a M:N design) is very apparent. Anyway async await is stack-based too so I don't see how this is relevant: if you have 1M requests served at the same time truly concurrently you're going to have either 1M event callbacks or 1M fibers; that's 1 million stacks either way. I seem to gather through what I read as a bitter tone that your experience with fibers was not that good. I appreciate another data point but I can't seem to reconcile it vs async await. But really, the only reason I brought up threads is because the GVL makes threads less useful in CRuby than in Rust (or Go since I mentioned it).
- logicchains 3y agoThe sole advantage of async/await over fibers is the possibility to achieve ultra-low-latency via the compiler converting the async/await into a state machine. This is important for Rust as a systems language, but if you don't need ultra-low-latency then something with a CSP model built on fibers, like Goroutines or the new Java coroutines, is much easier to reason about.
- newpavlov 3y agoFibers and async/await are backed by the same OS APIs, they can achieve more or less the same latency. The main advantage of async/await (or to be more precise stackless coroutines) is that they require less memory for task stacks since tasks can reuse executor's stack for non-persistent part of their stack (i.e. stack variables which do not cross yield points). It has very little to do with latency. At most you can argue that executor's stack stays in CPU cache, which reduces amount of cache misses a bit. Stackless coroutines also make it easier to use parent's stack for its children stacks. But IMO it's only because compilers currently do not have tools to communicate maximum stack usage bound of functions to programming languages.
- logicchains 3y ago>Fibers and async/await are backed by the same OS APIs, they can achieve more or less the same latency. The key requirement for ultra-low-latency software is minimising/eliminating dynamic memory allocation, and stackless coroutines allow avoiding memory allocation. For managed coroutines on the other hand (e.g. goroutines, Java coroutines) as far as I'm aware it's impossible to have an implementation that doesn't do any dynamic memory allocation, or at least there aren't any such implementations in practice.
- newpavlov 3y agoYes, it's what I wrote about in the last paragraph. If you can compute maximum stack size of a function, then you can avoid dynamic allocation with fibers as well (you also could provide stack size manually, but it would break horribly if the provided number is wrong). You are right that such implementations do not exist right now, but I think it's technically feasible, as demonstrated by tools such as https://github.com/japaric/cargo-call-stack https://github.com/japaric/cargo-call-stack The main stumbling block here is FFI, historically shared libraries do not have any annotations about stack usage, so functions with bounded stack usage would not be able to use even libc.
- dwaite 3y agoFor rust, fibers (as a user-space, cooperative concurrency abstraction) would mandate a lot of design choices, such as whether stacks should be implemented using spaghetti stacks or require some sort of process-level memory mapping library, or even if they were just limited to a fixed size stack. All three of these approaches would cause issues when interacting with code in another language with a different ABI. It can get really complicated, for example, when C code gets called from one fiber and wants to then resume another. One of the benefits of async/await is the 'await' keyword itself. The explicit wait-points give you the ability to actually reason about the interactions of a concurrent program. Yielding fibers are a bit like the 'goto' of the concurrency world - whenever you call a method, you don't know if as a side effect it may cause your processing to pause, and if when it continues the state of the world has changed. The need to be defensive when interfacing with the outside world means fibers tend to be better for tasks which run in isolation and communicate by completion. Green threads, fibers and coroutines all share the same set of problems here, but really user space cooperative concurrency is just shuffling papers on a desk in terms of solving the hard parts of concurrency. Rust async/await leaves things more explicit, but as a result doesn't hide certain side effects other mechanisms do.
- newpavlov 3y agoIn my opinion, by default fibers should use "full" stacks, i.e. a reasonable amount of unpopulated memory pages (e.g. 2 MiB) with guard page. Effectively, the same stack which we use for threads. It should eliminate all issues about interfacing with external code. But it obviously has performance implications, especially for very small tasks. Further, on top of this we can then develop spawning tasks which would use parent's stack. It would require certain language development to allow computing maximum stack usage bound of functions. Obviously, such computation would mean that programmers have to take additional restrictions on their code (such as disallowing recursion, alloca, and calling external functions without attributing stack usage), but compilers already routinely compute stack usage of functions, so for pure Rust code it should be doable. >It can get really complicated, for example, when C code gets called from one fiber and wants to then resume another. It's a weird example. How would a C library know about fiber runtime used in Rust? >Yielding fibers are a bit like the 'goto' of the concurrency world - whenever you call a method, you don't know if as a side effect it may cause your processing to pause, and if when it continues the state of the world has changed. I find this argument funny. Why don't you have the same issue with preemptive multitasking? We live with exactly this "issue" in the threading world just fine. Even worse, we can not even rely on "critical sections", thread's execution can be preempted at ANY moment. As for `await` keyword, in almost all cases I find it nothing more than a visual noise. It does not provide any practically useful information for programmer. How often did you wonder when writing threading-based code about whether function does any IO or not?
- ngrilly 3y agoCould you share an example of a fiber implementation not relying a fat runtime built in the language?
- newpavlov 3y agohttps://github.com/Xudong-Huang/may https://github.com/Xudong-Huang/may The project has some serious restrictions and unsound footguns (e.g. around TLS), but otherwise it's usable enough. There are also a number of C/C++ libraries, but I can not comment on those.
- ngrilly 3y agoNeat. Thanks for sharing! Interestingly, may-minihttp is faring very well in the TechEmpower benchmark [1], for whatever those benchmarks are worth. The code is also surprisingly straightforward [2]. [1] https://www.techempower.com/benchmarks/ https://www.techempower.com/benchmarks/ [2] https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/Rust/may-minihttp/src/main.rs https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...
- anonymoushn 3y agoFor example https://github.com/creationix/libco https://github.com/creationix/libco The required thing is mostly just to dump your registers on the stack and jump.
- noselasd 3y agoI do think implementations like that are not particularly useful though. You want a runtime to handle and multiplex blocking calls - otherwise if you perform any blocking calls (mostly I/O) in one fiber, you block everything - so what use are those fibers ?
- anonymoushn 3y agoThe answer is the same as in async Rust, right? "Don't do that." If you wanted to use this for managing a bunch of I/O contexts per OS thread then you would need to bring an event loop and a bunch of functions that hook up your event loop to whatever asynchronous I/O facilities your OS provides. Sort of like in async Rust.
- zozbot234 3y agoBecause stackful fibers suck for low-level code. See Gor Nishanov's review for the C++ committee http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p1364r0.pdf http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p136... (linked from https://devblogs.microsoft.com/oldnewthing/20191011-00/?p=102989 https://devblogs.microsoft.com/oldnewthing/20191011-00/?p=10... ). It even sums things up nicely: DO NOT USE FIBERS!
- gpderetta 3y agoTo be clear, not everybody agrees with Gor and although they still don't have much traction, stackful coroutines are still being proposed. Most (but admittedly not all) of Gor issues with stackful coroutines are due to current implementations being purely a library feature with no compiler support. Most of the issues (context switch overhead, tread local issues) can be solved with compiler support. The issue with lack of support in some existing libraries is unfair, neither do async/await and in fact it would be significantly harder to add support for those as they would require a full rewrite. The take away here is not to try to make M:N fully transparent, not avoid it completely. The issue with stack usage is real though, split stacks is only a panacea and OS support would be required. edit: for the other side of the coin, also from microsoft, about issues with the async/await model in Midori (an high performance single-address-space OS): https://joeduffyblog.com/2015/11/19/asynchronous-everything/ https://joeduffyblog.com/2015/11/19/asynchronous-everything/ They ended up implementing the equivalent of stackful coroutines specifically for performance.
- zozbot234 3y ago> The take away here is not to try to make M:N fully transparent But the whole point of the 'Virtual processors' or 'Light-weight process' patterns as popularized by Golang and such is to try and make M:N transparent to user code.
- gpderetta 3y agoI mean fully transparent to pre-existing code. Go had the benefit of having stackful coroutines from day one, but when retrofitting to an existing language, expecting the whole library ecosystem to work out of the box without changes is a very high bar.
- deleted 3y ago[deleted]
- lamontcg 3y ago> My opinion is that cancellation of tasks fundamentally should be cooperative and uncooperative cancellation is more of a misfeature, which is convenient at the surface level, but has deep issues underneath. Cooperative cancellation can be pretty annoying with mathematical problems. You can have optimization algorithms calling rootfinding problems calling ODE integrators and any of those can spin for a very long time and you need be threading through cancellation tokens everywhere and the numerical framework generally don't support it. You can and should use iteration counts in all the algorithms but once you're dealing with nested algorithms that can only guarantee that your problem will stop sometime this year, not that it stops within 5 seconds. With these problems I can promise that I'm just doing: lots of math, allocations with their associated page faults, no I/O, writing strings to a standard library Queue object for logging that are handled back in the Main thread that I'm never going to cancel (and whatever other features you think I might need -- which I haven't needed for years now -- I'd be happy to ship that information back to the main thread on a Queue). It feels like that problem should be solvable in the 21st century to me without making me thread cancellation tokens everywhere and defensively code against spinning without checking the token (where I can make mistakes and cause bugs, which I guess you'll just blame me for).
- pas 3y agobut why do you want cooperative cancellation? this seems ideal for a thread pool, and if the allocated time is up, ask the OS to stop them. ideally the system would allocate the threads to dedicated CPU cores that have all kinds of interrupt handling disabled (so the throughout is maximalized) if you want to do this without kernel support, then somehow the program needs to do the periodic checks. the JVM has this (safepoints), but there's no royal road for this.
- lamontcg 3y ago> but why do you want cooperative cancellation? I don't. Previous commenter doesn't believe in uncooperative cancellation though and .NET seems to be removing all support for it. I would love to have worker threads with all kinds of interrupts and I/O disabled on them and just be able to kill them if they wander off into a numerical field somewhere and get stuck in the mud.
- hsjsbeebue 3y agoWhat is the language agnostic answer to the same question? I imagine something to do with memory usage or avoiding thread or thread pool starvation issues. Maybe performance too?
- toast0 3y agoI don't have a lot of experience with async/await at high numbers of tasks, but I've run Erlang with millions of processes. It's a lot easier to run millions of Erlang processes on one machine than to run a million OS threads. I suspect async tasks would be similar; an OS thread needs its own stack, and that's going to use at least a page of memory, but often much more. Otoh, an async task or green thread might be able to use less. If you're running real OS threads, I think task switching is going to be real context switches, which might mean spectre mitigations clear your cpu caches, but task switching can avoid that. You may end up with more system calls with OS threads, because your runtime might be able to aggregate things a bit (blocking reads become kqueue/epoll/select, but maybe that's actually a wash, because you do still need a read call when the FD is ready, and real blocking only makes a single call)
- lmm 3y agoIMO the biggest reason to avoid threads is simply that it's ~impossible to write safe code using threads (e.g. without race conditions). Arguably with Rust's ownership system that's less true there than in other languages.
- kaba0 3y agoAsync-await is about concurrency, not parallelism. It can work in both a single-threaded and multi-threaded context, the latter exposing all the typical failure modes of multi-threaded code. Also, Rust’s ownership model only prevents data races, that’s only the tip of the iceberg of race conditions, and I don’t think that any general model makes it possible to statically determine that any given multithreaded code is safe. Nonetheless, that’s the only way to speed up most kind of code, so possibly the benefits outweigh the cost in many cases.
- treflop 3y ago
- adontz 3y agoThere are a lot of moments not covered. For example: - async/await runs in context of one thread, so there is no need for locks or synchronization. Unless one runs async/await in multiple threads to actually utilize CPU cores, then locks and synchronization are necessary again. This complexity may be hidden in some external code. For example instead of synchronizing access to a single database connection it is much easier to open one database connection per async task. However such approach may affect performance, especially with sqlite and postgres. - error propagation in async/await is not obvious. Especially when one tries to group up async tasks. Happy eyeballs are a classic example. - since network I/O was mentioned, backpressure should also be mentioned. CPython implementation of async/await notoriously lacks network backpressure causing some problems.
- graphenus 3y agoAsync/await just like threads is a concurrency mechanism and also always requires locks when accessing the shared memory. Where does your statement come from?
- romanovcode 3y ago> Where does your statement come from? This is how async/await works in Node (which is single-threaded) so most developers think this is how it works in every technology.
- bheadmaster 3y agoEven in Node, if you perform asynchronous operations on a shared resource, you need synchronization mechanisms to prevent interleaving of async functions. There has been more than one occasion when I "fixed" a system in NodeJS just by wrapping some complex async function up in a mutex.
- nurple 3y agoThis lacks quite a bit of nuance. In node you are guaranteed that synchronous code between two awaits will run to completion before another task(that could access your state) from the event loop gets a turn; with multi-threaded concurrency you could be preempted between any two machine instructions. So while you _do_ have to serialize access to shared IO resources, you do _not_ have to serialize access to memory(just add the connection to the hashset, no locks). What you usually see with JS for concurrency of shared IO resources in practice is that they are "owned" by the closure of a flow of async execution and rarely available to other flows. This architecture often obviates the need to lock on the shared resource at all as the natural serialization orchestrated by the string of state machines already naturally accomplishes this. This pattern was even quite common in the CPS style before async/await. For example, one of the first things an app needs do before talking to a DB is to get a connection which is often retrieved by pulling from a pool; acquiring the reservation requires no lock, and by virtue of the connection being exclusively closed over in the async query code, it also needs no locking. When the query is done, the connection can be replaced to the pool sans locking. The place where I found synchronization most useful was in acquiring resources that are unavailable. Interestingly, an async flow waiting on a signal for a shared resource resembles a channel in golang in how it shifts the state and execution to the other flow when a pooled resource is available. All this to say, yeah I'm one of the huge fans of node that finds rust's take on default concurrency painfully over complicated. I really wish there was an event-loop async/await that was able to eschew most of the sync, send, lifetime insanity. While I am very comfortable with locks-required multithreaded concurrency as well, I honestly find little use for it and would much prefer to scale by process than thread to preserve the simplicity of single-threaded IO-bound concurrency.
- andrewstuart 3y ago>> A common refrain is that threads can do everything that async/await can, but simpler. Who says that? Threads and async/await are different things, and it doesn’t make sense to say one can do what the other does. And threads definitely are not simpler than AA
- OtomotO 3y agoThreads are easier to spawn and that's where all the fuzz comes from, I argue. Especially in Rust, Async isn't as easy as in other languages with a runtime and it does indeed have some caveats (e.g. cancellation), but all the real fuzz comes from not understanding that they are different strategies for some similar but not the same problems. It makes little to no sense to use Async/Await for number crunching/CPU-compute intensive tasks, for example. One can use Threads for some IO waiting though, but it's definitely not the best solution for that particular problem. To me this whole discussion has two facets: 1) How can Async/Await be more ergonomic in Rust 2) How can we teach people that Async/Await is a different solution with different tradeoffs to Threads - there is a reason why Async/Await was created AFTER WE ALREADY HAD INVENTED THREADS!
- dboreham 3y agoAsync wasn't invented after threads. It was primarily popularized in a system that has been designed such that it couldn't use threads (the web browser). Everything else is post-justifcation for why it's better. It isn't better.
- OtomotO 3y agoAsync/Await as a syntax thing wasn't, but Async/Await as "don't just blindly use Threads for scaling to a myriad of incoming HTTP requests" was. I remember the Apache webserver story and that one has little to nothing to do with Webbrowsers or JavaScript ;)
- nurple 3y agoMy problem with rust's async/await is that it's _not_ a different strategy as the continuation tasks _are_ run multithreaded, so it's technically both strategies. IMO one of the biggest selling points of single-threaded async/await was how much complexity falls away compared to managing preemptive synchronization in the multi-threaded case. I can see why there's so much controversy over async/await in rust. If I had to take both the syntax and cognitive hit of using async/await _and_ multi-threading, I would also angrily call for its removal.
- plugin-baby 3y agois the difference between threads and async/await more than syntax? or language-specific?
- romanovcode 3y agoYes, they are different in most cases. In general await job is to pass the process to 3rd party e.g. database or http and wait for callback whereas thread job is to launch multiple CPU operations in parallel.
- jokethrowaway 3y agoAwait-async is implemented with a runtime which uses a thread pool or a single thread and allocates work when needed on any thread and waits for IO to yield a result. With threads you just fully control what blocking code is running on a single thread. If you are just running computations (or reading files, as filesystem api are not async) it's simpler to just use threads.
- plugin-baby 3y ago> or reading files, as filesystem api are not async This is interesting - all nodejs file system APIs are async by default.
- zaphar 3y agoThat's because javascript as a whole is async by default. Which really makes me confused as to why javascript didn't just go the route of Go and eliminate the distinction instead of falsely creating it with syntax.
- immibis 3y agoImplementation difference. Threads are usually handled by the kernel and each one has its own stack where you can put anything. And the kernel can switch threads at any time. Async/await has the compiler work out precisely what has to be saved across the marked locations which are the only places context switches can happen. Also it doesn't tell the kernel when it switches context.
- ikekkdcjkfke 3y agoWhat i would hope for is implicit async/await so i don't have to write it out / wrap all the time
- groestl 3y agoThat's eerily similar to what (green) threads are.
- hinkley 3y agoI’ve always thought async await is just a less clunky implementation of green threads. Not as a feature parity replacement, but trying to fill the same niche.
- yodon 3y ago>implicit async/await I keep suspecting C# will be the place where we see this, but probably not for another couple years.
- asabla 3y agoWe might not be that far away already. There is this issue[1] on Github, where Microsoft and the community discuss some significant changes. There is still a lot of questions unanswered, but initial tests look promising. Ref: https://github.com/dotnet/runtime/issues/94620 https://github.com/dotnet/runtime/issues/94620
- anonymoushn 3y agoWe have it in Zig, Lua, Python, C, etc.
- t-writescode 3y agoIn Kotlin, only the outer-most wrapping 'suspend' function swap needs to be wrapped; and when you're writing web servers and such, that's handled in the server middleware and so all your code can be `suspend fun blahblah()`
- Aardwolf 3y ago
- ggm 3y agoAs one who struggled with thread safe storage since threads started, the amount of code we carry around with us which looks ok, but turns out not to be viable in threads is remarkably high. I bump into this in C, Python3 quite a lot: you have to work harder to do anything which is not synchronous, no matter how you arrive in that place. For long lived work, it is not impossible there is no advantage overall to forking a lot of heavyweight processes which operate fast as a single execution state. If you have the cores, the CPU and the memory. Context switching delay is very possibly not your main problem. For example, I have typically 24 1 hour capture/log files, each 300m+ lines big and I need to ungzip them, parse/grep out some stuff, calculate some aggregates and then recombine. Its a natural pipeline of 3 or 4 processes. The effort to code this into a single language and uplift it to run threads inside, where it's basically 24 forked pipe sequences begs questions: What exactly is going to get faster, when I am probably resource limited by the gunzip process at the front? You think you can code faster than a grep DFA to select inputs? How sure are you that memory structure is faster than awk? Really sure? I tested some. radix tries are nice and have that log(n) length thing, but AWK was as fast. (IP address "hashing" lookup costs) (hint: more than one interpreted language simply forks gzip for you, to unzip an input stream) If you can go to C, then it's possible forking heavyweight processes in C, with good IPC buffer selections or mmap is "good enough"
- pjmlp 3y agoAs someone that was big into threading back in the 2000, with the experience gathered through times, I think with modern hardware resources we are better off with OS IPC, as it offers much better safety guarantees, specially in C like languages.
- coolThingsFirst 3y agoHow is async/await implemented under the hood?
- lmm 3y agoGenerally via continuations. An async function is transformed to continuation-passing style, await calls with the current continuation, and then you have a runtime that at its simplest is just, like, a queue of tasks and has special-cased primitives for doing things like async I/O where you suspend, and it just pulls tasks off the queue and runs them, and when one task suspends it stores the continuation and runs the next one.
- politelemon 3y agoDoes that runtime run the tasks across multiple cores?
- amaranth 3y agoIn Rust the answer is "it depends". Since the runtime is not provided by the language you can have implementations that are a single thread, thread-per-task, a thread pool, or whatever other setup you can think of. Tokio at least offers a single threaded version and a thread pool version.
- simonask 3y agoAsync/await in Rust is famously not based on continuations, at least not in any traditional sense, where a block of code is passed to a reactor system to be invoked whenever an operation completes. Instead it is based on "wakers", which are objects associated with a task, and can be used to notify the task's executor that the task is ready to make progress. It is then the job of an executor resume the task. So there is an extra layer of indirection (conceptually). There are pros and cons, but in essence the system trades a check on resume (often redundant) for the need to make a heap allocation and/or type erasure at each await point. (It's possible to avoid the latter in continuation-based implementations, like C++ coroutines, but it's pretty hard.)
- deleted 3y ago
- OtomotO 3y ago> "Smart programmers try to avoid complexity. So, they see the extra complexity in async/await and question why it is needed. This question is especially pertinent when considering that a reasonable alternative exists in OS threads." Hm, yes, but is that complexity really avoided if it's in the language/runtime? Sure, it's not in your code and it will probably have way more extensive testing and maybe also people that thought about the problem for months and you're just trying to implement Business feature #31514 and not reinventing the wheel, but as someone who has been bitten by specific implementations like the one in Quarkus (not a language, granted), I must say: Complexity is there to stay (and occasionally bite you), even if it's hidden!
- delusional 3y agoI tend to agree, but an important note is that I've never seen an async/await system that didn't ALSO interact with OS threads. Async await is not an alternative to OS threads but rather an additional layer.
- OtomotO 3y agoI have heard about them in embedded systems where there are no OS threads, because there is no OS, but there is Async tasks and a scheduler
- mike_hearn 3y agoThat just means there's an OS but it's one a bit like Win 3.1, with fully cooperative multitasking.
- IshKebab 3y agoYou mean like the async/await systems in the two most popular languages in the world - JavaScript and Python? Async/await is definitely an alternative to user space threads, especially for IO bound tasks.
- WorldMaker 3y ago
- feverzsj 3y agoRust async/await is as bad as C++ coroutines. They lack one of the most important language counterparts: async drop/destructor.
- OtomotO 3y agoThat's the main gripe imo: Missing features that are not optional for certain problems where the solution would otherwise make perfect sense. I get the pragmatism to "better ship something that is 80% finished now, than wait for it to be 100% finished in some years", but with Rust's async/await it was released in 2018 and the more time passes, the more it looks like some sharp edges are here to stay.
- stevefan1999 3y agoBecause you want to avoid context switching cost. In fact, async/await/promise/future/Task is also closely related to fiber as opposed to thread at least in Windows, but fiber is provided from an OS level while async/promise/future/Task is provided by the language itself. You can switch to a different fiber without doing a context switch, just like how you use a state machine to switch to different job using async/promise/future/Task
- immibis 3y agoContext switching cost is mostly artificial. You save kernel/user transitions, but you can also have user-mode threads without transitions. What else are you saving by using coroutines instead of threads? You're not saving register swapping or cache warmup. You are saving stack allocations but those don't cost time.
- mrkeen 3y ago> A common refrain is that threads can do everything that async/await can, but simpler. > OS threads don’t require any changes to the programming model, which makes it very easy to express concurrency. I think these claims need a bit of justification, or else why write the article?
- gudzpoz 3y agoI think the author is confusing two things here: 1. User-space threads / Green threads 2. Structured concurrency The former one is an advantage of async/await, but is not unique to it (see Go or Java Loom for examples that involves no function coloring problem). And the latter one can be implemented with both OS threads and green threads (see Structured concurrency JEPS for Java [1]). [1] https://openjdk.org/jeps/462 https://openjdk.org/jeps/462
- darthrupert 3y agoMy possibly unpopular opinion is that async/await is a mistake as a first class programming construct. That functionality should 100% be in libraries for two reasons: 1) that way it won't infect the actual language in any way and 2) it will be more difficult to use so people will only reach for it if they really need it. Sync code and threads is the way to go for 99% of the cases where concurrency is needed. Rust handles most of the footguns of that combination anyway.
- nurple 3y agoCouldn't disagree more. In my experience, single-threaded event-loop driven async/await should be used for every possible concurrency need, with the complexity of multi-threaded concurrency being reserved for the rare cases it's needed. As auto-scaled services and FaaS began to become popular, I've found most any need for multithreaded programming almost wholly unnecessary.
- darthrupert 3y agoPerhaps an acceptable solution would be that all blocking things be async by convention, at least potentially. It's the conflict between sync and async that I find distasteful. The coloring problem. Zig does almost something like this.
- zokier 3y agoThe problem is that in Rust the main way of doing async/await (Tokio) is multithreaded by default. So you get all the problems of multithreading, and additionally all the problems with async/await. The discourse would be very different if the default choice of executor would have been thread-local.
- struant 3y agoSounds like a problem with Rust. Not a problem with async/await.
- nurple 3y agoYes, this is a big problem in my mind and something I mentioned in another comment. Not that multi-threaded async/await is wrong, but the conflation of two methods of concurrency causes a lot of problems you don't see in single-threaded event-loops archs. In fact it makes me feel like the other thread talking about how async/await in rust was mainly forced on the community to capture js devs seems a bit unlikely as the only thing similar between the two are the spelling of the keywords.
- Animats 3y agoAsync/await with one thread is simple and well-understood. That's the Javascript model. Threads let you get all those CPUs working on the problem, and Rust helps you manage the locking. Plus, you can have threads at different priorities, which may be necessary if you're compute-bound. Multi-threaded async/await gets ugly. If you have serious compute-bound sections, the model tends to break down, because you're effectively blocking a thread that you share with others. Compute-bound multi-threaded does not work as well in Rust as it should. Problems include: - Futex congestion collapse. This tends to be a problem with some storage allocators. Many threads are hitting the same locks. In particular, growing a buffer can get very expensive in allocators where the recopying takes place with the entire storage allocator locked. I've mentioned before that Wine's library allocator, in a .DLL that's emulating a Microsoft library, is badly prone to this problem. Performance drops by two orders of magnitude with all the CPU time going into spinlocks. Microsoft's own implementation does not have this problem. - Starvation of unfair mutexes. Both the standard Mutex and crossbeam-channel channels are unfair. If you have multiple threads locking a resource, doing something, unlocking the resource, and repeating that cycle, one thread will win repeatedly and the others will get locked out.[1] If you need fair mutexes, there's "parking-lot". But you don't get the poisoning safety on thread panic that the standard mutexes give you. If you're not I/O bound, this gets much more complicated. [1] https://users.rust-lang.org/t/mutex-starvation/89080 https://users.rust-lang.org/t/mutex-starvation/89080
- exfalso 3y agoYes, 100%. I've mostly only dealt with IO-bound computations, but the contention issues arise there as well. What's the point of having a million coroutines when the IO throughput is bounded again? How will coroutines save me when I immediately exhaust my size 10 DB connection pool? It won't, it just makes debugging and working around the issues harder and difficult to reason about.
- AstralStorm 3y agoThe debugging issue is bigger than what it seems. Use of async/await model in particular ends up with random hanged micro tasks in some random place in code that are very hard to trace back to the cause because they're dispersed potentially anywhere. Concurrency is also rather undefined, as are priorities most of the time. This can be partly fixed by labelling, which adds more complexity, but at least is explicit. Then the programmer needs to know what to label... Which they won't do, and Rust has no training wheels to help with concurrency. Threads, well you have well defined ingress and egress. Priorities are handled by the OS and to some degree fairness is usually ensured.
- exfalso 3y agoIt's interesting to see an almost marketing-like campaign to save face for async/await. It is very clear from my experience that it was not only a technical mistake, it also cost the community dearly. Instead of focusing on language features that are actually useful, the Rust effort has been sidetracked by this mess. I'm still very hopeful for the language though, and it is the best thing we've got at the moment. I'm just worried that this whole fight will drag on forever. P.S the AsyncWrite/AsyncRead example looks reasonable, but in fact you can do the same thing with threads/fds as long as you restrict yourself to *nix.
- logicchains 3y agoIt's not a technical mistake, it's a brilliant solution for when you need ultra-low-latency async code. The mistake is pushing it for the vast majority of use-cases where this isn't needed.
- surajrmal 3y agoThe reason it's pushed everywhere is because it only works well if the ecosystem uses it. If the larger rust ecosystem used sync code in most places, async await in rust would be unusable by large swaths of the community.
- vacuity 3y agoI think it wouldn't be so painful even as it's pervasive if the ergonomics were far better. Unfortunately, things are still far off on that front. Static dispatch for async functions recently landed in stable, though without a good way to bound the return type. Things like a better pinning API, async drop, interoperability and standardization, dynamic dispatch for async functions, and structured concurrency are open problems, some moving along right now. It'll be a process spanning years.
- speed_spread 3y agoIt's not pushed, it's pulled by the hype. The Rust community got onboard that train and started to async all the things without regards to actual need or consequences.
- jayd16 3y agoAnother discussion where people don't get async/await, can't fathom why you would want a concurrency mechanism on a single thread and assume no one needs it. UI programming, communication with the GPU, and cross runtime communication are good examples but I'm sure there are more. Threads, green or otherwise, don't work for those cases but async/await does.
- anonymoushn 3y agoIt's definitely important to be able to manage a bunch of tasks explicitly in a single thread. If there are any other possible implementations of language features that result in the same binary and cause less trouble for the language's users, let's chat about those :)
- mike_hearn 3y agoYou can easily use threads with GUIs and I've written a bunch of GUI apps in the past that exploited threads quite effectively.
- jayd16 3y agoWhat popular GUI framework is multi threaded?
- mike_hearn 3y agoDepends what you mean by multi-threaded. There are plenty of frameworks that have some support for doing work on background threads. Take JavaFX for example: 1. Rendering is done on a background thread in parallel with the app thread. 2. The class lib makes it easy to run background tasks that update the GUI to reflect progress and results. 3. You can construct widget trees on background threads and then pass them to the main UI thread only to attach them/make them visible. With libs like ReactFX you can take event streams from the UI, map over them to a worker thread or pool of threads, and pass the results back to the UI thread. A few years ago I did a video tutorial showing a few of these things, the code can be seen here: https://github.com/mikehearn/KotlinFPWebinar/blob/master/src/main/java/typeahead/UIMain.kt#L39 https://github.com/mikehearn/KotlinFPWebinar/blob/master/src... Basically this is a simple auto-complete widget. It takes a stream of values from a text field the user is typing in, reduces the frequency of changes (so if you type fast it doesn't constantly do work it's about to throw away) and then passes the heavy work of computing completions onto a background thread, then takes the results back onto the UI thread. val edits = EventStreams.valuesOf(edit.textProperty()).forgetful() val fewerEdits = edits.successionEnds(Duration.ofMillis(100)).filter { it.isNotBlank() } val inBackground = fewerEdits.threadBridgeFromFx(thread) val completions = inBackground.map { ngrams.complete(it) } val inForeground = completions.threadBridgeToFx(thread) val lastResult = inForeground.map { FXCollections.observableList(it) }.toBinding(FXCollections.emptyObservableList()) completionsList.itemsProperty().bind(lastResult) Now if you mean what UI libraries support writes to properties from arbitrary background threads without an explicit thread switch, I think the answer is only Win32 and Motif. But that's not a particularly important feature. JavaFX can catch the case if you access live UI from the wrong thread so such bugs are rarely an issue in practice. In particular it's easy to synchronize with the UI thread with a few simple helpers, e.g.: thread(name = "background") { while (true) { var speed = synchronizeWithUI { speedSlider.value } doSomeWork(speed) } }
- anacrolix 3y agoasync/await is syntax. Most of what people associate with it is actually the benefit of virtual threading. Asynchrony is how virtual threading is achieved in user space. Rust has direct native threading by default and can't do virtual threading without a runtime. Async/await makes the use of a runtime explicit. It could be possible to have both virtual and OS threading without special syntax but it would require a marker trait, algebraic effects or monads. Without those things you either need to choose user or OS level threads by default. If user level threads are the default, you need a runtime by default. (Even if that runtime is only for concurrency.). Go made that the default. Again, you don't need async/await to have virtual threading.
- tsimionescu 3y agoAsync/await is syntax, but it's not equivalent to virtual threads. It is special syntax for a certain pattern of writing concurrent (and possibly parallel) code: one in which you launch a concurrent operation specifically to get back one result. When you start an OS thread or a virtual thread, that thread can do anything. When you launch a task, it can only do one thing: return one result of the type you asked for. Async/await is perfect for operations that map well onto this structure. For example, most IO reads and writes fit well into this model - you trigger the IO operation to get back a specific result, then do some other work while it's being prepared, and when you need the result, you block until it's available. Other common concurrent operations don't map well on this at all - for example, if you want to monitor a resource to do something every time its state changes, then this is not well modeled at all as a single operation with a unique result, and a virtual or real thread would be a much better option. Also, having both virtual and OS threads doesn't need any special syntax. You just need two different functions for creating a thread - StartOSThread(func) and StartVirtualThread(func), and a similar split for other functions that directly interact with threads (join, cancel, etc), and a function for telling whether you are currently running in a virtual thread. Everything else stays the same. This is what Java is doing with Project Loom, I'm not speaking just in principle. The huge difficulty with virtual threads is implementing all blocking operations (IO, synchronization primitives, waits, etc) such that they use async OS primitives and yield to the virtual thread scheduler, instead of actually blocking the OS thread running the virtual thread.
- globular-toast 3y ago> I recall that the thing that made me stick with Rust is the Iterator trait. It blew my mind that you could make something an Iterator, apply a handful of different combinators, then pass the resulting Iterator into any function that took an Iterator. What languages did the author use before? Is this any different from the interface pattern seen in many other languages?
- KaiserPro 3y agoI think its partly down to what you grew up with. If you're of the JS (or python 3) generation then you're _probably_ more comfortable with async. However there are things where async seems to be more of a semantic fit for what youre doing. for example FastAPI is all async, and it makes sense. I started using it, because it scaled better than anything else. They have don't a nice job of making the interface as painless as possible. It almost doesn't feel like surprise goto. I do a lot of stream processing, so for me threads is a better fit. It scales well enough, and should I need to either go to multiprocessing (not great) or duplicate to a new stand-alone process, its fairly simple (Keeping everything message based also helps.) async/threads is _almost_ like shop bought coke vs sodastream. They are mostly the same, but have slightly different semantics.
- avodonosov 3y agoIssues with the article: 1. Only one example is given (web server), solved incorrectly for threads. I will elaborate below. 2. The question is framed as if people specifically want OS threads instead of async/await . But programmers want threads conceptually, semantically. Write sequential logic and don't use strange annotations like "async". In other words, if async/await is so good, why not make all functions in the language implicitly async, and instead of "await" just write normal function calls? Then you will suddenly be programming in threads. OS threads are expensive due to statically allocated stack, and we don't want that. We want cheap threads, that can be run in millions on a single CPU. But without the clumsy "async/await" words. (The `wait` word remains in it's classic sense: when you wait for an event, for another thread to complete, etc - a blocking operation of waiting. But we don't want it for function invocations). Back to #1 - the web server example. When timeout is implemented in async/await variant of the solution, using the `driver.race(timeout).await`, what happens to the client socket after the `race` signals the timeout error? Does socket remain open, remains connected to the client - essentially leaked? The timeout solution for threaded version may look almost the same, as it looks for async/await: `threaded_race(client_thread, timeout).wait`. This threaded_race function uses a timer to track a timeout in parallel with the thread, and when the timeout is reached it calls `client_thread.interrupt()` - the Java way. (The `Thread.interrupt()`, if thread is not blocked, simply sets a flag; and if the thread is blocked in an IO call, this call throws an InterruptedException. That's a checked exception, so compiler forces programmer to wrap the `client.read_to_end(&mut data)` into try / catch or declare the exception in the `handle_client`. So programmer will not forget to close the client socket).
- littlestymaar 3y ago> But programmers want threads conceptually, semantically. Write sequential logic and don't use strange annotations like "async". In other words, if async/await is so good, why not make all functions in the language implicitly async, and in stead of "await" just use normal function calls? Then you will suddenly be programming in threads. Some programmers do, but many want exactly the opposite as well. Most of the time I don't care if it's an OS blocking syscall or a non-blocking one, but I do care about understanding the control flow of the program I'm reading and see where there's waiting time and how to make them run concurrently. In fact, I'd kill to have a blocking/block keyword pair whenever I'm working with blocking functions, because they can surreptitiously slow down everything without you paying attention (I can't count how many pieces of software I've seen with blocking syscalls in the UI thread, leading to frustratingly slow apps!).
- nottorp 3y agoNo offense but you don't serve 1 million clients with 1 thread per client. So what this ends up as is describing the mechanism that Rust uses to hide the state machine you'd do by hand in an older language.
- ardies 3y agoJust tried changing my python code to use AA instead of the 2 threads I use and it just complicated the code more. I had to create a thread anyway to deal with the tkinter event loop issue. And then I still have to check inside the functions if the task is still active. But the worst part is converting all the stack of functions to to be async/await.
- bsaul 3y ago"Obviously, the embryonic web tried to solve this problem. The original solution was to introduce threading" Are we calling 1970's mainframes "the embryonic web" now ?
- immibis 3y agoWell, apparently they did consist of a lot of tangled threads...
- DaiPlusPlus 3y agoThey’re beautiful: https://www.embecosm.com/app/uploads/Cray_1_Wiring.jpg https://www.embecosm.com/app/uploads/Cray_1_Wiring.jpg
- littlestymaar 3y agoSpiders would be proud of the people who did this. They definitely deserve to be called “web”.
- mike_hearn 3y agoThat sentence is super confusing. The web has never supported multi-threading.
- paulsutter 3y agoIf you are doing anything complex you really want to use a polling loop. That’s how video games, avionics systems, industrial equipment, etc is programmed
- guappa 3y agoThat's what async/await abstracts internally.
- samsquire 3y agoThank you for this post, very interesting. Parallelism and async and multithreading is my hobby. I think nodejs and browsers made typeless but coloured async easily digested and understood by everyone. Promises for example. The libuv event loop used by nodejs is powerful for single threaded IO scalability, but I am looking for something that is multicore and scalable. I like thread-per-core architectures. The code written to an architecture has a high overhead in terms of mental understandability to new colleagues and barrier to transform it. Refactoring Postgres multiprocess implementation or nginx's worker model, it would be a major effort and undertaking. Firefox did a lot of work with Electrolysis Quantum to make it multiprocess. I was yesterday and recently trying to implement a lockfree work queue that any thread can pick up work and "steal" work from other threads. I have a multithreaded runtime I am building in C which is a "phaser", it does synchronization in bulk. So rather than mutexes, you have mailboxes which are queues.
- neonsunset 3y agoPlease. Stop posting about async/await here. It just gives space for everyone and their mother to post their superstitions and misconceptions about what is otherwise an elegant and extremely powerful model. The Rust implementation is great, so is the C# one. They have their own tradeoffs but I would never choose anything else, and 8 out of 10 developers who do disagree never stopped reading past "me sees await means no thread blocky" instead of focusing on structured concurrency patterns it enables. Hell, C# does not even need any fancy terms for this because it is this low ceremony. Worse languages, however, do require more effort so have to justify it with inventing words.
- sapiogram 3y ago> Please. Stop posting about async/await here. I agree with everything else you said, but that's all the more reason to post it imo. The misconceptions aren't going away just because you don't hear about them, and every time I hear the different arguments, my own understanding grows a little bit. It's annoying, but quite healthy.
- EVa5I7bHFq9mnYK 3y agoThreads are not useful for I/O bound code. Anyway the requests are getting serialized into a single queue within the network driver or a disk driver. Actually, they make things worse by 1) issuing multiple out-of-order request to the disk, and 2) wasting time and memory switching that thread context back and forth. For CPU-bound tasks, it makes sense to create multiple threads, but only up to amount of actual cores available, otherwise it makes things worse again by cache thrashing. Thread pool(s) + message queues is what worked best for me.
- gpderetta 3y agoAny semi-modern card or ssd provides multiqueue support in hardware. And even with a single queue, concurrency can help keeping a queue full.
- simpaticoder 3y agoAsync/await vs threads is yet another entry in the ~800 volume ongoing series "Where the Indirection Go?" In this case, you put concurrency inside the process (async) or outside the process (threads). Your CPU is, in both cases, constantly rotating between workloads, and that indirection can be inside or outside the process. Once you start caring about either human usability or fractional performance, the distinction matters. Otherwise, it doesn't.
- makerdiety 3y agoBecause asynchronous programming reduces to its equivalent in digital circuit design being contingent on analog circuit engineering decisions.
- runiq 3y agoSorry, not a native speaker here and I can't parse that sentence. Would you be so kind to dumb it down a little?
- deleted 3y ago[deleted]
- zaphar 3y agoThe complaints around async/await vs threads to my mind have not been that one is more or less complex than the other. It is that it bifurcates the ecosystem and one of them ends up being a second class citizen causing friction when you choose the wrong one for your project. While you can mix and match them it's hacky and inefficient when you need to. As it stands now the Rust ecosystem has decided that if you want to do anything involving IO you are stuck with an all async/await ecosystem. Since nearly everything you might want to do in Rust probably involves IO with very few exceptions that means for the most part you should probably ignore non-async libraries regardless of whether the rest of your application wants it to be async or not. There is a hypothetical world where Rust used abstractions that are even more composable than async/await, whose composability really wants everything else to be async/await too. If that had happened then I think most of the complaints would have disappeared.
- HideousKojima 3y ago>As it stands now the Rust ecosystem has decided that if you want to do anything involving IO you are stuck with an all async/await ecosystem. I mean C# works basically the same way, even through there are non-async options for IO using the async options basically forces you to be async all the way back to Main(). There are ways to safely call async methods from sync methods but they make debugging infinitely harder.
- zaphar 3y agoWell, yes. That doesn't mean it's not annoying though. It happens in every language that provides syntactic support for the distinction between async/await and non async. It's, I think, core to the syntactic and semantic abstractions that were popularized by Javascript.
- K0nserv 3y agoI agree with your diagnosis. It's what I concluded in my own Rust async blog post[0](which surely are mandatory now). It's even worse than bifurcating the ecosystem because even within async code it's almost always closely tied to the executor, usually Tokio. I talk about this as an extension to function colouring, adopting without.boats's three colour proposition with blue(non-IO), green(blocking-IO), and red(async-IO). In the extended model it's really blue, green, red(Tokio), purple(async-std), and orange(smol) etc. I find that the sans-IO pattern is the best solution to this problem. Under this pattern you isolate all blue code and use inversion of control for I/O and time. This way you end up with the core protocol logic being unaware of IO and it becomes simple to wrap it in various forms of IO. 0: https://hugotunius.se/2024/03/08/on-async-rust.html https://hugotunius.se/2024/03/08/on-async-rust.html
- germandiago 3y agoIterators blew the writer's mind... Most languages have iterators, come on... Java, C#, Python, C++, only to name a few
- assbuttbuttass 3y agoI have to say, I am not convinced by this article that async composes better. The nice thing about green threads/fibers is you can make concurrency an internal detail: a function might spawn threads internally, or block, but the caller is free to use it as any other normal function, including passing it to a map() or filter() combinator. By contrast, async forces the caller to acknowledge that it's not a regular function, and async functions don't compose at all with normal code. You have to write async-only versions of map() filter() and any other combinators. Maybe async composes better with other async, but with threads, you can just compose with any other existing code.
- pornel 3y agoThe big one not mentioned is cancellation. It's very easy to cancel any future. OTOH cancellation with threads is a messy whack-a-mole problem, and a forced thread abort can't be reliable due to a risk of leaving locks locked. In Rust's async model, it's possible to add timeouts to all futures externally. You don't need every leaf I/O function to support a timeout option, and you don't need to pass that timeout through the entire call stack. Combined with use of Drop guards (which is Rust's best practice for managing in-progress state), it makes cancellation of even large and complex operations easy and reliable.
- jeffbee 3y agoCancellation is not worth worrying over in my experience. If an op is no longer useful, then it is good enough if that information eventually becomes visible to whatever function is invoked on behalf of the op, but don't bother checking for it unless you are about to do something very expensive, like start an RPC.
- pornel 3y agoIt's been incredibly important for me in both high-traffic network back-ends, as well as in GUI apps. When writing complex servers it's very problematic to have requests piling up waiting on something unresponsive (some other API, microservice, borked DNS, database having a bad time, etc.). Sometimes clients can be stuck waiting forever, eventually causing the server to run out of file descriptors or RAM. Everything needs timeouts and circuit breakers. Poorly implemented cancellation that leaves some work running can create pathological situations that eat all CPU and RAM. If some data takes too long to retrieve, and you time out the request without stopping processing, the client will retry, asking for that huge slow thing again, piling up another and another and another huge task that doesn't get cancelled, making the problem worse with each retry. Often threading is mixed with callbacks for returning results. The un-cancelled callbacks firing after the other part of the application aborted an operation can cause race conditions, by messing up some state or being misattributed to another operation.
- jeffbee 3y ago
- naasking 3y agoAsync/await have one property that is slightly more difficult to achieve with threads, but isn't really discussed much and hasn't been used much to my knowledge: because context is captured fairly explicitly as records, this makes it easy to persist that context to durable storage and/or migrate them to other machines without any additional runtime machinery. Achieving this with threads is possible but requires additional runtime support. Basically it requires scanning stack frames, which the garbage collector already does, so you're extending the GC to record stack in a separate place.
- cryptonector 3y agoAs well, this functions as state compression by the developer. This is what makes async/await and CPS so much better than thread-per-client: smaller memory footprint!
- cryptonector 3y ago> Why choose async/await over threads? I assume the intended question is: why use async/await over thread-per-client designs? Easy: because async/await allows you to more easily compress the memory footprint of client/request/work state because there is no large stack to allocate to each client/whatever. One can still use threads (or processes) with async/await so as to use as many CPUs as possible while still benefiting from the state compression mentioned above. State compression -or, rather, not exploding the state- is critical for performance considering how slow memory is nowadays. Async/await, manual continuation passing style (CPS) -- these are great techniques for keeping per-client memory footprint way down. Green threads with small stacks that don't require guard pages to grow are less efficient, but still more efficient than threads. Threads are inefficient because of the need to allocate them a large stack, and to play MMU games to set up those stacks and their guard pages. Thread-per-client simply does not scale like async/await and CPS.
- the__alchemist 3y agoI am suspicious that Async, slowly taking over the library ecosystems, will be what drives me away from Rust. Over the past year, it's taken over embedded. I was previously worried about excessive use of generics, but this was, gradually over the past ~2 years replaced with Async. The std-lib HTTP ecosystem has already gone almost full-async. I was able to spin up a simple web server without it using the rouille library recently. I chose this one because it was one of two options that wasn't Async. I think I may be the only person who A: Enjoys rust for embedded targets (Eg Cortex-M), but doesn't enjoy Async. The easy path is to go with the flow and embrace Async in those domains (Embedded, web backends). I say: No. Threads, interrupts, DMA, etc are easier to write and maintain, and critically, don't split code into two flavors. You don't just add an Async library to your program: It attempts to take your program over. That is the dealbreaker.
- littlestymaar 3y ago> The easy path is to go with the flow and embrace Async in those domains (Embedded, web backends). I say: No. Why are you making a principle of not using async though? It's just one particular feature of the language, there's nothing wrong in not being particularly enthusiast about it, but entirely refusing to use it sounds really weird to me. Every JavaScript junior use async right now, it's not as it is was a particularly difficult programming paradigm to learn. Sure async in rust is a bit more complicated than in JS due to the fact it's lower level, but it's not worse than anything else in Rust! > Threads, interrupts, DMA, etc are easier to write and maintain, Not really. I've no doubt that you master them better, but there's no reason to think you won't be able to be as proficient with async. And in fact, async is just strictly more powerful than sync code: you can do exactly what you're doing right now (just write await), plus you get a few features for free (concurrency, cancelation). There's a reason why it's popular actually! > and critically, don't split code into two flavors In practice, there isn't either. No if you're writing an application, it's only when you're writing a library and you want to support explicitly the use case of people like you that have moral issues with async.
- galangalalgol 3y agoIt can be frustrating to: People who target wasm People who target embedded And superfluous to: People who have compute bound problems People with one incoming and one outgoing connection at all times.
- unstruktured 3y agoTo me, implementing an async api using algebraic effects is the end game. Gets rid of the need of an async keyword and no monads required!
- mattgreenrocks 3y agoAsync/await makes more sense in some languages than others. For most PLs, algebraic effects look ideal. Just seems like we're figuring out how to talk about them to users and what the UX should be.
- debugnik 3y ago> Gets rid of the need of an async keyword You still need an async effect handler though, which is effectively the same syntax wise.
- jupp0r 3y agoOne of the main benefits of async/await in Rust is that it can work in situations where you don't even have threads or dynamic memory. You can absolutely use it to write very concise code that's waiting on an interrupt on your microcontroller to have read some data coming in over I2C from some buffer. It's a higher level abstraction that allows your code to use concurrency (mostly) without having tons of interactions with the underlying runtime. Every major piece of software that I have worked on has implemented this in one form or another (even in non-modern C++ where you don't have any coroutine concepts, Apple's grand central dispatch, intel's Thread Building Blocks, etc). If you don't then your business logic will either be very imperformantly block on IO, have a gazillion of threads that make development/debugging a living hell, or be littered with implementation details of the underlying runtime or a combination of all 3. If you don't use existing abstractions in the language (or through some library), you will end up building them yourselves, which is hard and probably overall inferior to widely used ones (if there are any). I have done so in the past, see https://github.com/goto-opensource/asyncly https://github.com/goto-opensource/asyncly for C++.
- shmerl 3y agoI always understood async/await as expression of concurrency and threads as expression of parallelism. Which aren't the same thing. Concurrency breaks computation into chunks that can be interleaved even on a single CPU advancing computation concurrently overall but not necessarily physically at the same time for each chunk. Parallelism on the other hand breaks computation into chunks that literally can run in parallel on different CPUs. You can even combine both in some fashion.
- trevor-e 3y agoAs an observer who has used async/await in other languages (Javascript, Swift) I'm really confused why this is such a contentious topic in Rust. swift-concurrency has had its issues as well, but almost everyone I know finds it way more ergonomic and useful than how things were done before. As someone currently learning Rust, is there something particular about the language compared to how Swift did it? Outside of HN I would not know this is such a controversial topic.
- anybodyz 3y agoI worked with a fairly large code base which leveraged threads to avoid having to use callbacks in a C++ code base. This allowed the engineers to use the more familiar linear programming style to make blocking network calls which would stall one thread, while others were unblocked to proceed as responses were received. The threads would interoperate with each other using a similar blocking rpc system. But what ended up happening was that the system would tend to block unnecessarily across all threads, particularly in cross thread communication. These were due to many external network calls being performed sequentially when they could have been performed in parallel, and likewise cross-thread communication constantly blocking due to sequential calls. The end result was a system who'd performance was just fundamentally hobbled by threads that were constantly waiting on other threads or series of external calls, and it was very difficult to understand the nature of the gordian knot of blocking behavior that was causing these issues. The main problem with using threads is that the moment you introduce a different cpu into the mix you need to deal with synchronization primitives around your state, and this means that engineers will use fewer threads (at least in our case) than necessary to reduce the complexity of the synchronization work needed to be done which means that you lose the advantage of asynchronous parallelism. Or at least that is what happened in this particular case. The cost of engineering synchronization for async/await is zero, because this parallelism happens on a single thread. Since the cpu "work" to be done for async/io is relatively small, this argues for using single threaded 'callback' style solutions where you maximize the amount of parallelism and decrease the amount of potential blocking as well as minimizing the complexity of thread synchronization as much as possible. In cases where you want to leverage as many cpu's as possible, it's often the case that you can better benefit from cpu parallelism by simply forking your process on multiple cores.
- aidenn0 3y agoELI5: If you have to modify the language anyways (to add async/await) why not just have green threads that wrap NBIO with seemingly blocking calls?
- dragonwriter 3y ago> A common refrain is that threads can do everything that async/await can, but simpler This is the first time I've heard that, and given how many languages have added async/await after threading support, I don't think it is really a broad consensus.
- alexxys 3y agoThe article talks a lot about async/await but fails to clearly state the main advantage of async code over threads. Async code in general (not only in Rust) allows a server to process thousands of client connections concurrently with minimal latency in a single thread. Even if each client request needs several seconds to process it (assuming the processing is IO-bound). One thread (or more generally, a small number of threads) is much cheaper resource-wise than thousands of threads (in a thread per client scenario).
- zac23or 3y agoAsync and Thread are of different abstraction levels[1]. The easier to use, the higher the abstraction level. Rule of thumb: use the highest level of abstraction for your case. Of course, the Async implementation can be atrocious or in conflict with something already used by your code, forcing you to use a lower level of abstraction. [1]https://blog.codinghorror.com/the-wrong-level-of-abstraction/ https://blog.codinghorror.com/the-wrong-level-of-abstraction...
- tison 3y agoDays before I'd prefer manipulating between threads manually, because I used to do it for years. But as it goes much mature the async-await especially with stackless coroutine implementation, writing in async-await is more pleasure than handling threads. Handmade futures can have some pitfalls, like this blog wrote: https://blog.waynest.com/2022/12/async-cancellation/ https://blog.waynest.com/2022/12/async-cancellation/ However, I don't like Rust's trend of making global, implicit async runtime, especially with #[xxx:main]. Runtime should be specified clearly, like Netty channel must explicitly bind to an EventLoopGroup. And Java's CompletableFuture without specifying a concrete executor causes many issues already.
- fennecfoxy 3y agoImo, async/await is useful in that it's a simpler approach - developers don't need to worry about threading and all its inherent problems which for the large majority of web and semi-web related tasks is completely fine; the same reason we use languages with GC; no point managing your own memory for most tasks. Someone writing a driver or something is obviously going to use proper threading and deal with all of the intricacies and gotchas.