4 ms·

There are some fundamental assumptions in the Rust async system: - The program is mostly I/O bound. - All tasks have equal priority. If your program isn't li

by Animats 3mo ago

There are some fundamental assumptions in the Rust async system: - The program is mostly I/O bound. - All tasks have equal priority. If your program isn't like that, the Tokio model is a bad match to the problem. Real time control is not like that. MMO and metaverse game programs are not like that. Most web stuff is, but that's a special case. A big special case, but a special case.

RealityVoid 3mo ago
Real time control can be like that as long as the computing parts are clearly bound within your performance requirement.
LAC-Tech 3mo ago
There's a few runtimes for IO bound work loads; monoio from bytedance comes to mind.
oersted 3mo ago
To be fair, the Rust async model itself was intentionally designed not to be prescriptive in the way you describe. You can build, and there exists, different task executors that can handle things like priority and many other execution models. Async is just a way to describe a tree of concurrent tasks that may depend on (wait on) each other at certain points. It is mostly declarative. Tokio has taken over as the default choice, but there's a reason why it's not part of the standard library, it is not meant to be the only choice.
- VorpalWay 3mo ago
  Outside of Embassy in embedded, tokio is the only realistic choice though, because it is likely that any third party async crate has a dependency on it already. Yes, smol, monio, glommio etc exist, but they are marginalised (and as far as I can tell they don't really help that much with mixed IO / compute workloads). In fact, async/await in Rust falls apart with a mixed IO / compute workload since scheduling is cooperative. As soon as you want preemption (most of the time for what I do), it is not the right choice. Seeing how embassy (embedded async rust) handles preemption reinforces this: it uses a separate scheduler per preemption level. This works fine, but is a bit clunky. Basically you are at this point just using async to help write a state machine per preemptive thread, which can be useful for some code patterns (in particular those common in embedded, where you are often waiting for IO). But to talk between threads you are back to channels, mutexes etc.
  - oersted 3mo ago
    Yes I do agree. It's not that Tokio has taken over just due to community momentum. There's a certain subtle lock-in that happens due to how the Rust type system and dependency system work together. It's a hard problem to address. Regarding mixed IO/compute and preemptive scheduling, well that's what threads are. They are not as lightweight, but they are there and they are quite ergonomic to use in Rust. I could even imagine an async executor that simply starts a thread per task, so that you can keep the nice syntax, but that's obviously not ideal from a performance standpoint. Tokio already dispatches tasks to a thread pool, it is not completely cooperative, it's the sensible way to do this. And there's always tokio::spawn_blocking too. PS: Actually, it is underrated how well designed classic threading is in Rust. It was just going out of vogue when they did it, but they addressed so much of the complexity that was around for years in Java, C++ and the like. They did actually mostly achieve the holy grail of fearless concurrency and with a more ergonomic design.
    - imtringued 3mo ago
      I don't really know much about tokio ecosystem lock-in, but async is fundamentally an ecosystem split. In principle nothing stops you from running computationally expensive code inside an async context, it just needs to be designed to support self pre-emption. That means if you have an async JSON parser it needs to await inside long running loops or recursive functions.
  - random17 3mo ago
    How do you properly mix IO/compute (in any language)? In Rust what I’ve done in the past is have two Tokio runtimes, one for IO and one for compute. I know you can also use Rayon but the abstractions are not always flexible/convenient enough. But in either case the boundary between the two types of async work is never easy to cross.
    - quietbritishjim 3mo ago
      In Python Trio, you would have a thread pool for each type of computation (potentially a "pool" of 1 thread shared between all tasks if that works for your application). You would await trio.to_thread.run_sync(...) [1] (pass it a normal non-async function, and a token representing the thread pool). This is a pretty simple wrapper that takes care of queuing the work to the thread pool, waiting for it to complete and for a message to be passed back to the Trio thread. It works for simple situations. It certainly preserves backpressure, although a slow CPU task can impact other users of the same thread pool. [1] https://trio.readthedocs.io/en/stable/reference-core.html#trio.to_thread.run_sync https://trio.readthedocs.io/en/stable/reference-core.html#tr...
- nextaccountic 3mo ago
  Unfortunately, rather than exposing a common interface for many executors, the Rust async ecosystem went with a Tokio monoculture Because of that, many libraries will depend on Tokio rather than working out of box with those different executors. Things like libraries that define protocols, etc. This creates a strong disincentive for anyone to mess with other executors Worse yet: with no common interface, some libraries will have feature flags to support a bunch of executors, which increases the maintenance load. Some still keep around support for dead executors like async_std while skipping support for newer executors like glommio Anyway nowadays there are some abstraction crates in crates.io but it is too little, too late. Tokio already has an insurmountable lock in. The only thing that can start to reverse that is the stdlib itself to offer cross-executor APIs

by Animats 3mo ago

RealityVoid 3mo ago

Real time control can be like that as long as the computing parts are clearly bound within your performance requirement.

LAC-Tech 3mo ago

There's a few runtimes for IO bound work loads; monoio from bytedance comes to mind.

oersted 3mo ago

To be fair, the Rust async model itself was intentionally designed not to be prescriptive in the way you describe. You can build, and there exists, different task executors that can handle things like priority and many other execution models. Async is just a way to describe a tree of concurrent tasks that may depend on (wait on) each other at certain points. It is mostly declarative. Tokio has taken over as the default choice, but there's a reason why it's not part of the standard library, it is not meant to be the only choice.

VorpalWay 3mo ago
Outside of Embassy in embedded, tokio is the only realistic choice though, because it is likely that any third party async crate has a dependency on it already. Yes, smol, monio, glommio etc exist, but they are marginalised (and as far as I can tell they don't really help that much with mixed IO / compute workloads). In fact, async/await in Rust falls apart with a mixed IO / compute workload since scheduling is cooperative. As soon as you want preemption (most of the time for what I do), it is not the right choice. Seeing how embassy (embedded async rust) handles preemption reinforces this: it uses a separate scheduler per preemption level. This works fine, but is a bit clunky. Basically you are at this point just using async to help write a state machine per preemptive thread, which can be useful for some code patterns (in particular those common in embedded, where you are often waiting for IO). But to talk between threads you are back to channels, mutexes etc.
- oersted 3mo ago
  Yes I do agree. It's not that Tokio has taken over just due to community momentum. There's a certain subtle lock-in that happens due to how the Rust type system and dependency system work together. It's a hard problem to address. Regarding mixed IO/compute and preemptive scheduling, well that's what threads are. They are not as lightweight, but they are there and they are quite ergonomic to use in Rust. I could even imagine an async executor that simply starts a thread per task, so that you can keep the nice syntax, but that's obviously not ideal from a performance standpoint. Tokio already dispatches tasks to a thread pool, it is not completely cooperative, it's the sensible way to do this. And there's always tokio::spawn_blocking too. PS: Actually, it is underrated how well designed classic threading is in Rust. It was just going out of vogue when they did it, but they addressed so much of the complexity that was around for years in Java, C++ and the like. They did actually mostly achieve the holy grail of fearless concurrency and with a more ergonomic design.
  - imtringued 3mo ago
    I don't really know much about tokio ecosystem lock-in, but async is fundamentally an ecosystem split. In principle nothing stops you from running computationally expensive code inside an async context, it just needs to be designed to support self pre-emption. That means if you have an async JSON parser it needs to await inside long running loops or recursive functions.
- random17 3mo ago
  How do you properly mix IO/compute (in any language)? In Rust what I’ve done in the past is have two Tokio runtimes, one for IO and one for compute. I know you can also use Rayon but the abstractions are not always flexible/convenient enough. But in either case the boundary between the two types of async work is never easy to cross.
  - quietbritishjim 3mo ago
    In Python Trio, you would have a thread pool for each type of computation (potentially a "pool" of 1 thread shared between all tasks if that works for your application). You would await trio.to_thread.run_sync(...) [1] (pass it a normal non-async function, and a token representing the thread pool). This is a pretty simple wrapper that takes care of queuing the work to the thread pool, waiting for it to complete and for a message to be passed back to the Trio thread. It works for simple situations. It certainly preserves backpressure, although a slow CPU task can impact other users of the same thread pool. [1] https://trio.readthedocs.io/en/stable/reference-core.html#trio.to_thread.run_sync https://trio.readthedocs.io/en/stable/reference-core.html#tr...
nextaccountic 3mo ago
Unfortunately, rather than exposing a common interface for many executors, the Rust async ecosystem went with a Tokio monoculture Because of that, many libraries will depend on Tokio rather than working out of box with those different executors. Things like libraries that define protocols, etc. This creates a strong disincentive for anyone to mess with other executors Worse yet: with no common interface, some libraries will have feature flags to support a bunch of executors, which increases the maintenance load. Some still keep around support for dead executors like async_std while skipping support for newer executors like glommio Anyway nowadays there are some abstraction crates in crates.io but it is too little, too late. Tokio already has an insurmountable lock in. The only thing that can start to reverse that is the stdlib itself to offer cross-executor APIs