3 ms·

> how can you have one runtime that is multi-threaded and one runtime that is not, and expect to be able to seamlessly interchange them? I feel like I do this

by saurik 3y ago

> how can you have one runtime that is multi-threaded and one runtime that is not, and expect to be able to seamlessly interchange them? I feel like I do this in C++ right now without issue? I routinely mix/match coroutines from multiple runtimes, including one I built myself (which theoretically might could be multithreaded but very much right now is not and I know I rely on that still), one from cppcoro (which is a bit broken--I filed a bug with a detailed analysis, but it was never fixed--so I can only use a few parts that happen to add a lot of value), and one from boost asio (which is very much multithreaded and was a somewhat-impressive retrofit onto a more abstract design purely involving callbacks); I also effectively have a fourth, as another library I am using--libwebrtc--maintains its own I/O thread paradigm, and I have chosen to reinterpret a number of its delegation callbacks into coroutines. It involved some trivial adapters in a few places, but I developed those years ago and have long since forgotten as it all works so easily to willy-nilly co_await anything I want from wherever I am... is this really so difficult?

thinkharderdev 3y ago
In rust it is yes because rust statically guarantees that you don't have data races (in safe rust at least). So you have marker traits `Send` and `Sync` which indicate that a type can be sent between threads (Send) or shared between threads (Sync) safely. So for a multi-threaded executor which can scheduler tasks on different threads when they resume has to make sure futures are `Send` whereas a single-threaded executor does not have that constraint.
- fiddlerwoaroof 3y ago
  Aren’t the same data races possible in async without threads? As soon as you suspend one task and start another, you have the problem that the currently running task can break the invariants of the suspended one, regardless of whether you’re doing a single-threaded event loop or threads running in parallel.
  - eximius 3y ago
    I think you'd need to be violating some other rule of rust to do that. e.g., single mutable access.
  - fnordpiglet 3y ago
    Rust isn’t functional, so if you have state that’s shared in some way you can’t expect it to be immutable unless you manage it immutably. However you are assured that you won’t need to worry about thread safety and reentrant code in rust because you are guaranteed the same memory won’t be modified or modified/read at the same time by two threads. Obviously in single threaded asynchronous code this doesn’t happen anyways. That said, if you don’t use shared state that allows multiple borrows, you won’t see state changing between futures even in the single threaded cases due to the ownership model of rust.
    - fiddlerwoaroof 3y ago
      Yeah, this was more or less my impression.
  - skybrian 3y ago
    In a single-threaded system, you only need to worry about concurrency when there’s an await keyword. Everywhere else, it’s as if you have an exclusive lock. Any functions that aren’t async can be treated as atomic. This makes it much easier to reason about concurrency. With a multithreaded system, async or not, you have to worry about the concurrency issues that come up when sharing data between multiple threads, because that’s what you’re doing. It’s odd how Rust ended up with the worst of both worlds by default. I think people got overconfident because Rust otherwise handles multithreading so well.
    - fiddlerwoaroof 3y ago
      > In a single-threaded system, you only need to worry about concurrency when there’s an await keyword. Everywhere else, it’s as if you have an exclusive lock Except that async code written this way in JavaScript/Typescript often ends up being subtly broken by evolution in where the awaits occur as the software is maintained. IMO, it’s generally better to design async code with a shared-nothing mentality anyways.
- saurik 3y ago
  This honestly doesn't sound like a problem as such types fall into one of two categories: ones which need to execute on one thread--in which case resuming them should always resume on their native runtime as they are thread-locked: I already have to deal with this as I adapt between the runtimes in C++ and it simply isn't a concern--and ones whose storage in virtual memory are somehow fundamentally locked to a specific CPU core and I honestly have never myself coded one of these despite having done some extremely low-level development. Like, here: if I am in my single-threaded runtime and I await something on a different runtime with a billion threads, MY continuation does NOT need to be able to resume on any of those threads as it CAN'T. To achieve "seamless interoperability" I just need to be able to await the other routine and resume when it completes, not somehow make the two runtimes merge into one unified one and violate their constraints. The ONLY data from my coroutines which should end up on a different thread is what I explicitly pass to the routine, not my continuation.
  - thinkharderdev 3y ago
    > whose storage in virtual memory are somehow fundamentally locked to a specific CPU core There are some pretty common reasons why a future in not Send: 1. It is reliant on some thread-local state in which case you can't move it to another thread 2. It uses something which relies of being single threaded for sounds. An example would be `Rc` the standard reference counted pointer in the std. It uses a `usize` for the refcount so it is not safe that have two `Rc` for the same data on different threads. If you need a reference counted pointer that is thread safe you need to use `Arc` which uses an `AtomicUsize` for the ref count and so is Send. > I just need to be able to await the other routine and resume when it completes, not somehow make the two runtimes merge into one unified one and violate their constraints. The ONLY data from my coroutines which should end up on a different thread is what I explicitly pass to the routine, not my continuation. Sure, and you could do this in Rust now perfectly fine. Spawn a future on a separate runtime (or a CPU intensive task on a regular thread) and await the result on the current runtime. But by default what happens whenever you hit an `await` is that the coroutine is suspended and goes onto the runtime's run queue until it is woken back up and gets rescheduled. In Tokio's multi-threaded runtime it can be rescheduled on next wake on any worker thread so it must be `Send`. If you use the single threaded tokio runtime there is only one thread so it doesn't need to be `Send`. And even in the multi-threaded tokio runtime you can still spawn tasks that are pinned to the current worker thread using LocalSet. In writing application code this is (to me at least) mostly a non-issue. Most futures will be Send anyway so the Send bound is not a big deal. But if you do have something that is not Send then you can always use LocalSet to spawn it. The issue I think is really in writing library code where you start to have to add Send bounds everywhere so it jives with multi-threaded runtimes. Like say you have a trait with a method that returns a `Stream` but the concrete type of the `Stream` is not important as long as it produces the required output. So you have ``` trait MakeThingStream { fn make_it(&self) -> Box<dyn Stream<Item = Thing>>; } ``` Well now all the compiler knows is that the output implements `Stream<Item = Thing>`. But this may not be Send so you'll get compiler errors if you try to use this in a multi-threaded runtime. So you add Send/Sync bounds: ``` trait MakeThingStream { fn make_it(&self) -> Box<dyn Stream<Item = Thing> + Send + Sync>; } ``` Great, now it plays nicely with multi-threaded runtimes but even if it's being used in a single-threaded runtime you still require the Send/Sync bounds.
insanitybit 3y ago
A lot of these complaints like "you need to have Send + Sync + 'static" and "oh no you need an arc or mutex" are identical problems in C++, except in C++ it's totally unsafe if you forego those.
- saurik 3y ago
  I am under the maybe-totally-wrong impression that people are saying that the runtimes are incompatible, not that you merely need to think harder about the scoping rules and type traits; do I misunderstand what is going on?
  - insanitybit 3y ago
    Oh, with regards to incompatibility, that only happens if you use `spawn`. I don't know what that would look like in C++.

by saurik 3y ago

thinkharderdev 3y ago

In rust it is yes because rust statically guarantees that you don't have data races (in safe rust at least). So you have marker traits `Send` and `Sync` which indicate that a type can be sent between threads (Send) or shared between threads (Sync) safely. So for a multi-threaded executor which can scheduler tasks on different threads when they resume has to make sure futures are `Send` whereas a single-threaded executor does not have that constraint.

fiddlerwoaroof 3y ago
Aren’t the same data races possible in async without threads? As soon as you suspend one task and start another, you have the problem that the currently running task can break the invariants of the suspended one, regardless of whether you’re doing a single-threaded event loop or threads running in parallel.
- eximius 3y ago
  I think you'd need to be violating some other rule of rust to do that. e.g., single mutable access.
- fnordpiglet 3y ago
  Rust isn’t functional, so if you have state that’s shared in some way you can’t expect it to be immutable unless you manage it immutably. However you are assured that you won’t need to worry about thread safety and reentrant code in rust because you are guaranteed the same memory won’t be modified or modified/read at the same time by two threads. Obviously in single threaded asynchronous code this doesn’t happen anyways. That said, if you don’t use shared state that allows multiple borrows, you won’t see state changing between futures even in the single threaded cases due to the ownership model of rust.
  - fiddlerwoaroof 3y ago
    Yeah, this was more or less my impression.
- skybrian 3y ago
  In a single-threaded system, you only need to worry about concurrency when there’s an await keyword. Everywhere else, it’s as if you have an exclusive lock. Any functions that aren’t async can be treated as atomic. This makes it much easier to reason about concurrency. With a multithreaded system, async or not, you have to worry about the concurrency issues that come up when sharing data between multiple threads, because that’s what you’re doing. It’s odd how Rust ended up with the worst of both worlds by default. I think people got overconfident because Rust otherwise handles multithreading so well.
  - fiddlerwoaroof 3y ago
    > In a single-threaded system, you only need to worry about concurrency when there’s an await keyword. Everywhere else, it’s as if you have an exclusive lock Except that async code written this way in JavaScript/Typescript often ends up being subtly broken by evolution in where the awaits occur as the software is maintained. IMO, it’s generally better to design async code with a shared-nothing mentality anyways.
saurik 3y ago
This honestly doesn't sound like a problem as such types fall into one of two categories: ones which need to execute on one thread--in which case resuming them should always resume on their native runtime as they are thread-locked: I already have to deal with this as I adapt between the runtimes in C++ and it simply isn't a concern--and ones whose storage in virtual memory are somehow fundamentally locked to a specific CPU core and I honestly have never myself coded one of these despite having done some extremely low-level development. Like, here: if I am in my single-threaded runtime and I await something on a different runtime with a billion threads, MY continuation does NOT need to be able to resume on any of those threads as it CAN'T. To achieve "seamless interoperability" I just need to be able to await the other routine and resume when it completes, not somehow make the two runtimes merge into one unified one and violate their constraints. The ONLY data from my coroutines which should end up on a different thread is what I explicitly pass to the routine, not my continuation.
- thinkharderdev 3y ago
  > whose storage in virtual memory are somehow fundamentally locked to a specific CPU core There are some pretty common reasons why a future in not Send: 1. It is reliant on some thread-local state in which case you can't move it to another thread 2. It uses something which relies of being single threaded for sounds. An example would be `Rc` the standard reference counted pointer in the std. It uses a `usize` for the refcount so it is not safe that have two `Rc` for the same data on different threads. If you need a reference counted pointer that is thread safe you need to use `Arc` which uses an `AtomicUsize` for the ref count and so is Send. > I just need to be able to await the other routine and resume when it completes, not somehow make the two runtimes merge into one unified one and violate their constraints. The ONLY data from my coroutines which should end up on a different thread is what I explicitly pass to the routine, not my continuation. Sure, and you could do this in Rust now perfectly fine. Spawn a future on a separate runtime (or a CPU intensive task on a regular thread) and await the result on the current runtime. But by default what happens whenever you hit an `await` is that the coroutine is suspended and goes onto the runtime's run queue until it is woken back up and gets rescheduled. In Tokio's multi-threaded runtime it can be rescheduled on next wake on any worker thread so it must be `Send`. If you use the single threaded tokio runtime there is only one thread so it doesn't need to be `Send`. And even in the multi-threaded tokio runtime you can still spawn tasks that are pinned to the current worker thread using LocalSet. In writing application code this is (to me at least) mostly a non-issue. Most futures will be Send anyway so the Send bound is not a big deal. But if you do have something that is not Send then you can always use LocalSet to spawn it. The issue I think is really in writing library code where you start to have to add Send bounds everywhere so it jives with multi-threaded runtimes. Like say you have a trait with a method that returns a `Stream` but the concrete type of the `Stream` is not important as long as it produces the required output. So you have ``` trait MakeThingStream { fn make_it(&self) -> Box<dyn Stream<Item = Thing>>; } ``` Well now all the compiler knows is that the output implements `Stream<Item = Thing>`. But this may not be Send so you'll get compiler errors if you try to use this in a multi-threaded runtime. So you add Send/Sync bounds: ``` trait MakeThingStream { fn make_it(&self) -> Box<dyn Stream<Item = Thing> + Send + Sync>; } ``` Great, now it plays nicely with multi-threaded runtimes but even if it's being used in a single-threaded runtime you still require the Send/Sync bounds.

insanitybit 3y ago

A lot of these complaints like "you need to have Send + Sync + 'static" and "oh no you need an arc or mutex" are identical problems in C++, except in C++ it's totally unsafe if you forego those.

saurik 3y ago
I am under the maybe-totally-wrong impression that people are saying that the runtimes are incompatible, not that you merely need to think harder about the scoping rules and type traits; do I misunderstand what is going on?
- insanitybit 3y ago
  Oh, with regards to incompatibility, that only happens if you use `spawn`. I don't know what that would look like in C++.