4 ms·

It's exciting to see another thread-per-core async runtime for Rust. It's truly understated how difficult the Send + Sync requirements in Tokio are for writing

It's exciting to see another thread-per-core async runtime for Rust. It's truly understated how difficult the Send + Sync requirements in Tokio are for writing regular code. It's typically rare for async tasks in Tokio to be used across two threads simultaneously, but now all of your data must be Send+Sync. Plus, Tokio is unique in that it's one of the very few runtimes in existence that's work-stealing (meaning the task can move off its original thread). Most other runtimes in other languages do not have that requirement, meaning you can use traditional Cell/RefCell/Rc instead of their slower, atomic variants. Right now, the best you can do is write a thread pool that spawns a Tokio LocalSet to run !Send futures. In fact, this is what Actin-web does to achieve its crazy performance. Web requests finish so quickly you rarely need work-stealing to achieve good performance, and often the cost of atomics/stealing is greater than the performance gain.

verdagon 5y ago
I've been thinking about how one could get around the Send + Sync requirements, it's a fascinating conundrum, from a language design standpoint. If we didn't have the Send + Sync requirements, and multiple async functions are running concurrently on the same thread, and multiple of them locked the same RefCell, might that cause a panic?
- masklinn 5y ago
  > If we didn't have the Send + Sync requirements, and multiple async functions are running concurrently on the same thread, and multiple of them locked the same RefCell, might that cause a panic? Hopefully.
asymptotic 5y ago
> the best you can do is write a thread pool that spawns a Tokio LocalSet to run !Send futures Can you go into this a bit more or provide a code reference? I'm new to writing async Rust code and am interested in this idea.
- jkelleyrtp 5y ago
  Yeah, it's actix-web's runtime. I brought it up to tide's maintainers and got a bunch relevant links. Feel free to click through the tide issue for more context. - https://github.com/http-rs/tide/issues/837 https://github.com/http-rs/tide/issues/837 - https://github.com/rust-lang/wg-async-foundations/issues/87 https://github.com/rust-lang/wg-async-foundations/issues/87 - https://github.com/rust-lang/wg-async-foundations/issues/128 https://github.com/rust-lang/wg-async-foundations/issues/128
masklinn 5y ago
> It's truly understated how difficult the Send + Sync requirements in Tokio are for writing regular code. It's typically rare for async tasks in Tokio to be used across two threads simultaneously, but now all of your data must be Send+Sync. https://docs.rs/tokio/1.14.0/tokio/task/fn.spawn_local.html https://docs.rs/tokio/1.14.0/tokio/task/fn.spawn_local.html > Plus, Tokio is unique in that it's one of the very few runtimes in existence that's work-stealing (meaning the task can move off its original thread). Most other runtimes in other languages do not have that requirement, meaning you can use traditional Cell/RefCell/Rc instead of their slower, atomic variants. Most other languages don't have any concept of Send/Sync, or non-atomic Cell/RefCell/Rc. Rather than the compiler stopping you you only find out about the issue when you hit it, and if you're lucky you can skate by a long long while (or alternatively the language has no way to sync and everything's send). Work-stealing runtimes may be more common than you think because of that: Erlang's BEAM, Go's scheduler(s), Java's fork/join, .net's TPL, many most if not all OpenMP implementations, Apple's GCD, ... implement work-stealing in various measures. Also... thread-per-core makes work-stealing even more necessary because the OS can't perform the balancing? The only ways to avoid work-stealing eventually becoming necessary (for a general-purpose scheduler) is either to have a completely single-threaded scheduler, or to not have an application-level scheduler at all and use OS threads.
- jkelleyrtp 5y ago
  Quoting from a reply on an issue in the rust async-wg repo: > Are people doing this? Does it work? :) > Kind of every single successful async framework since the inception of eventloops :-) > libevent, libev, libuv, boost asio, GTK, QT, seastar, nginx, javascript + node.js, dpdk, netty, grizzly, dart, etc. > Besides Rust the main frameworks which tried to do move tasks between executors are Go and C#'s Threadpool executor (although I think the ASP.NET default executor might have fixed threads). > Therefore the state of the world is actually more that Rust would need to prove that its approach of defaulting to thread-safe is a viable alternative than questioning the effectiveness of single-threaded eventloops. Their effectiveness in terms of reducing context switches and guaranteeeing good cache hit rates was more or less what triggered people to move towards the model, despite the ergonomic challenges of using callbacks.

verdagon 5y ago

I've been thinking about how one could get around the Send + Sync requirements, it's a fascinating conundrum, from a language design standpoint. If we didn't have the Send + Sync requirements, and multiple async functions are running concurrently on the same thread, and multiple of them locked the same RefCell, might that cause a panic?

masklinn 5y ago
> If we didn't have the Send + Sync requirements, and multiple async functions are running concurrently on the same thread, and multiple of them locked the same RefCell, might that cause a panic? Hopefully.

asymptotic 5y ago

> the best you can do is write a thread pool that spawns a Tokio LocalSet to run !Send futures Can you go into this a bit more or provide a code reference? I'm new to writing async Rust code and am interested in this idea.

jkelleyrtp 5y ago
Yeah, it's actix-web's runtime. I brought it up to tide's maintainers and got a bunch relevant links. Feel free to click through the tide issue for more context. - https://github.com/http-rs/tide/issues/837 https://github.com/http-rs/tide/issues/837 - https://github.com/rust-lang/wg-async-foundations/issues/87 https://github.com/rust-lang/wg-async-foundations/issues/87 - https://github.com/rust-lang/wg-async-foundations/issues/128 https://github.com/rust-lang/wg-async-foundations/issues/128

masklinn 5y ago

> It's truly understated how difficult the Send + Sync requirements in Tokio are for writing regular code. It's typically rare for async tasks in Tokio to be used across two threads simultaneously, but now all of your data must be Send+Sync. https://docs.rs/tokio/1.14.0/tokio/task/fn.spawn_local.html https://docs.rs/tokio/1.14.0/tokio/task/fn.spawn_local.html > Plus, Tokio is unique in that it's one of the very few runtimes in existence that's work-stealing (meaning the task can move off its original thread). Most other runtimes in other languages do not have that requirement, meaning you can use traditional Cell/RefCell/Rc instead of their slower, atomic variants. Most other languages don't have any concept of Send/Sync, or non-atomic Cell/RefCell/Rc. Rather than the compiler stopping you you only find out about the issue when you hit it, and if you're lucky you can skate by a long long while (or alternatively the language has no way to sync and everything's send). Work-stealing runtimes may be more common than you think because of that: Erlang's BEAM, Go's scheduler(s), Java's fork/join, .net's TPL, many most if not all OpenMP implementations, Apple's GCD, ... implement work-stealing in various measures. Also... thread-per-core makes work-stealing even more necessary because the OS can't perform the balancing? The only ways to avoid work-stealing eventually becoming necessary (for a general-purpose scheduler) is either to have a completely single-threaded scheduler, or to not have an application-level scheduler at all and use OS threads.

jkelleyrtp 5y ago
Quoting from a reply on an issue in the rust async-wg repo: > Are people doing this? Does it work? :) > Kind of every single successful async framework since the inception of eventloops :-) > libevent, libev, libuv, boost asio, GTK, QT, seastar, nginx, javascript + node.js, dpdk, netty, grizzly, dart, etc. > Besides Rust the main frameworks which tried to do move tasks between executors are Go and C#'s Threadpool executor (although I think the ASP.NET default executor might have fixed threads). > Therefore the state of the world is actually more that Rust would need to prove that its approach of defaulting to thread-safe is a viable alternative than questioning the effectiveness of single-threaded eventloops. Their effectiveness in terms of reducing context switches and guaranteeeing good cache hit rates was more or less what triggered people to move towards the model, despite the ergonomic challenges of using callbacks.