6 ms·

> even if I do get frustrated with mind-numbing Async issues from time to time. A noticeable portion of async issues come from the fact that a lot of people us

by ekidd 3y ago

> even if I do get frustrated with mind-numbing Async issues from time to time. A noticeable portion of async issues come from the fact that a lot of people use Tokio's multithreaded async runtime. Tokio allows you to mix async and native theads, which is both a virtuoso technical accomplishment and also a bit ridiculous. If you use Tokio's single-threaded runtime, things get simpler. The remaining async challenges are mostly the usual "Rust tax", turned up to 11. Rust wants you to be painfully aware that memory allocations are expensive and that sharing memory in a complex concurrent system is risky. In sync Rust, the usual advice is "Don't get too tricky with lifetimes. Use `clone` when you need to." In async Rust without native threads, the rules are something like: 1. Boxing your futures only costs you a heap allocation, and it vastly simplifies many things. 2. If you want a future to remain around while you do other stuff, have it take ownership of all its parameters. Where people get in the most trouble is when they say, "I want to mix green threads and OS threads willy-nilly, and I want to go to heroic lengths to never call `malloc`." Rust makes that approach look far too tempting. And worse, it requires you to understand and decide whether you're taking that approach. But if you remember "Box more futures, own more parameters, and consider using a single-threaded runtime" then async Rust offers some pretty unique features in exchange for a pretty manageable amount of pain. Also, seriously, more people should consider Kotlin. It has many Rust-like features, but it has a GC. And you don't need to be constantly aware of the tradeoffs between allocation and sharing, if that's not a thing you actually care about.

bluejekyll 3y ago
> Tokio allows you to mix async and native theads, which is both a virtuoso technical accomplishment and also a bit ridiculous. Can you describe what issues you’re referring to? I use the multi-threaded Tokio runtime, I’ve not noticed any overhead with that in terms of development vs. single threaded. Also, multi-threaded async runtimes are generally what you want to make sure you don’t have any single tasks blocking others that could make progress in parallel.
- danenania 3y ago
  "Also, multi-threaded async runtimes are generally what you want to make sure you don’t have any single tasks blocking others that could make progress in parallel." Is there an advantage to multi-threaded async if you're IO-bound? If you want a bunch of concurrent system calls or network requests it seems like single-threaded async can handle that pretty nicely ala Node.js.
  - toast0 3y ago
    Depends on your load, but many I/O bound loads are bigger than what you can do in a single thread.
foooorsyth 3y ago
If you're counting on people to drop down into a single-threaded runtime when using async constructs, you've really lost the mark. Parallelism is the user's goal most of the time, right? Perhaps in web context to avoid runaway thread spawning and slow loris attacks it's not (the main selling point of node.js), but otherwise people want to go fast. >Also, seriously, more people should consider Kotlin. I like Kotlin, but I find its coroutine machinations to be far more confusing that just plain threads (over which Java already had/has some nice quality of life abstractions). And debugging broken Kotlin coroutine code is hell. You will not get a normal-looking stack trace when things go wrong.
- proto_lambda 3y ago
  The main goal of using async for me is to not have to handle all the IO wait state machines myself. It does a pretty good job of that, and as long as my program consists of a bunch of tasks concurrently waiting for IO to finish, a single thread is perfectly fine.
- biorach 3y ago
  > Parallelism is the user's goal most of the time, right? I'm not sure you're right about that.
  - foooorsyth 3y ago
    In the general sense? Probably not. In the context of Rust users? It certainly begs the question: why use Rust if you aren't going for speed? Just write it in Node/Go/JVM-lang if performance doesn't matter and you just want an event loop that looks like threads.
- felipellrocha 3y ago
  Take the browser. It gets an incredible amount of performance out people’s devices even though 99% of javascript is bound to a single thread. The way they accomplish that is via their asynchronous architecture. So, not always.
ReactiveJelly 3y ago
Maybe I should try smol instead of Tokio. I'm guessing a lot of libraries are made to fit with Tokio, but I imagine for a typical desktop app, a multi-threaded runtime is total overkill. Plus I could just run multiple runtimes if I want. Like I suppose if I needed a blocking task that was also async, I could send that to a thread pool and then internally spawn a worker async runtime. It's a thinker.
- eska 3y ago
  No need to switch. Just change the tokio flags
  - masklinn 3y ago
    That does not actually do anything. All Tokio APIs which demand Send closures on the MT runtime still do on the ST runtime, because the runtime flavour is not part of the API.
- Klonoar 3y ago
  Or, you know, that single threaded runtime that OP just mentioned. ;P
- swsieber 3y ago
  A single threaded Tokio runtime still has a separate thread pool for heavy tasks. Maybe block_on? I can't remember where I read that tho. Edit: not block_on, but spawn_blocking. See https://www.reddit.com/r/rust/comments/16ebdi1/comment/jzy7fai/ https://www.reddit.com/r/rust/comments/16ebdi1/comment/jzy7f...
masklinn 3y ago
> If you use Tokio's single-threaded runtime, things get simpler. A major issue when dealing with the tokio runtime is Send bounds requirements, I don’t think the single-threaded runtime changes anything because these are API-level issues. You can use spawn_local to avoid migrations but you can do that on the multithreaded runtime just as well. And then a lot of tools and libraries which get layered over tokio (or assume a tokio environment) will require send futures anyway.
- phamilton 3y ago
  LocalSet does not require Send. https://docs.rs/tokio/latest/tokio/task/struct.LocalSet.html https://docs.rs/tokio/latest/tokio/task/struct.LocalSet.html EDIT: Sorry to join in the multiple responses. Clarity for others: "current_thread" by itself does not relax Send, and many libraries aren't configurable to make everything use LocalSet.
  - deleted 3y ago
    [deleted]
  - masklinn 3y ago
    I mentioned spawn_local, which is basically the same feature as localset (you can only spawn_local within the scope of a localset).
- bryanlarsen 3y ago
  Unless I'm mistaken, switching to a single threaded tokio runtime made Send requirements go away for me.
  - veber-alex 3y ago
    Changing the type of runtime doesn't change the API at all. Even in a single threaded runtime spawn requires Send and you need to use LocalSet + spawn_local for !Send futures.
    - 3y ago
timcavel 3y ago
[dead]
aeturnum 3y ago
> If you use Tokio's single-threaded runtime, things get simpler. I don't do a lot of low level work, but coming out of Elixir / Erlang I would expect a greenlet system to manage the # of threads based on the hardware its running on. I.e. not single threaded or "you manage the threads too" but "a standard piece of code spreads your greenlet processes between N managed hardware threads where N is determined by hardware + settings." Is that not a thing that the Rust async libraries support?
- watermelon0 3y ago
  By default, Tokio uses multi-threaded runtime, but you can force it to use the single-threaded one: https://docs.rs/tokio/latest/tokio/attr.main.html#current-thread-runtime https://docs.rs/tokio/latest/tokio/attr.main.html#current-th...
- ekidd 3y ago
  Elixir/Erlang is actually a really interesting case, because: 1. It has a garbage collector, and 2. It relies even more heavily on immutable data than Rust. This means that the Beam VM can seamlessly move green threads around CPUs and preempt them at arbitrary points, all without breaking code. You never need to know who "owns" something, and you never need to worry about another process mutating it while you're looking at it. The Beam VM is amazing. Tokio operates under different constraints: There's no garbage collector, ownership can be "lent" to other code, and mutable state exists in a carefully controlled fashion. Despite this, Tokio absolutely supports spreading green threads across all your CPUs and moving them as needed using a work-stealing scheduler (IIRC). But this only works if all your closures are `Send` (safe to move between CPUs). And any closure that outlives the creating code must generally be `'static` (it does not refer to references borrowed from its creator's scope). Oh, and Rust keeps trying create and manage your async & multithreaded processes without trying to allocate heap memory at all. Unless you explicitly ask it to allocate memory. Which you often should. It is totally possible to make your closures `Send + 'static`. I maintain several production Rust programs which do that, no problem. But doing so requires understanding a moderate amount of Rust and paying a "cognitive tax" by making a bunch of extra decisions about how to represent things. And I think that cognitive tax is a unwise tradeoff for many problems and teams. But I'm still very happy with my async Rust projects, because they get a lot of value out of fast, memory-efficient async code.
seabrookmx 3y ago
> consider Kotlin Or C#. C# has async/await (it originated there) and does have a multithreaded event loop, but due to having GC is just a lot easier to work with than Rust+Tokio. If you're in the "don't colour my function" camp, GoLang is also worth throwing in the mix.
- CharlieDigital 3y ago
  C# is probably what most teams want but don't know it. - Language is very, very similar to TypeScript. If you're already doing TS on the backend with Node (or even JS), it's a very small lift to C# - Very rich standard libraries and first party libraries; reduces the need to import a bunch of third party code - .NET minimal web APIs are very similar to Express now and perhaps even easier since you don't need to import anything to get a microservice up and running - .NET AOT with .NET 8 will dramatically improve the cold-start for use cases like serverless functions (I find the cold start already pretty good with .NET 7 on Google Cloud Run with the CPU Boost feature turned on). - C# has a lot of functional features as a result of F# - Compiles fast and has hot reload via `dotnet watch` - Provides access to low level primitives where extra performance is needed
  - seabrookmx 3y ago
    100%. > NET AOT with .NET 8 will dramatically improve the cold-start Don't get your hopes up on AOT. It still has lots of limitations, reflection being a big one. ASP.NET initialization relies heavily on reflection so it will be a while before we can have AOT compiled, HTTP microservices. > I find the cold start already pretty good with .NET 7 on Google Cloud Run Thanks for the tip! We're mostly using GKE but have a few services on Cloud Run that might benefit. Do you use the "always allocate CPU" option? We're seeing some memory creep we suspect would be solved by giving the GC cycles when a request isn't in flight.
    - CharlieDigital 3y ago
      > ASP.NET initialization relies heavily on reflection Yes; the first thing I tried when .NET 7 went RTM was switch my web API over to AOT only to have it hang during build. .NET 8 AOT is specifically focused on ASP.NET; they've switched over many of the reflection paths to use source generation instead. I have hopes that some time during the .NET 8 lifecycle or .NET 9 horizon, we'll see full support: https://learn.microsoft.com/en-us/aspnet/core/fundamentals/native-aot?view=aspnetcore-8.0 https://learn.microsoft.com/en-us/aspnet/core/fundamentals/n... > Do you use the "always allocate CPU" option? With the CPU Boost option, the cold start is really, really good. Whether you need a warm instance is up to your own threshold for latency on cold starts. I generally avoid using "always allocate CPU" because I'm cheap. The magic of Google Cloud and Cloud Run is really that you don't need to be always on.

by ekidd 3y ago

bluejekyll 3y ago

> Tokio allows you to mix async and native theads, which is both a virtuoso technical accomplishment and also a bit ridiculous. Can you describe what issues you’re referring to? I use the multi-threaded Tokio runtime, I’ve not noticed any overhead with that in terms of development vs. single threaded. Also, multi-threaded async runtimes are generally what you want to make sure you don’t have any single tasks blocking others that could make progress in parallel.

danenania 3y ago
"Also, multi-threaded async runtimes are generally what you want to make sure you don’t have any single tasks blocking others that could make progress in parallel." Is there an advantage to multi-threaded async if you're IO-bound? If you want a bunch of concurrent system calls or network requests it seems like single-threaded async can handle that pretty nicely ala Node.js.
- toast0 3y ago
  Depends on your load, but many I/O bound loads are bigger than what you can do in a single thread.

foooorsyth 3y ago

If you're counting on people to drop down into a single-threaded runtime when using async constructs, you've really lost the mark. Parallelism is the user's goal most of the time, right? Perhaps in web context to avoid runaway thread spawning and slow loris attacks it's not (the main selling point of node.js), but otherwise people want to go fast. >Also, seriously, more people should consider Kotlin. I like Kotlin, but I find its coroutine machinations to be far more confusing that just plain threads (over which Java already had/has some nice quality of life abstractions). And debugging broken Kotlin coroutine code is hell. You will not get a normal-looking stack trace when things go wrong.

proto_lambda 3y ago
The main goal of using async for me is to not have to handle all the IO wait state machines myself. It does a pretty good job of that, and as long as my program consists of a bunch of tasks concurrently waiting for IO to finish, a single thread is perfectly fine.
biorach 3y ago
> Parallelism is the user's goal most of the time, right? I'm not sure you're right about that.
- foooorsyth 3y ago
  In the general sense? Probably not. In the context of Rust users? It certainly begs the question: why use Rust if you aren't going for speed? Just write it in Node/Go/JVM-lang if performance doesn't matter and you just want an event loop that looks like threads.
felipellrocha 3y ago
Take the browser. It gets an incredible amount of performance out people’s devices even though 99% of javascript is bound to a single thread. The way they accomplish that is via their asynchronous architecture. So, not always.

ReactiveJelly 3y ago

Maybe I should try smol instead of Tokio. I'm guessing a lot of libraries are made to fit with Tokio, but I imagine for a typical desktop app, a multi-threaded runtime is total overkill. Plus I could just run multiple runtimes if I want. Like I suppose if I needed a blocking task that was also async, I could send that to a thread pool and then internally spawn a worker async runtime. It's a thinker.

eska 3y ago
No need to switch. Just change the tokio flags
- masklinn 3y ago
  That does not actually do anything. All Tokio APIs which demand Send closures on the MT runtime still do on the ST runtime, because the runtime flavour is not part of the API.
Klonoar 3y ago
Or, you know, that single threaded runtime that OP just mentioned. ;P
swsieber 3y ago
A single threaded Tokio runtime still has a separate thread pool for heavy tasks. Maybe block_on? I can't remember where I read that tho. Edit: not block_on, but spawn_blocking. See https://www.reddit.com/r/rust/comments/16ebdi1/comment/jzy7fai/ https://www.reddit.com/r/rust/comments/16ebdi1/comment/jzy7f...

masklinn 3y ago

> If you use Tokio's single-threaded runtime, things get simpler. A major issue when dealing with the tokio runtime is Send bounds requirements, I don’t think the single-threaded runtime changes anything because these are API-level issues. You can use spawn_local to avoid migrations but you can do that on the multithreaded runtime just as well. And then a lot of tools and libraries which get layered over tokio (or assume a tokio environment) will require send futures anyway.

phamilton 3y ago
LocalSet does not require Send. https://docs.rs/tokio/latest/tokio/task/struct.LocalSet.html https://docs.rs/tokio/latest/tokio/task/struct.LocalSet.html EDIT: Sorry to join in the multiple responses. Clarity for others: "current_thread" by itself does not relax Send, and many libraries aren't configurable to make everything use LocalSet.
- deleted 3y ago
  [deleted]
- masklinn 3y ago
  I mentioned spawn_local, which is basically the same feature as localset (you can only spawn_local within the scope of a localset).
bryanlarsen 3y ago
Unless I'm mistaken, switching to a single threaded tokio runtime made Send requirements go away for me.
- veber-alex 3y ago
  Changing the type of runtime doesn't change the API at all. Even in a single threaded runtime spawn requires Send and you need to use LocalSet + spawn_local for !Send futures.
  - 3y ago

timcavel 3y ago

[dead]

aeturnum 3y ago

> If you use Tokio's single-threaded runtime, things get simpler. I don't do a lot of low level work, but coming out of Elixir / Erlang I would expect a greenlet system to manage the # of threads based on the hardware its running on. I.e. not single threaded or "you manage the threads too" but "a standard piece of code spreads your greenlet processes between N managed hardware threads where N is determined by hardware + settings." Is that not a thing that the Rust async libraries support?

watermelon0 3y ago
By default, Tokio uses multi-threaded runtime, but you can force it to use the single-threaded one: https://docs.rs/tokio/latest/tokio/attr.main.html#current-thread-runtime https://docs.rs/tokio/latest/tokio/attr.main.html#current-th...
ekidd 3y ago
Elixir/Erlang is actually a really interesting case, because: 1. It has a garbage collector, and 2. It relies even more heavily on immutable data than Rust. This means that the Beam VM can seamlessly move green threads around CPUs and preempt them at arbitrary points, all without breaking code. You never need to know who "owns" something, and you never need to worry about another process mutating it while you're looking at it. The Beam VM is amazing. Tokio operates under different constraints: There's no garbage collector, ownership can be "lent" to other code, and mutable state exists in a carefully controlled fashion. Despite this, Tokio absolutely supports spreading green threads across all your CPUs and moving them as needed using a work-stealing scheduler (IIRC). But this only works if all your closures are `Send` (safe to move between CPUs). And any closure that outlives the creating code must generally be `'static` (it does not refer to references borrowed from its creator's scope). Oh, and Rust keeps trying create and manage your async & multithreaded processes without trying to allocate heap memory at all. Unless you explicitly ask it to allocate memory. Which you often should. It is totally possible to make your closures `Send + 'static`. I maintain several production Rust programs which do that, no problem. But doing so requires understanding a moderate amount of Rust and paying a "cognitive tax" by making a bunch of extra decisions about how to represent things. And I think that cognitive tax is a unwise tradeoff for many problems and teams. But I'm still very happy with my async Rust projects, because they get a lot of value out of fast, memory-efficient async code.

seabrookmx 3y ago

> consider Kotlin Or C#. C# has async/await (it originated there) and does have a multithreaded event loop, but due to having GC is just a lot easier to work with than Rust+Tokio. If you're in the "don't colour my function" camp, GoLang is also worth throwing in the mix.

CharlieDigital 3y ago
C# is probably what most teams want but don't know it. - Language is very, very similar to TypeScript. If you're already doing TS on the backend with Node (or even JS), it's a very small lift to C# - Very rich standard libraries and first party libraries; reduces the need to import a bunch of third party code - .NET minimal web APIs are very similar to Express now and perhaps even easier since you don't need to import anything to get a microservice up and running - .NET AOT with .NET 8 will dramatically improve the cold-start for use cases like serverless functions (I find the cold start already pretty good with .NET 7 on Google Cloud Run with the CPU Boost feature turned on). - C# has a lot of functional features as a result of F# - Compiles fast and has hot reload via `dotnet watch` - Provides access to low level primitives where extra performance is needed
- seabrookmx 3y ago
  100%. > NET AOT with .NET 8 will dramatically improve the cold-start Don't get your hopes up on AOT. It still has lots of limitations, reflection being a big one. ASP.NET initialization relies heavily on reflection so it will be a while before we can have AOT compiled, HTTP microservices. > I find the cold start already pretty good with .NET 7 on Google Cloud Run Thanks for the tip! We're mostly using GKE but have a few services on Cloud Run that might benefit. Do you use the "always allocate CPU" option? We're seeing some memory creep we suspect would be solved by giving the GC cycles when a request isn't in flight.
  - CharlieDigital 3y ago
    > ASP.NET initialization relies heavily on reflection Yes; the first thing I tried when .NET 7 went RTM was switch my web API over to AOT only to have it hang during build. .NET 8 AOT is specifically focused on ASP.NET; they've switched over many of the reflection paths to use source generation instead. I have hopes that some time during the .NET 8 lifecycle or .NET 9 horizon, we'll see full support: https://learn.microsoft.com/en-us/aspnet/core/fundamentals/native-aot?view=aspnetcore-8.0 https://learn.microsoft.com/en-us/aspnet/core/fundamentals/n... > Do you use the "always allocate CPU" option? With the CPU Boost option, the cold start is really, really good. Whether you need a warm instance is up to your own threshold for latency on cold starts. I generally avoid using "always allocate CPU" because I'm cheap. The magic of Google Cloud and Cloud Run is really that you don't need to be always on.