46 ms·

Local async executors and why they should be the default

• paletteOvO 3y ago
  async/await is ugly and hard to use and understand for me. It is pretty reasonable that rust chose it because it is a zero-cost abstraction. But i just don't like it.
• deleted 3y ago
  [deleted]
• deleted 3y ago
  [deleted]
• deleted 3y ago
  [deleted]
• evanrelf 3y ago
  > Posted on June 9, 2022
  • rklaehn 3y ago
    Yeah, it is old. But I only found it recently and did not see a discussion of it yet. I found this quite interesting. I love rust, but async rust always seems almost like a different language. Lifetimes become much more complex and much less useful, you use Box and Arc way too much. It seems like you would be better off if everything was heap allocated. Working with local futures brings back some of the things I love about rust, but it is certainly not prominently featured in libraries like tokio and the entire async ecosystem.
    • blowski 3y ago
      > posted on June 9, 2022 > this is old ???
      • mlvljr 3y ago
        [dead]
• mgaunard 3y ago
  Multithreading, why does everyone always do it wrong? One of life's big questions.
  • worldsavior 3y ago
    Because there are no actual good sources to implement async/multi threading programs.
  • rklaehn 3y ago
    I think rust does it right in many cases. Raw rust multithreading is a pleasure to use, and some libraries like rayon are extremely unobtrusive. That is why I am somewhat unhappy with the state of async rust. It does not yet have the quality and unobtrusiveness of sync rust libraries like rayon.
    • buildbot 3y ago
      Yeah, I just recently picked up rust and rayon is so awesome. It is great to be able to instantly parallelize an iterator.
  • junon 3y ago
    Pray tell, what's the objective right way?
    • monsieurbanana 3y ago
      He's saying nobody has found it yet
    • darthrupert 3y ago
      99% of the cases: don't. 1%: very carefully.
  • Thoeu388 3y ago
    Redis nailed multithreading :)
  • chrismsimpson 3y ago
• weinzierl 3y ago
  "If you write regular synchronous Rust code, unless you have a really good reason, you don't just start with a thread-pool. You write single-threaded code until you find a place where threads can help you, and then you parallelize it, [..]" I cannot agree more with that. As someone who's done a good deal of Java in my day job, I can tell you a thing or two about spawning threads willy-nilly. At least it is easier to avoid in Rust, but I'd still prefer it the other way round: opt-in, instead of opt-out.
  • deleted 3y ago
    [deleted]
  • thinkharderdev 3y ago
    It is opt-in. If you're using Tokio then you can specify whether you want to use a single-threaded or multi-threaded runtime. Multi-threaded is "default" in the sense that if you just use `#[tokio::main]` then you get a multi-threaded runtime but you can also just do `#[tokio::main(flavor = "current_thread")]` to get a single threaded executor. More to the point, even using a multi-threaded runtime won't spawn threads willy-nilly. It will default to using N worker threads (where N is the number of CPU cores available).
    • exomancer 3y ago
      > It is opt-in. If you're using Tokio then you can specify whether you want to use a single-threaded or multi-threaded runtime. Multi-threaded is "default" in the sense that if you just use `#[tokio::main]` then you get a multi-threaded runtime but you can also just do `#[tokio::main(flavor = "current_thread")]` to get a single threaded executor. Doing something extra to get a behavior different than default is the definition of opt-out my friend.
      • earthling8118 3y ago
        That argument doesn't hold up. Adding `#[tokio::main]` is opting in because it is that extra something that has to be done. Adding that line is opting in to the multithreaded runtime.
• resonious 3y ago
  > Yes the RwLock and mpsc comes from Tokio and lets you .await instead of blocking a thread, but these are not async primitives, these are multi-threading synchronization primitives. The only reason all this async stuff even exists is because we want concurrency. We want to say "while this one task waits for I/O, this other task will do stuff". So it's not too surprising to me that an intro to async would include synchronization primitives. Those primitives aren't really "thread"-specific if by thread you mean OS thread. When you do async like this, you're basically re-implementing OS threads in user space.
  • hinkley 3y ago
    Async/await is green threads with better scoping and syntactic sugar. Which lets be honest, is something green threads desperately needed.
    • duped 3y ago
      They're not green threads. Futures are stackless coroutines.
      • Ygg2 3y ago
        And green threads aren't a panacea. They introduce runtimes, which was something Rust avoided on purpose.
        • spinningslate 3y ago
          True. Rust's mandate for "zero runtime cost abstractions" is one of the better justifications for async/await. Because async/await, too, has significant limitations, notably: 1. Coloured functions 2. Placing the concurrency design decision (async vs not) on the implementer of a function, not the caller 3. Debugging complexity (1) and (2) are mutually, negatively, reinforcing. As the implementer of a library function, I can't possibly know all the possible scenarios in which my function will be called. If I make it synchronous, then I'm blocking my caller. Which might be an issue - or might not. If I make it async then I don't block the caller - but I've just consigned the entire call stack to be async. There's clearly strong opinions on both sides. Some are strong advocates of async/await. Personally, I'm not. Multiple, concurrent, fine-grained sequences of actions fit my mental model much better. Async/await doesn't give me that: I find reading async/await code to be a bit like reading control flow with GOTOs. My preferences are undoubtedly influenced by working with Erlang, which has supported fine-grained sequences since the beginning. There are no coloured functions; just functions. As the caller, I can decide if I want things to run sequentialy (call from another function) or concurrently (spawn as a separate Erlang process). I have the context I need to make that decision. It's not a panacea by any means (see e.g. Structured Concurrency [0]). But it fits my mental model much better. YMMV of course. -- [0]: https://en.wikipedia.org/wiki/Structured_concurrency https://en.wikipedia.org/wiki/Structured_concurrency
• zokier 3y ago
  While the article mostly focuses on the cognitive cost, which I deeply sympathize with, I do wonder about the runtime performance cost. Are there any good benchmarks actually quantifying the impact of all that extra thread-safety and the hoops that it adds? I'm not asking simply due personal interest in seeing the numbers, but also because if we want to steer the community towards this non-threadsafe direction it would help to have material to back the ideas and I suspect Rust community would be more responsive to complaints about perf than cognitive cost.
  • rklaehn 3y ago
    Anecdotal evidence, but many large rust async code bases perform significantly better if you are using a single threaded runtime vs. a multithreaded runtime. Here is an issue for the quinn crate that implements QUIC: https://github.com/quinn-rs/quinn/issues/1433 https://github.com/quinn-rs/quinn/issues/1433 We have had very similar experiences when developing https://github.com/n0-computer/iroh https://github.com/n0-computer/iroh And this is with quinn still carrying around the synchronization primitives to make everything Send. Another piece of anecdotal evidence: the functional collection library https://github.com/bodil/im-rs https://github.com/bodil/im-rs comes in two flavours: one using Arc and one using Rc. The reason for this is that you get a significant performance increase by using Rc (where manipulating the refcount is just an integer inc/dec) vs Arc where manipulating the refcount needs to be an atomic op. Last but not least - often making futures Send requires you to box your futures. Send and lifetimes just does not play well together. Do something like this: trait Foo { type OpFut<'a>: Future<Output = ()> + 'a; fn op(&self) -> Self::OpFut<'_> } This is what eventually you would want something like async_trait to desugar to. Use the above in a Send context, and you get a completely undecipherable higher-ranked lifetime error. So you are back to boxing all futures.
    • Nullabillity 3y ago
      > Last but not least - often making futures Send requires you to box your futures. Send and lifetimes just does not play well together. Do something like this: You should ~never need to box your futures to make them Send. You do sometimes need to box them to make them Unpin, but most of the time this is a tradeoff where it's possible to find a way to avoid moving them instead. It seems like these screeds (both the OP and this reply) come from the misunderstanding that "X is hard, Y sometimes comes up in the errors when trying to do X, therefore X would be easy if we got rid of Y". In reality, Y often just demonstrates intricacies of X that you weren't thinking about.
• runiq 3y ago
  This is bad editorializing. You're putting words in the author's mouth that cannot even be found on that page. Don't do that. Edit: Thanks to the mods or whoever fixed it.
  • beltsazar 3y ago
    I wonder if HN should have a rule that prohibits editorializing.
    • dvt 3y ago
      Not sure if this is facetious, but HN does have a rule that prohibits editorializing. From the Guidelines: "Otherwise please use the original title, unless it is misleading or linkbait; don't editorialize."
      • beltsazar 3y ago
        No, I was genuinely asking because I had found many HN posts were editorialized.
    • voxelghost 3y ago
      Maybe HN should auto-pull title from URL, and then add the posters title as a form of "subtitle/comment"
      • pindab0ter 3y ago
        This assumes those titles are usable, which they often are not.
        • yencabulator 3y ago
          Populating the field automatically might nudge people toward doing the right thing, though.
          • 3y ago
• hoangnguyenvu 3y ago
  reason?
• datadeft 3y ago
  For me the biggest issue with Async is the management of multiple dependent async calls. It has some weird thing going on and I am not sure which pattern to use exactly. Some functions expect exactly same async fn signature some not and I am not sure why and which one to use.
  • duped 3y ago
    I'm confused what you mean here. If you have a function that "depends on" another function you call it within the other function. If it's async you .await it. Do you mean something about spawning tasks or passing callbacks around?
    • datadeft 3y ago
      I have this function that I still struggle to write. async fn function_handler(_event: Request) -> Result<Response<Body>, Error> { let aws_clients = AwsClients { s3_client: S3_CLIENT.get().await.to_owned(), glue_client: GLUE_CLIENT.get().await.to_owned(), athena_client: ATHENA_CLIENT.get().await.to_owned(), }; println!("{}", aws_clients); let config = CONFIG.get().await; let version = VERSION.get().await; let db_eid_cache = create_db_eid_cache(aws_clients.clone(), config); let table_eid_cache = create_table_eid_cache(aws_clients.clone(), config); let job_eid_cache = create_job_eid_cache(aws_clients.clone(), config); let table_summaries_cache = create_table_summaries_cache(aws_clients.clone(), config); let job_summaries_cache = create_job_summaries_cache(aws_clients.clone(), config); let query_summaries_cache = create_query_summaries_cache(aws_clients, config); let rez = tokio::join!( db_eid_cache, table_eid_cache, job_eid_cache, table_summaries_cache, job_summaries_cache, query_summaries_cache, ); let six_pack = SixPack { db_eid_cache: rez.0, table_eid_cache: rez.1, job_eid_cache: rez.2, table_summaries_cache: rez.3, job_summaries_cache: rez.4, query_summaries_cache: rez.5, }; let lambda_response = LambdaResponse { config, result: &six_pack, version: &version, now: utc_now(), }; let response_body = serde_json::to_string(&lambda_response)?; Ok(Response::builder() .status(200) .header("content-type", "application/json") .body(response_body.into()) .map_err(Box::new)?) } Not sure how to do error handling properly or what would be the idiomatic way to implement it.
• fsckboy 3y ago
  the title should be "Local Async Executors and Why They Should be the Default (Rust)"
• ikekkdcjkfke 3y ago
  In C# i always wondered why they couldn't hide the async/await logic for most cases. I never need to fire off two IO futures at the same time, so just make the thread do other stuff if i'm waiting for IO feedback, don't make me type out async/await in all impacted functions, let the compiler figure out when it can process other stuff
  • peheje 3y ago
    the use of async and await is a design decision that requires knowledge of the program's logic and desired behavior. It's not just a matter of compiler optimization, and that's why the compiler can't automatically figure out where to use these keywords. Suppose we have a service where users place orders, and we need to: 1. Save the order. 2. Deduct items from inventory. 3. Send a confirmation email. If we perform these operations asynchronously but sequentially using `await`: ```csharp public async Task PlaceOrder(Order order) { await SaveOrder(order); await DeductItems(order); await SendConfirmationEmail(order); } ``` They happen in the correct order, which is important. But, if the compiler removed the `await`, causing all three operations to start at the same time: ```csharp public async Task PlaceOrder(Order order) { SaveOrder(order); DeductItems(order); SendConfirmationEmail(order); } ``` This could lead to issues like sending the confirmation email before the order is saved, showing the importance of `await`. The compiler cannot optimize this without understanding the business logic.
    • ikekkdcjkfke 3y ago
      I was thinking more in terms of making good old 'blocking' calls and that the compiler can 'taskify' those old blocking calls to let the thread do some other task instead of waiting on the 'blocking'
      • notamy 3y ago
        Sometimes you want to spawn a [future/task/promise/...] and not immediately await it, but instead pass it around. Suppose something like: should_be_future = async_function() should_execute = async_function() # <--- await goes here handle_future(should_be_future) handle_result(should_execute) While the compiler could figure out where to insert awaits automatically, it adds cognitive overhead for the developer -- suddenly the same way of calling a function can result in either a [future/task/promise/...] or the expected type.
• dpc_01234 3y ago
  Async is and probably will always be less usable than blocking Rust. It is a very, very useful mode of operating when you really need two of its biggest benefits: lightweight cooperative concurrency and task cancellation, but it comes at a big usability cost. Rust software should use async tactically - in places where it is needed. Unfortunately handling http, which is a large part of many applications is actually a place where async has benefits. But if you plan to run your http behind nginx anyway (for TLS termination) even there using blocking http server might be a good idea. > If you write regular synchronous Rust code, unless you have a really good reason, you don't just start with a thread-pool. You write single-threaded code until you find a place where threads can help you, and then you parallelize it, I disagree with this one. When you work on a software project you should have the basic architecture figured out already, and main part of that is breaking your software into structurally parallel parts that can work independently. Adding ad-hoc parallelism after the fact works only for small scale things and will lead to rather accidental concurrency architecture. Then for each part (groups of threads), figure out if it *needs* async. Between each part you'd communicate via channels or some shared data structures that rather easily can be made to work with both async/blocking code. So e.g. an async http server, benefits from lightweight async concurrency, makes rpc-like channel-based calls to blocking IO / CPU/bussiness-logic intense workers (that don't benefit from async) where it makes sense. Each part is written in the best "type of Rust for its use-case". Or if you need ability to cancel certain computations inside the larger framework (e.g. simulating agents etc.) you might want to nest async executor inside a blocking code. Note: there's a lot of types of program archetypes out there (CRUD, ETL, data-intensive, embedded, frontent SPA, native mobile app) and I've noticed that many people are boxed in the type they happen to work on. CRUD applications (which are very common) are often 90% http handling-based and it might make sense to write them whole in async Rust.
  • sanxiyn 3y ago
    Your CRUD web application server almost certainly doesn't need async Rust. Using a blocking HTTP server is not "might be a good idea", it simply is a good idea. I recommend Rouille for this: https://github.com/tomaka/rouille https://github.com/tomaka/rouille. In case you are worried about performance, check the benchmark. Blocking Rouille is faster than builtin async server in Node.js.
• lionkor 3y ago
  I find all the async stuff in Rust incredibly ugly, cumbersome, and its one of the biggest reasons I prefer C++, still. C++ lets me just write single- or multithreaded code, because none of the dependencies force their `async` stuff on me. Yeah, its up to me to ensure things are synchronized, but I'd rather do that than try to figure out how to get some dependency that isnt meant to use async to work in some async move closure.
  • zokier 3y ago
    Your dependencies are what you choose them to be, you are not forced to use async libraries if you don't want to.
    • klabb3 3y ago
      Technically true, but in practice you’re constrained by whatever is provided by the ecosystem. And the truth is that many Rust projects were forced to choose async or sync, so these are often disjoint and fragmented. Even within async, crate authors need to pick and choose which async ecosystem to support, due to lack of support in std for spawning tasks and running simple IO, which most projects need. This is slightly alleviated by the Tokio hegemony, but that itself is a deferred migration minefield for Rust to handle.
      • zokier 3y ago
        Libraries don't just sprout out from nothingness, someone has written those. And if there isn't a library that suits you, that someone can be you then. Key observation is that the ecosystem is only additive, it doesn't deny you of anything.
        • cmrdporcupine 3y ago
          This is true to an extant, except it's also a cultural problem: it spreads across crates, and soon you've got tokio spread everywhere. But it's also a concern for me that a language feature was added that requires runtime framework support to function. This is bad separation of concerns. It (async) is viral syntactic sugar that doesn't need to exist; asynchronous programming has been done for decades without it.
• distcs 3y ago
  Can a mod fix the title please? The poster of this story has editorialized the title so bad that it has no connection with the actual title. Actual title: Local Async Executors and Why They Should be the Default Posted title: Async rust – are we doing it all wrong? Really? Why this kind of terrible editorializing?
  • foldr 3y ago
    I agree that this could be less editorialized, but the actual title doesn't include 'Rust', so that needs to be added at least.
    • distcs 3y ago
      Agreed. Perhaps we can have a syntax to declare what is editorialized. We already add "(YEAR)" and "[pdf]" and "[video]". We could "[Rust]" or other keywords like that about the post missing from the title?
• dvt 3y ago
  > If you know anything about asynchronous sockets it should be that multi-threading a socket doesn't actually yield you more requests / second, and it can actually lower it... Re-read this a few times, and I'm fairly convinced it is not generally true. The author is also being a bit confusing about what exactly he means by "socket" here. Because while it's true that multi-threading over a server socket (e.g. the one that binds to the port when you launch the server) will not yield performance gains, multi-threading clients (that have their own sockets, including file descriptors) definitely will. That's the whole point of nginx thread pools[1]. Note that nginx does zero "CPU-bound" work, it literally just serves files. Node/Deno being single-threaded is purely a limitation of Javascript. Tomcat, Jetty, etc. are all multi-threaded. I'm a bit tired, so I can't comment on the rest of the post in detail, but this was a bit of a red flag. [1] https://gist.github.com/denji/8359866 https://gist.github.com/denji/8359866
  • gpderetta 3y ago
    Having multiple threads calling 'accept' itself can be a win and Linux (and epoll) has explicit support for it.
• namjh 3y ago
  > Making things thread safe for runtime-agnostic utilities like WebSocket is yet another price we pay for making everything multi-threaded by default. The standard way of doing what I'm doing in my code above would be to spawn one of the loops on a separate background task, which could land on a separate thread, meaning we must do all that synchronization to manage reading and writing to a socket from different threads for no good reason. Why so? Libraries like quinn[1] define "no IO" crate to define runtime-agnostic protocol implementation. In this way we won't suffer by forcing ourselves using synchronization primitives. Also, IMO it's relatively easy to use Send-bounded future in non-Send(i.o.w. single-threaded) runtime environment, but it's almost impossible to do opposite. Ecosystem users can freely use single threaded async runtime, but ecosystem providers should not. If you want every users to only use single threaded runtime, it's a major loss for the Rust ecosystem. Typechecked Send/Sync bounds are one of the holy grails that Rust provides. Albeit it's overkill to use multithreaded async runtimes for most users, we should not abandon them because it opens an opportunity for high-end users who might seek Rust for their high-performance backends. [1]: https://github.com/quinn-rs/quinn https://github.com/quinn-rs/quinn
  • exomancer 3y ago
    > Also, IMO it's relatively easy to use Send-bounded future in non-Send(i.o.w. single-threaded) runtime environment, but it's almost impossible to do opposite. Ecosystem users can freely use single threaded async runtime, but ecosystem providers should not. We have Send and non-Send primitives in Rust for a reason. You could use Arc/Mutex/AtomicUsize/... everywhere on a single thread, but you should use Rc/RefCell/Cell<usize>/... instead whenever possible since those are just cheaper. The problem is that in the ecosystem we are building the prevailing assumption is that anything async must also be Send, which means we end up using Send primitives even in non-Send contexts, which is always a waste. > If you want every users to only use single threaded runtime, it's a major loss for the Rust ecosystem. Running single threaded executors does not prohibit you from using threads, it just depends on how you want to do that. You can: 1. Have a single async executor running a threadpool that requires _everything_ to be Send. 2. Have a single threadpool, each thread running its own async executor, in which case only stuff that crosses thread boundaries needs to be Send. The argument is that there are many scenarios where 2 is the optimal solution, both for performance and developer experience, but the ecosystem does not support it well.
• samsquire 3y ago
  This article is against multithreaded executors by default and that synchronous code is easier to read and more practical than async code. I completely understand this. My hobby and main interest is multithreading, async, coroutines, parallelism so I love articles like this so thank you. I am trying to design a solution that lets us have our cake and eat it to. I want multithreaded coroutines or multithreaded async executors by default. I am trying to design a server and runtime that is largely parallel, concurrent, efficient, easy to understand, easy to reason about, async and easy to read and maintain. I want: * Tokio is not bad but I think a codebase that uses async rust requires a high level of skill, cognitive load and understanding. It's not straightforward! * synchronous straight-line flow code is the easiest to read and follow * promises and callbacks aren't easy to read or follow control flow * making a single threaded program parallel after writing it is almost a complete rewrite * work stealing thread pools solve the starvation problem, so they're good! * IO shouldn't block CPU and CPU shouldn't block IO * I am trying to design a syntax that is data orientated that means programs can be parallelised after writing them, so parallelisation comes for free * we can use the LMAX Disruptor pattern for efficient cross-thread communication. I use a lockfree multiconsumer multiproducer ringbuffer in my programs. I have an epoll-server which multiplexes clients/sockets over threads, this is more efficient than a thread-per-socket/client. I need to change it into a websocket server. Imagine you're a search engine company and you want to index links between URLs. How would you model this with async rust and thread pools? task download-url for url in urls: download(url) task extract-links parsed = parse(document) return parsed task fetch-links for link in document.query("a") return link task save-data db.save(url, link) How would you do control flow and scheduling and parallelism and async efficiently with this code? `db.save()`, `download()` are IO intensive whereas `document.query("a")` and `parse` is CPU intensive. I think its work diagram looks like this: https://github.com/samsquire/dream-programming-language/blob/main/Slide2.PNG?raw=true https://github.com/samsquire/dream-programming-language/blob... I've tried to design a multithreaded architecture that is scalable which combines lightweight threads + thread pools for work + control threads for IO epoll or liburing loops: Here's the high level diagram: https://github.com/samsquire/ideas5/blob/main/NonblockingRuntime.drawio.png https://github.com/samsquire/ideas5/blob/main/NonblockingRun... The secret is modelling control flow as a data flow problem and having a simple but efficient scheduler. I wrote about schedulers here and binpacking work into time: https://github.com/samsquire/ideas4#196-binpacking-work-into-network-and-time-and-server-communication https://github.com/samsquire/ideas4#196-binpacking-work-into... I also have a 1:M:N lightweight thread scheduler/multiplexer: https://github.com/samsquire/preemptible-thread https://github.com/samsquire/preemptible-thread
• nubinetwork 3y ago
  [flagged]
• binary132 3y ago
  Rust acolytes need to come to terms with the fact that if they want Rust to be the next C++, it's going to be the next C++. Heck, even C++ async is simpler. Anyway, who needs a package manager when you've got a package manager?
  • falcor84 3y ago
    >who needs a package manager when you've got a package manager That flew over my head; could you explain?
    • foota 3y ago
      I think the idea is that being the next C++ means being just as hard to write as C++, and then saying "why do we need the next C++ when we already have C++?", but I'm curious if my interpretation is right :)
      • binary132 3y ago
        Yes, the first part is what I was going for. Actually it is even worse than that, because C++ has a more limited syntax. Rust's syntax and semantics must support more features. This fundamentally limits its accessibility.
    • binary132 3y ago
      Lots of languages have tools which reimplement package management specifically for their libraries, and this pattern has become fairly "normal", but the traditional norm is to access and provide libraries as system packages via the system's package manager. With that in mind, things like Cargo become less magical and more redundant with other tools like Make.
• chrismsimpson 3y ago
  Suggesting that single threaded concurrency is the right way to do it when building tooling is completely asinine.
  • IshKebab 3y ago
    He isn't suggesting that. He's suggesting it is the right place to start, in the same way that we normally start writing sync code with a single thread.
    • chrismsimpson 3y ago
      Nope. Direct quote: > “..and when you need to utilize multiple CPU cores, you just spawn multiple processes that listen on the same socket. This is a much better way of structuring servers..”
      • IshKebab 3y ago
        That's not a contradiction. He's saying that later when you need multithreading you can add it. You generally don't start with it. Of course for some projects you know up front you'll need multithreading but the point is that that isn't the default position.
        • chrismsimpson 3y ago
          Yes it is. Spinning up another process is not multithreading, like, literally. It’s multiprocess. I never cease to be amazed with the complete lack of basic CS literacy from JS script kiddies.
          • IshKebab 3y ago
            I was using "multithreading" as a shorthand for "making use of multiple CPU hardware threads".
• pdimitar 3y ago
  Just a personal take: after not coding with Rust for several months, I find it more and more difficult to return to an async code I was writing. The whole thing just reads... ugly and inconsistent. It needs too much already-accumulated knowledge. As the article correctly points out, you need a bunch of stuff that are seemingly unrelated (and syntactically speaking you would never guess they belong together). And as other commenters pointed out, you need to scan a lot of docs -- many useful Tokio tools are not just not promoted, they are outright difficult to find at all. Now don't get me wrong, I worked on projects where a single Rust k8s node was ingesting 150k events per second. I have seen and believed and I want to use Rust more. But the async story needs the team's undivided attention for a long time at this point, I feel. Against my own philosophy and values I find myself attracted to Golang. It has a ton of ugly gotchas and many things are opaque... and I still find that more attractive than Rust. :( This article is a sad reminder for me -- I am kind of drifting away from Rust. I might return and laugh at myself for this comment several months down the line... but at the moment it seems that my brain prefers stuff that's quicker to grok and experiment with. Not to mention writing demos and prototypes is objectively faster. If I had executive power in the Rust leadership I'd definitely task them with taking a good hard look at the current state of async and start making backwards-incompatible changes (backed by new major semver versions of course). Much more macros or simply better-reading APIs might be a very good start. Start making and promoting higher-order concurrency and parallelism patterns i.e. the `scoped_pool` thingy for example.
  • rklaehn 3y ago
    I fully agree that the async story needs attention. Something as drastic as backwards-incompatible changes might not be needed. But definitely much more documentation about best practices, highlighting the option of local async tasks, making the APIs for that more convenient, and also some low hanging fruits in terms of language syntax. It seems a bit like while there was a lot of excitement about the async syntax a few years ago, now there is a MVP syntax and everything has just slowed down a lot.
    • pdimitar 3y ago
      > now there is a MVP syntax and everything has just slowed down a lot. Sadly this is spot on. Seems like after async was delivered there was only one big push from tokio 0.1 to 0.2 to 1.0 and that was it, the ecosystem and/or language maintainers seem to have lost interest in making it better to read and use.
• insanitybit 3y ago
  I'm sympathetic to this point but I think that: a) Saying Node + Deno are good is a stretch. Node has horrible performance, even for simple routing. And I'll source that[0]. b) Saying that adding `Send + Sync + 'static` bounds is a serious burden is, to me, overstating things. > the far better model for writing performant servers. It's completely workload dependent. For a chat server it's almost definitely not going to be more performant and you may end up with worse latency. > it only costs you friction everywhere else in your entire codebase, and quite often performance as well. I am unconvinced tbh. I do not believe that adding Send + Sync + 'static bounds is onerous, I do not believe satisfying those bounds is hard (it's almost always just a matter of moving the value), and I do not believe that the vast majority of programs benefit from TPC architecture. I recognize that there is a problem here - that we are optimizing for one runtime at the expense of others - but I am not convinced at this point that the problem matters. [0] https://www.techempower.com/benchmarks/#section=data-r21&test=json https://www.techempower.com/benchmarks/#section=data-r21&tes...
• captainmuon 3y ago
  Wait, async is multithreaded by default in Rust? For me the whole point of using async in JavaScript or Python (originally with Twisted's @inlineCallbacks) was to get concurrency without threads. Imagine writing code for a computer game bot: move left, wait for enemy, attack enemy... You normally can't write it like this because it would block the rest of your program. Async allows you to go from "program sequential" to "bot sequential" for lack of better terms. If you are IO bound there is no need for threads. I often like to use one network thread and one GUI thread to keep things separate, and to prevent the occasional blocking in one to cause latency in the other. You just need to have a method to post calls to the other event loop. Works well in Python, as well with Qt-based apps. C# on the other hand made the same "mistake" of being very general. I think you can even have a coroutine suspend on one thread and wake on another. It looks like you switch threads in the middle of a function. I guess that is useful when you want to write performant multithreaded servers. I just want to write easy sequential code without worrying about locks or state machines.
  • maeln 3y ago
    > Wait, async is multithreaded by default in Rust? For me the whole point of using async in JavaScript or Python (originally with Twisted's @inlineCallbacks) was to get concurrency without threads. Async in rust is nothing by default because it doesn't have a runtime. You need to use a runtime/executor for async to work, the most popular being Tokio, which can be configured to be single-threaded or multi-threaded. EDIT: But in a way since Rust make 0 assumption about how async function are going to be executed, it does design around the fact that they might be multi-threaded.
  • kibwen 3y ago
    > Wait, async is multithreaded by default in Rust? No, whether or not async is multithreaded depends on whether or not your executor is multithreaded. Rust doesn't ship an executor by default, you need to bring your own. Tokio, the most popular general-purpose executor, is multithreaded by default, and can be configured to be single-threaded.
  • duped 3y ago
    The point being made isn't whether or not the futures are evaluated on one thread or many, but that by default, library authors assume that something may be evaluated on multiple threads which imposes some constraints on both the argument and return types from functions. So for example, the tokio executor could run single threaded or on a thread pool. However, tokio::task::spawn takes a future as an argument that may be evaluated on a different thread, so any caller of this API and the designers of the library behind it need to guarantee the argument is thread safe. That constraint is reflected through the type system by constraining the argument type of tokio::task::spawn<T>(fut: Future<Output = T> + impl Send + 'static) which means "fut is a type that implements Future with the output of type T and is Send (safe to send to a different thread) and does not have any lifetimes that outlive the 'static lifetime (contains no references to non-static data)".
• rich_sasha 3y ago
  Moaning aside, what is (if any?) the direct equivalent to single-threaded asyncio, like Python or node.js, in Rust? The thing I enjoy about async in Python is that it's very easy to write "thread"-safe code - you know exactly where you might give up execution context, and 90% of the time you have no need for mutexes and locks. But as this article complains, in Rust, it seems to be sync or multi-threaded.
  • zokier 3y ago
    The article mentions smols LocalExecutor and tokios LocalSet, both which are single-threaded
• packetlost 3y ago
  I'm still not sure what async (cooperative multitasking) gives over green threads (preemptive userspace multitasking)
  • surajrmal 3y ago
    Lower overhead. It's the same reason you would need to avoid a GC. It may not be for everyone, but that isn't a rust goal necessarily.
    • packetlost 3y ago
      I'm aware that Rust's threading has higher overhead because they're system threads, but what about green/user threads? Is there intrinsic overhead to userspace threads that async doesn't have? I think every "task"/thread needs it's own callstack space, and you're paying scheduling overhead no matter what. Is there literature on the topic?
      • lycopodiopsida 3y ago
        As far as I understand, green/user threads were on the table in rust team initially, but were sorted out because having green threads also means there is a runtime managing them which rust tries to avoid. What I do not understand is where the principal difference with async lies: it has no runtime, but you have to bring your own for it to work anyway in form of e.g. tokio. What rust goal conflicts with a similar solution for green threads?
        • packetlost 3y ago
          I think it was, and Tokio appears to implement green threads as well with `tokio::spawn()`
      • bombela 3y ago
        The short answer is that an async function in rust (and C++ and C#) is rewritten by the compiler to become a state machine (a struct + an enum to discriminate the current state + an union store local variables). You could do it by hand, though of course it's tedious and error prone. Hence why almost nobody writes software like that. For the compiler, the state machine code is amenable to optimizations. You can find examples online where a chain of async function is optimized to a single call in the main. In contrast green thread and the like are very similar to a kernel thread. The difference is they are lighter because there is less bagage to carry around. But composing green threads require an arguably light context switch.
• gwbas1c 3y ago
  When I was between jobs, I decided to learn rust by writing some async code. I really got stung and spent days doing things that would take minutes in C# or Node. Part of the problem is that, in Rust, stack memory is easier to use than heap memory. When writing traditional threaded code, this isn't much of an issue because naturally most of our code is working with values on the stack. BUT: When we look more closely in how async works in C# or Javascript, the compiler, under the hood, breaks up an async method into multiple methods and puts values that appear to be on the stack onto the heap. (Of course, I'm over simplifying.) It just works, and it just works well. But, in Rust, making something async implicitly moves what appears to be on the stack onto the heap. It can quickly become hard to reason about. I wish I knew about techniques that this article describes. Maybe it would make my code easier? In my current hobby projects, I'm doing traditional blocking IO because it's not "worth it" to write async. (In comparison, in Node, async code helps avoid nesting callbacks within callbacks, and in C#, doing IO in async is a best practice.)
• pie_flavor 3y ago
  The thing the article calls bad is not the thing the examples illustrate. The set-up is supposed to be that multithreading is a pain, but none of the examples actually do that. Take the initial list. "These are not async primitives, these are multi-threading synchronization primitives." Yeah, but they're multi-threaded forms of stuff you still need. If you needed RwLock in multi-threaded mode, you need RefCell in single-threaded mode, and the API is virtually identical. If your state needed to be in an Arc, it'll need to be in an Rc. And if you needed sync::mpsc, you'll instead need... sync::mpsc. A Send bound is no great burden. The only type that you will regularly interact with that is not Send is the lock type from a Mutex or RwLock, which is good because if you hold it across a long await you can slow down your app, a bug prevention mechanism that does not exist in straight multithreading. The only point that actually illustrates a threading-caused problem is the thing about multithreading sockets, which it admits is almost imperceptible, and which you can also solve by not doing that. Almost everything the author identifies as a parallelism problem is a 'static problem. Tokio is missing a scoped spawn like std has, and if it gained one then the much described multithreading woes would reduce to basically nothing.
• Animats 3y ago
  It's a frustrating area. As I've mentioned before, I'm writing a high-performance metaverse client in Rust, something which has many of the problems of both a web browser and a MMO. If you want to have a good looking metaverse, it takes a gamer PC multiple CPUs and a good GPU to deal with the content flood. (This is why Meta's Horizon looks so bad.) Now you have to use all that hardware effectively. So what I'm writing uses threads. About 20 of them. They're doing different things at different priorities towards a coordinated goal. This is different from the two usual use cases - multiple servers running in the same address space, and parallel computation of array-type data. Concurrency problems so far: - Single-thread async is simple. Multi-thread async is complicated. Multi-thread async plus other threads not managed by the async system isn't used enough to be well supported. - Rust is good at preventing race conditions, but it doesn't yet have a static deadlock analyzer. It needs one. - Different threads at different priorities do work in both Linux and Windows, but not all that well. With enough low-priority compute-bound threads to keep all CPUs busy, high-priority threads do not get serviced in a timely manner. I was spoiled by previous work on QNX, which, being a true real-time operating system, takes thread priorities very seriously. On QNX, compute-bound background work has almost no effect on the high-priority stuff. Linux just doesn't work well at 100% CPU utilization. Unblocking a lock does not wake up a higher priority waiting thread immediately. This can delay high-priority threads unnecessarily. - The WGPU crowd has spent a year getting their locking sorted out so that you can load content into GPU memory while the GPU is rendering something else. It's a standard feature of Vulkan graphics that you can do this, but it has to be supported at all levels above Vulkan too. For me, that's WGPU and Rend3. That stack is close enough to ready to test, but not ready for prime time yet. - There's no way to cancel a pending HTTP request made with "ureq". "reqwest" supports that, but you have to bring in all the async and Tokio stuff, which means you now have multi-thread async plus other threads. This is only a problem for what I'm doing when the user closes the window, and the program needs to exit quickly and cleanly. I'm getting a 5-10 second stall at exit because of this. - Crossbeam-channel is not "fair"; it's possible to starve out some requests. Parking-lot is fair, but doesn't have thread poisoning, which means that clean shutdowns after a panic are hard. - Running under Wine with 100% CPU utilization with multiple threads results in futex congestion in Wine's memory allocation library, and performance drops by over 99%, with all CPUs stuck in spinlocks. The program is still running correctly, but at about 0.5 frames per second instead of 60 FPS. Bug reported and recognized by the Wine crew, but it's hard to fix. I can make this happen under gdb running my own code and see all those threads in the spinlocks, so I was able to file a good bug report. But I haven't generated a simple test case. It's a Wine-only problem; doesn't affect Microsoft Windows. So that's life in a heavily threaded world. Individual CPUs have not become much faster in over a decade. Everybody has been stuck at 3-4 GHz for a long time now. CPUs with many cores are widely available in everything from phones to game consoles. To use modern hardware effectively, you need threading.
• deleted 3y ago
  [deleted]
• deleted 3y ago
  [deleted]
• dathinab 3y ago
  > why multi-threaded task executors should be the default many many reasons and it's subtile and complicated For one ironically it's just way easier to use, especially for less experienced programmers. In a task system like that it's just way easier to accidentally cause major issues when it's many tasks on one thread compared to multiple threads (at least with the guard rails rust provides for threading safety). On the other hand the performance overhead of by-default multi-threaded is just fine for a ton of use-cases to a point you could argue worrying about it is premature optimization. Through it's important to state that a lot of this comes from the ecosystem around rust and not rust itself, as stuff like `LocalSet` shows you can have a non-multi threaded runtime and there is no reasons all the libraries you might use couldn't provide non multi thread safe versions. Some do. Just many decided that avoiding the performance overhead of being thread save isn't worth the maintenance overhead and additional foot guns it can provide. Now naturally you can say "but node/deno/etc." but they are a completely different beast then "just" being not multi threaded by default. Like e.g. they don't have multi threaded code at all. Just single threaded code communicating through serialized messages (kinda). They also e.g. handle all I/O completely separated from you application code and don't have any non serialized communication between threads etc. etc. Interestingly if you look at the design choices of the I/O Event loop (reactor) of tokio there are some conceptual similarities. Also there AFIK there is a company which is building something similar to the node model for rust using WASM. I mean in the end the node-style approach is grate for building servers, but rust isn't just for building servers but much more general with much more low level use cases. Now the main point where rust could improve quite a bit is to make it much easier to write a library which work very efficient with both cases. Code which crosses thread boundaries and code which doesn't. Currently you are often split between either implementing it twice, doing terrible unusable generics tricks or using tons of `cfg` (probably generated using macros/annotations) and hope no dependency accidentally imports the multi-thread feature when you don't need it. Non of this is really viable but it's a surprisingly hard problem. Currently the best idea I can come up for it is generic modules which make the "terrible to use generics tricks" usable, but it's probably not enough by a long stretch. (Even if a solution is found it might not work for Waker, even if it does you still might want sync/send Wakers in some cases.)
• dathinab 3y ago
  Rust asyncs design was probably the mostly correct decision for what rust is: A general system programming language which you can use in most situations people today use C,C++ and more. If rust would only be targeting web server programming the right decision might have been no async and green threads, it's just much easier to use. But that rust would likely never have succeeded as most of it's initial success cases where for use-cases where you wouldn't want green threads. Nicely we might still get no async and green threads: In form of run times which run WASM compiled rust code in a node like fashion. Probably in combination with some serverless/edge-compute providers which hopefully will be nice to use.
• deleted 3y ago
  [deleted]