6 ms·
The main thing I want for Async Rust is the ability to not use it. What I'm doing involves a virtual world viewer with maybe a dozen threads. Some are compute
by Animats 6y ago
The main thing I want for Async Rust is the ability to not use it.
What I'm doing involves a virtual world viewer with maybe a dozen threads. Some are compute bound. Some are talking to the GPU. Some are waiting for I/O. The normal situation is about 2 or 3 CPUs of work. Sometimes more.
I don't want an async model, which assumes you're I/O bound, interfering with keeping all those CPUs busy. Already I've had to switch from using "reqwest", which now seems to always bring in tokio, to "ureq", a minimal HTTP client. The way to do HTTP requests used to be with "hyper", but that became a lower level for "reqwest", and then "tokio" was slipped in underneath to make it "async". It always uses async, even if you make a blocking request.
"Async" is a specialized tool for web servers with very large numbers of clients. Outside of that niche, it's seldom needed.
- songqin 6y agoYour comment touches on a few misconceptions I see a lot. Firstly, `reqwest` exposes both an async and a synchronous API, allowing the developer to choose which one to use. They are largely interchangeable code-wise. [1] Secondarily, and more broadly, async is possible to opt out of. You must understand that most web and network related libraries will be async by default for performance, because people who write in Rust and people who write web servers typically care greatly about performance. This is the intersection of those two groups. That being said, there are options outside of that ecosystem. [2] If you truly want to use an asynchronous library without migrating your application to run entirely on an async runtime like tokio, you can run it inside of a synchronous function without much trouble. I've put together a playground link for you. [3] 1. https://docs.rs/reqwest/0.11.2/reqwest/blocking/index.html https://docs.rs/reqwest/0.11.2/reqwest/blocking/index.html 2. Iron: https://github.com/iron/iron https://github.com/iron/iron Rouille: https://github.com/tomaka/rouille https://github.com/tomaka/rouille 3. https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=fac87a37c5d29b3342b41d7c8ed015d3 https://play.rust-lang.org/?version=stable&mode=debug&editio...
- Animats 6y ago"reqwest" exposes a synchronous API, but it's doing async stuff underneath. If you turn on logging, you can see 30 or so async events associated with a single HTTP client side request. It's starting up and shutting down a polling thread just to make one HTTP client request. You must understand that most web and network related libraries will be async by default for "performance". That's what scares me - async contamination of the low level Rust ecosystem. The async enthusiasts have to be kept in check to prevent breaking Rust as a systems language.
- songqin 6y agoI had understood your concern as wanting to avoid the complexity of asynchronous code execution in your codebase, I did not realize your concern is about writing very low level systems code. In that case, you are doing the right thing: libraries like ureq, minreq, Isahc, curl, and more all offer what you want. It is unclear to me what you mean by keeping the community "in check". There are a lot of people who rely on and enjoy the async story, and they will continue to produce code that improves that story. Simultaneously, there are people who do not need that, and they are not hindered by this. People will build what they want and need. You've just picked some libraries from some of the biggest async contributors in the community and requested that they be kept in check so that you don't have to switch to a synchronous alternative, of which there are plenty.
- AgalmicVentures 6y agoYou're replying to John Nagle, as in https://en.m.wikipedia.org/wiki/Nagle%27s_algorithm https://en.m.wikipedia.org/wiki/Nagle%27s_algorithm -- perhaps it would be worth considering his words further, before dismissing his concerns? Surely we can all agree that spinning up unnecessary threads is undesirable?
- bobthebuilders 6y agoAppeal to authority fallacy
- deleted 6y ago
- deft 6y agoNo, its not a misconception. It still uses tokio. It still triples my compile time.
- h0l0gr4ph1c 6y agoComplexity kills code. Being able to reason about what your code is doing, is FAR more valuable to me than async. Having tokio act as my runtime and switch tasks as it sees fit will be debug hell. The problem I see is the current async story is opt-out. It's use async or go find something else. Async should be opt in. As in, the code works regardless of an async runtime, async is added magic if you want it, but it will run like normal single threaded code if not.
- Animats 6y agoThe problem I see is the current async story is opt-out. It's use async or go find something else. Async should be opt in. That sums this discussion up nicely.
- songqin 6y ago> The problem I see is the current async story is opt-out. Surely this is only the case if you have picked asynchronous libraries to use?
- steveklabnik 6y agoYes, in fact, you cannot even use async Rust without writing your own executor or bringing one in via a library. It is very, very much opt in. That was a hard constraint on the design. However, I think what the parent is getting at is the feeling of the total package, not the technical details. If every library you want to use is async, you can't really "opt out" exactly, even if technically the feature is opt out.
- h0l0gr4ph1c 6y agoBy opt in I mean I can opt in to using an executor if I want async. If I don't code still works and is might not be as performant. Couldn't you have made another hard constraint to make async code work as normal if the programmer wanted?
- steveklabnik 6y ago
- lmm 6y agoYou want libraries to be able to offer implementations that are polymorphically async-or-not (perhaps using higher-kinded types). That's how I'm used to working in Scala.
- geofft 6y agoThis is also a problem over in Python-land. One of the interesting approaches is automatically generating non-async code from async code by editing out the async annotations (at the source level). The Trio folks call this "bleaching" the code, in reference to the "What color is your function?" blog post https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ https://journal.stuffwithstuff.com/2015/02/01/what-color-is-... complaining about how the common approach to async in JavaScript divides the world into "red" (async) and "blue" (sync) functions that interoperate poorly - you can bleach your code if it's the wrong color. https://github.com/urllib3/urllib3/issues/1323 https://github.com/urllib3/urllib3/issues/1323 is a discussion of this. "Solution: we maintain one copy of the code – the version with async/await annotations – and then a little script maintains the synchronous copy by automatically stripping them out again. It's not beautiful, but as far as I can tell all the alternatives are worse. https://github.com/python-trio/unasync https://github.com/python-trio/unasync is the current production implementation of this approach.
- kmeisthax 6y agoIn Rust, you don't need to do this. If you want a blocking version of a non-blocking function, you spin on poll. Several async libraries ship a block_on function that does just that. If you can't do that, but don't want to bring in a proper big-boy executor; you can spin off another thread, have it block, and join it when it's done. (or use something like crossbeam's scoped threads) Colored functions are a problem in JavaScript because it has to share an event loop with the rest of the browser. For Node.js and Python, that is less of an issue, but those languages are also either practically or actually single-threaded, which also means async behavior is contagious. Rust does not have this problem, at least not until you're writing WASM programs with it (in which case, you're restricted to sharing one thread with the rest of the browser again).
- hda2 6y ago> The main thing I want for Async Rust is the ability to not use it. This is what many of us want. Sadly this doesn't seem to be a use-case many in rust are interested in supporting as a first class citizen (from what I've read at least). I remember seeing someone being down-voted into oblivion for the mere suggestion of some sort of basic fallback executor in the standard library. Rust's async story is a weird one.
- GolDDranks 6y agoJust to be clear, you are saying that many 3rd party libraries in the ecosystem are not interested in "supporting as a first class citizen", right? Because as for the language itself and the standard library, they are committed to support both as first-class citizens.
- atweiden 6y agoI'd just like to mention popol here [1]: Popol is designed as a minimal ergonomic wrapper around poll, built for use cases such as peer-to-peer networking, where you typically have no more than a few hundred concurrent connections. It’s meant to be familiar enough for those with experience using mio, but a little easier to use, and a lot smaller. [1]: https://cloudhead.io/popol/ https://cloudhead.io/popol/
- alfiedotwtf 6y ago> "Async" is a specialized tool for web servers with very large numbers of clients. Outside of that niche, it's seldom needed. Web is just 2 ports out of the 65535 TCP/IP ports. Network servers can benefit from an async model depending on the server design. Let’s not sweep all of networking into “web”.