5 ms·

> The "async" system is optimized for someone who needs to run a very large web server, with a huge number of clients sending in requests. Can you please elabo

> The "async" system is optimized for someone who needs to run a very large web server, with a huge number of clients sending in requests. Can you please elaborate on this? I see a lot of similar concerns in other contexts too. Linux kernel's scheduler for example. Is it a throughput/latency tradeoff?

cmrdporcupine 3y ago
The current popularity of the async stuff has its roots in the classic "c10k" problem. (https://en.wikipedia.org/wiki/C10k_problem https://en.wikipedia.org/wiki/C10k_problem) A perception among some that threads are expensive, especially when "wasted" on blocking I/O. And that using them in that domain "won't scale." Putting aside that not all of use are building web applications (heterodox here in HN, I know)... Most people in the real world with real applications will not hit the limits of what is possible and efficient and totally fine with thread-based architectures. Plus the kernel has gotten more efficient with threads over the years. Plus hardware has gotten way better, and better at handling concurrent access. Plus async involves other trade-offs -- running a state machine behind the scenes that's doing the kinds of context switching the kernel & hardware already potentially does for threads, but in user space. If you ever pull up a debugger and step through an async Rust/tokio codebase, you'll get a good sense for what the overhead here we're talking about is. That overhead is fine if you're sitting there blocking on your database server, or some HTTP socket, or some filesystem. It's ... probably... not what you want if you're building a game or an operating system or an embedded device of some kind. An additional problem with async in Rust right now is that it involves bringing in an async runtime, and giving it control over execution of async functions... but various things like thread spawning, channels, async locks, etc. are not standardized, and are specific per runtime. Which in the real world is always tokio. So some piece of code you bring in in a crate, uses async, now you're having to fire up a tokio runtime. Even though you were potentially not building something that has anything to do with the kinds of things that tokio is targeted for ("scalable" network services.) So even if you find an async runtime that's optimized in some other domain, etc (like glommio or smol or whatever) -- you're unlikely to even be able to use it with whatever famous upstream crate you want, which will have explicit dependencies into tokio.
- collinvandyck76 3y ago
  >now you're having to fire up a tokio runtime I've been developing in (mostly async) Rust professionally for a about a year -- I haven't written much sync rust other than my learning projects and a raytracer I'm working on, but what are the kind of common dependencies that pose this problem? Like wanting to use reqwest or things like that?
  - Animats 3y ago
    > Like wanting to use reqwest or things like that? Yes. Reqwest cranks up Tokio. The amount of stuff it does for a single web request is rather large. It cranks up a thread pool, does the request, and if there's nothing else going on, shuts down the thread pool after a while. That whole reqwest/hyper/tokio stack is intended to "scale", and it's massive overkill for something that's not making large numbers of requests. There's "ureq", if you don't want Tokio client side. Does blocking HTTP/HTTPS requests. Will set up a reusable connection pool if you want one.
    - zozbot234 3y ago
      You don't have to do that, Tokio also provides a single-threaded runtime that just runs async tasks on the main thread.
    - chromatin 3y ago
      reqwest also has a blocking version, which I use in projects not already using an async rt https://docs.rs/reqwest/latest/reqwest/blocking/index.html https://docs.rs/reqwest/latest/reqwest/blocking/index.html
      - cmrdporcupine 3y ago
        The blocking implementation still depends on and uses tokio, last I looked. I've seen this with multiple Rust packages. "Yes, we offer a synchronous blocking version..." and then you look and it's calling rt.block_on behind the scenes. Which is a pretty large facepalm IMHO

cmrdporcupine 3y ago

The current popularity of the async stuff has its roots in the classic "c10k" problem. (https://en.wikipedia.org/wiki/C10k_problem https://en.wikipedia.org/wiki/C10k_problem) A perception among some that threads are expensive, especially when "wasted" on blocking I/O. And that using them in that domain "won't scale." Putting aside that not all of use are building web applications (heterodox here in HN, I know)... Most people in the real world with real applications will not hit the limits of what is possible and efficient and totally fine with thread-based architectures. Plus the kernel has gotten more efficient with threads over the years. Plus hardware has gotten way better, and better at handling concurrent access. Plus async involves other trade-offs -- running a state machine behind the scenes that's doing the kinds of context switching the kernel & hardware already potentially does for threads, but in user space. If you ever pull up a debugger and step through an async Rust/tokio codebase, you'll get a good sense for what the overhead here we're talking about is. That overhead is fine if you're sitting there blocking on your database server, or some HTTP socket, or some filesystem. It's ... probably... not what you want if you're building a game or an operating system or an embedded device of some kind. An additional problem with async in Rust right now is that it involves bringing in an async runtime, and giving it control over execution of async functions... but various things like thread spawning, channels, async locks, etc. are not standardized, and are specific per runtime. Which in the real world is always tokio. So some piece of code you bring in in a crate, uses async, now you're having to fire up a tokio runtime. Even though you were potentially not building something that has anything to do with the kinds of things that tokio is targeted for ("scalable" network services.) So even if you find an async runtime that's optimized in some other domain, etc (like glommio or smol or whatever) -- you're unlikely to even be able to use it with whatever famous upstream crate you want, which will have explicit dependencies into tokio.

collinvandyck76 3y ago
>now you're having to fire up a tokio runtime I've been developing in (mostly async) Rust professionally for a about a year -- I haven't written much sync rust other than my learning projects and a raytracer I'm working on, but what are the kind of common dependencies that pose this problem? Like wanting to use reqwest or things like that?
- Animats 3y ago
  > Like wanting to use reqwest or things like that? Yes. Reqwest cranks up Tokio. The amount of stuff it does for a single web request is rather large. It cranks up a thread pool, does the request, and if there's nothing else going on, shuts down the thread pool after a while. That whole reqwest/hyper/tokio stack is intended to "scale", and it's massive overkill for something that's not making large numbers of requests. There's "ureq", if you don't want Tokio client side. Does blocking HTTP/HTTPS requests. Will set up a reusable connection pool if you want one.
  - zozbot234 3y ago
    You don't have to do that, Tokio also provides a single-threaded runtime that just runs async tasks on the main thread.
  - chromatin 3y ago
    reqwest also has a blocking version, which I use in projects not already using an async rt https://docs.rs/reqwest/latest/reqwest/blocking/index.html https://docs.rs/reqwest/latest/reqwest/blocking/index.html
    - cmrdporcupine 3y ago
      The blocking implementation still depends on and uses tokio, last I looked. I've seen this with multiple Rust packages. "Yes, we offer a synchronous blocking version..." and then you look and it's calling rt.block_on behind the scenes. Which is a pretty large facepalm IMHO