16 ms·

I don't really get these modern async APIs. In languages like Javascript I thought they only made sense because JS interpreters are (historically) single-thread

• jayd16 6y ago
  Maybe its a matter of taste but I find async styles much easier to read than event driven coding.
  • sanxiyn 6y ago
    Comparison is to threads, not to event callbacks. Threads are (at least in Rust) even easier to read than async.
• artursapek 6y ago
  It's syntactic sugar that makes code a lot easier to reason about.
  • sanxiyn 6y ago
    Synchronous code is even easier to reason about than code using async syntax sugar.
    • nialv7 6y ago
      No, multithreaded synchronous code is not easier to reason about than async. I think most would agree.
      • akvadrako 6y ago
        You probably think async code can't have deadlocks like threaded code can. Async is still multi-threaded it's just that the threads are done in user-space and contain an implicit global lock.
        • nicoburns 6y ago
          Async is not pre-emptive though (if using a single theaded executor). And thats where most of the unintutive behaviour of threads comes from.
        • nialv7 6y ago
          Notice I didn't say async code is easier to reason about.
      • sanxiyn 6y ago
        I disagree, at least in Rust. async is hard. Threading is easy.
        • nialv7 6y ago
          Do you have something that backs up your belief?
• xucheng 6y ago
  In a sense, the async API allows you to create green threads. Comparing with the normal threads, which rely on OS schedule and introduce context switches, green threads allow tasks actively yielding the control. This for example can be used to run multiple IO tasks concurrently all in one single OS thread. See https://en.wikipedia.org/wiki/Green_threads https://en.wikipedia.org/wiki/Green_threads
  • sanxiyn 6y ago
    Yes, but why would you want that? Using OS threads is better in most cases.
    • 10000truths 6y ago
      OS threads are preemptive and their scheduling is beyond your control, which introduces potential data races. Green threads allow for cooperative scheduling, which is much harder to mess up.
      • sanxiyn 6y ago
        This is moot. Rust prevents data races.
        • brundolf 6y ago
          It doesn't prevent race conditions, which may be what the GP meant to say
      • Yoric 6y ago
        In my experience, past some level of code complexity, you get the pretty much same kind of races with cooperative scheduling as with preemptive scheduling – with the added risk that you did not expect a race with preemptive scheduling. Still a fan of async programming, but that's a false benefit in my books :)
    • Groxx 6y ago
      Most is not all. And some systems (especially embedded) do not have OS threads - they can still make use of green threads / async code.
• steveklabnik 6y ago
  Async/await matters in Rust specifically because the borrow checker makes writing code without it difficult, inefficient, and unergonomic: http://aturon.github.io/tech/2018/04/24/async-borrowing/ http://aturon.github.io/tech/2018/04/24/async-borrowing/ (note that some of the details have changed here, but the thrust of it is very much the same.) That said, if you can get your job done without this stuff, that's fine too, but the reasons it was pursued specifically involve the above.
  • sanxiyn 6y ago
    It all comes down to "is it such a common scenario?". Honest answer is no. async is for specialized situations and shouldn't be the default.
    • steveklabnik 6y ago
      Depending on the kind of work you're doing, it may or may not be. If you're doing a lot of network IO, then yes, it is very common.
    • ibraheemdev 6y ago
      async is not the default. The standard library is 100% blocking, and Rust does not come with a runtime. However, async makes sense for a lot of people, which is why libraries like tokio and async-std are so popular.
      • sanxiyn 6y ago
        async does not make sense for a lot of people and I am deeply worried by hype-driven popularity of Tokio and async-std in Rust. Especially harmful is that common libraries like reqwest (for HTTP requests) pulls async.
        • ksenzee 6y ago
          Shouldn't an HTTP request library make async available? HTTP is the textbook example for async.
• ibraheemdev 6y ago
  > when you have a huge number of very small tasks running concurrently because that's generally where OS-driven parallelism tends to suffer but is it such a common scenario Web servers are all about I/O and handling small tasks (requests), and are a perfect use case for asynchronous programming. > That sounds like premature optimization in many situations IMO Maybe in some cases... but then just don't use futures. Rust does not have a runtime, so it gives you the choice. std is all blocking, so you can spawn threads and do event-driven programming, which might be just fine for a lot of people. async is more ergonomic for Rust specific reasons, makes sense for a lot of use cases, and was a highly requested language feature, so it was added to the language. OS threads can work just fine for many people. If that includes you, you don't have to use async.
  • sanxiyn 6y ago
    You may want async if you are writing the next nginx, but for most web application servers threading is perfectly fine.
    • Xorlev 6y ago
      Again, depends on if you're doing a lot of IO, especially if you want request-based IO parallelism. It's worth mentioning that even if you're IO-bound at your DB, running an async application server now means you don't need to tie up a thread waiting on it. Memory usage aside, you more or less don't need to think about threads much, whereas a threadpool (one waiting on IO) is something you have to actively manage.
    • deleted 6y ago
      [deleted]
    • maccam94 6y ago
      But nginx is usually just a proxy for connecting to app servers, and your app servers still need to handle all of those requests.
    • mycoliza 6y ago
      Rust is intended as a systems programming language, it's for people who are writing "the next nginx". It turns out that there are also a bunch of people who want to write webapp servers in Rust, too, but that's never really been the goal.
• akvadrako 6y ago
  There are some pretty bad usability problems with most async APIs too. One is they make your functions colored; async functions world best with other async functions while normal blocking functions work best with other blocking functions. They also introduce a lot of noise; putting async/await everywhere doesn't tell you anything interesting. Considering a normal-sized Linux server can handle a million threads without much trouble, it really seems like misplaced effort.
  • sanxiyn 6y ago
    Zig's "colorblind" async is very exciting for this reason: https://kristoff.it/blog/zig-colorblind-async-await/ https://kristoff.it/blog/zig-colorblind-async-await/
    • haberman 6y ago
      I'm skimming your link trying to understand the design. It sounds like there is a global flag for whether the whole program is in evented or blocking mode? > during compile-time, it’s possible to inspect if the overall program is in evented mode or not, and properly designed code might decide to move to a threaded model when in blocking mode, for example.
      • sanxiyn 6y ago
        Yes, it's a top level application decision, not a library decision. In fact, this is exactly how allocation and unwinding is handled in Rust.
        • zozbot234 6y ago
          So, it's like the choice of a Tokio executor? That's a top-level default, too.
    • lmm 6y ago
      Sounds like a horribly complex special case. I'd far rather just have higher-kinded types and be able to write sometimes-async code using normal polymorphism (like I do in Scala all the time).
• Yoric 6y ago
  > Unless you actually think that async code is more expressive and easy to read and write than basic event loops but then you must be a lot smarter than I am because I have to take an aspirin every time I need to dig into async-heavy code. I don't know about the end of this sentence, but yeah, I'm the kind of person who enjoys very much writing async code and finds that it (sometimes) models the problem much better than sync code and much, much, much better than event-driven/polling programming. But then, I'm also the kind of person who enjoys coding with CML channels (aka Rust mpsc channels aka Go channels aka Erlang mailboxes aka pi-calculus channels etc.), so I guess I might be a mutant.
• nialv7 6y ago
  I don't quite understand why one would find async code hard to read. Basically if you just ignore the async/await keyword, the code should read mostly the same as synchronous code (which is the point of the async/await effort). Maybe you have a concrete example of convoluted async code?
  • pvorb 6y ago
    While it might not be that hard to read, it can certainly be harder to reason about.
    • mratsim 6y ago
      Concurrent code is hard to reason about in general: async or threading included.
• tijsvd 6y ago
  I had the same thing initially. The upside of async over a simple event loop is, in my experience, when things become less simple, and you end up with hard-to-read little state machines all over the place. With async, you can have your event loop, but the state machines are handled by the compiler. Code is like threaded code. That can be very convenient. Threads, obviously, accomplish the same thing, and arguably more easily. But threads have a performance problem when they must interact heavily. Cross-thread communication is expensive. Single-threaded async task interaction is very cheap, comparatively (I use tokio only in single-threaded mode, as an event loop replacement; its multi-threaded scheduler performs terribly on serious I/O). I think the interaction problem is often more important than just the number of tasks. As for the function coloring argument, I've started to see the async keyword as documentation. A non-async function is "regular logic", it must complete without blocking. An async function is a state machine; as such it must be part of a larger state machine (the event loop), and it can go down into smaller sub-state machines (i.e. call other async functions). If you make that distinction explicit, it all makes a lot of sense.
  • sanxiyn 6y ago
    > A non-async function is "regular logic", it must complete without blocking. Maybe one can enforce this convention in the particular project, but there's no ecosystem-wide consensus on this, and in fact I don't want this to be consensus. I write blocking non-async functions every day. Why am I wrong to do so?
    • josephg 6y ago
      There is consensus in some ecosystems. Javascript absolutely maintains that invariant. There are (almost) no blocking functions in the javascript / node standard libraries and we work hard to keep it that way. Go maintains similar discipline at the OS syscall level. I feel like the "what color is your function" thing is incomplete. There are arguably 3 types of functions: - Functions which do all their work synchronously and return without blocking - Async functions which contain an internal state machine - Functions which block on expensive IO or long computations Mixing blocking functions and async functions in the same kernel thread leads to various performance disasters. Javascript is so meticulous about not having blocking IO in part because its basically impossible to tell from a function's signature whether it will block the thread. Lua has this problem - callback oriented lua feels like a natural fit for the language, but lots of 3rd party libraries are packed with blocking calls. Writing asyncronous lua feels like fighting a river. You have to constantly guard against calling blocking code, and most API docs won't tell you where they block.
• efaref 6y ago
  How many threads can you spawn before the system grinds to a halt? If you're processing thousands of requests per second and each request gets its own thread then you will start to queue on thread spawning. Don't forget that each thread gets its own stack taking up megabytes of memory. The async concept has been used for decades in pretty much every product I've worked on professionally, from enterprise raid controllers to network protocol implementations and telephony software. An engineer I respect once told me that really it's the only way to write services at scale, and anything else is just a step on the road until you reinvent it. He was probably exaggerating, but it is very important, and nearly ubiquitous. Having used custom frameworks for async code in C and C++, it's really refreshing to have it baked into the language and well supported. It's yet another arrow in Rust's fantastic quiver.
  • sanxiyn 6y ago
    rouille (my Rust web framework of choice) spawns threads each request, and it handles thousands of requests per second just fine. Computers are fast, and Rust doesn't slow down your computer. If you can't handle thousands of requests per second with thread per request, that's more about your software stack, not about threading. I guess it was different in the past when computers were slow. I can believee that.
    • secondcoming 6y ago
      > it handles thousands of requests per second just fine It probably doesn't. Spawning threads per request is a lazy pattern. We use Apache to handle billions of requests per day, and even its thread pool can be an issue.
    • pvorb 6y ago
      And once starting and stopping threads adds to much delay to your request processing, there could be a thread pool that grows as needed and which will reuse threads that haven't been closed yet. This mechanism is implemented by Apache httpd, Tomcat and pretty much every classic application server.
      • 6y ago
• MrBuddyCasino 6y ago
  > I guess I'm trying to understand if it's me who's missing something No you’re not. There is a similar situation in Kotlin, which supports coroutines. Makes things more complicated and is often used for very questionable reasons. This is why I’m excited about project Loom, which will use the same old thread abstraction but can be configured to use fibers under the hood instead. Java devs don’t even care that its not traditional threading, its the same API! This simultaneously solves the „coloredness“ problem of functions.
  • EvilEy3 6y ago
    > Makes things more complicated and is often used for very questionable reasons. Why is it complicated? Because you didn't bother to learn new API and it is sufficiently different from older one?
• lights0123 6y ago
  A very useful thing is that async stuff is easily cancelled in Rust, not so much threads—just making it go out of scope causes it to be cancelled in Rust, while you'd need to add cancellation logic to threads.
• shmerl 6y ago
  I think async in this sense leverages concurrency, which isn't necessarily tied to parallelism. I.e. parallelism means executing several tasks on different compute units (cores) at the same time (multiple threads abstraction). Concurrency simply means that you can execute several tasks over a period of time, but it can happen on the same compute unit (so even within one thread). Tasks could be interleaved and still all progress over time. I guess some ideal usage is a combination of parallelism and concurrency, but using a separate thread for each task isn't necessarily the most optimal method, because threads have their gotchas like context switching and etc.
• hinkley 6y ago
  > use OS-provided async IO primitives to poll file descriptors and do event-driven programming etc... My understanding from reading along with many of these conversations is that the ultimate goal of the async IO frameworks is to amortize system call costs across multiple IO operations, instead of having >1 per. I've heard some speculation about system call overhead going up in order to guarantee correctness (for Spectre and Meltdown class scenarios, but also for some other classes of concurrency issues). In which case io_uring is the carrot and system call slowdown the stick.
• rtpg 6y ago
  the "async/futures" way of writing code lets you write code that "looks like synchronous code" (write step 1, then step 2, then step 3, etc), while getting really good safety guarantees and not having to manage a state machine yourself. The model doesn't work for all forms of concurrency, but I think it works for a lot of things that people at the "top of the stack" (application developers) do. I don't know what kind of code you're looking at, in general, but you should be able to massage most async stuff into a list of things if you're not in callback soup. Granted, lots of people don't try to stay out of the callback soup, but.... I feel like even that is better than just like "try to validate concurrency invariants", which is a much harder problem for arbitrary code IMO?
• jeff-davis 6y ago
  Async seperates the concurrency from the runtime completely. You have code that returns a Future, and creates more Futures along the way. None of this imposes any constraint on the runtime, except that you need some runtime to evaluate the future. But that runtime could be quite simple and execute in the current process/thread (cf. the CurrentThread runtime), meaning you don't need support for threads at all. Contrast this with the thread model, where the runtime needs to create and destroy threads where the code asks for it. In other words, your program logic is mixed up with runtime considerations. For a practical example, let's say you want to use rust to extend some C code to create a network client that calls back into the C code. What if the C code is not thread-safe? With async, no problem, just use the CurrentThread runtime. This is my use case, anyway.
  • megaman821 6y ago
    That is an interesting positive for async/await that I had not heard before. Why are there so many libraries that depend on Tokio then? If only the edges need to actually use an async runtime, the middle-tier libraries can just be plain futures.
    • jeff-davis 6y ago
      Because the standard library (and a lot of other libraries) are mostly blocking. You can't use blocking code effectively with async. In other words with async you have to use code that knows how to yield, but the benefit is that you have a lot of flexibility in the runtime. That means you can scale up, scale down, and fit it into weird environments (like in other programs that aren't expecting concurrency). Threading (or process model) has a lot of upside though, too. For one, it's pre-emptive, so scheduling can be more "fair". For another, it's a lot easier to debug (erlang does a great job here). I'd generally default to using threads, unless I have a specific reason for async and/or the code is fairly low-level and might be used in a variety of environments. Disclaimer: don't take my claims as authoritative. I know a few things from seeing what works and not, but I could be wrong on some of the finer points.
• KingOfCoders 6y ago
  I really like the Golang runtime. It uses coroutines (async) to speed up things (no context switches, low memory overhead) and automatically moves IO to threads where they can wait.
• scottlamb 6y ago
  I think async was inevitable for Rust, for better or for worse. My experience is that just using OS-provided threads isn't good enough for say a high performance webserver—compare the pre-tokio hyper benchmark results to the post-tokio ones for example. And Go-like green threads aren't really possible in Rust given the choice to have no runtime. (Having a stack for every coroutine also probably isn't as good for squeezing out that last bit of performance. Every stack/guard page means more TLB pressure, for example, and programs can spend a lot of time on TLB cache misses.) Rust aims to be suitable for high-performance, low-level environments, so here we are. I agree async has usability problems. Hopefully they'll get better over time; I'm looking forward to eventually having generators rather than dealing with futures::stream::StreamExt and the like. I don't think everyone needs to use async all the time. Take a web app, for example. If the webserver is directly Internet-facing, it has to deal with lots of keepalive connections, so the core webserver logic (hyper or equivalent) should be async. But if you're not dealing with too many active requests and don't care about the performance difference, I don't think there's any reason you shouldn't have all your request handling just use threads, sending replies to hyper with a channel and blocking when necessary. Last I checked I couldn't find an ergonomic and efficient "half-async" bounded channel implementation. By which I mean one that allows you to treat the sender as blocking and the receiver as async, or vice versa. That'd be really useful for writing synchronous programs that use async libraries. I certainly don't see any reason one couldn't exist. Maybe it already does and I missed it.
  • pimeys 6y ago
    I was trying to look for what channel I used back in the days I needed to write so called half-async code, but I can't find it anymore. If I remember correctly, at least the 0.1 version of futures had a channel with the other end being sync and the other async. Today I'd maybe take a look into crossbeam and their queue implementations for this kind of synchronization. https://docs.rs/crossbeam/0.8.0/crossbeam/ https://docs.rs/crossbeam/0.8.0/crossbeam/
• dastx 6y ago
  I've not done nearly enough multithreaded programming, so this may be out of my depth, and what I'm saying and asking maybe completely wrong. Isn't async by design more efficient even when you have multiple threads you can spawn? My understanding is the event loop would do async tasks in the thread's quiet times, and put those tasks to sleep while it's waiting for io and other things, meaning the thread isn't blocked. Comparing that to (my understanding of) threads, while you're not blocking the main thread, you're still blocking the spawned threads while waiting for io. Isn't this thread blocking something you would want to avoid if you can regardless of whether or not you have additional threads to play with?
  • fsociety 6y ago
    Yes! Programs can be much more efficient with non-blocking operations and a scheduler. This doesn't just benefit high performance web servers. You gain some distinct advantages too by doing this, because you can then just run a single thread in your worker pool and if you ever need to make the program multi-threaded - god forbid - you can add locks to shared resources (or in Rust's case the compiler will help you with this) and then bump up the number of threads. CPUs are really really fast today, I think engineers generally underestimate the amount of performance you can get with a single-thread and non-blocking I/O. Async language constructs make this process a bit easier. The only issue IMO is that it is awkward to call async functions synchronously, or that it can have hidden costs to do so. I think languages will improve on this. I believe on Linux, using non-blocking system calls on threads can help reduce expensive context switches too... whereas spawning multiple threads and having them use blocking system calls can cause more context switches. I will say though.. I've seen developers just async-ify everything in codebases without thinking about why or if it is beneficial.
• littlestymaar 6y ago
  > you can just spawn threads, share data through channels or mutexes, use OS-provided async IO primitives to poll file descriptors and do event-driven programming […] > I have to take an aspirin every time I need to dig into async-heavy code. I've seen that a lot on the internet and I guess it must depend where you come from, because I find async/await orders of magnitude easier to reason about than threads+channel (and I'm not even talking about using epoll manually, which is just inscrutable as soon as there is a little bit of complexity involved)
• NMDaniel 6y ago
  You're talking about the runtime aspects of executing concurrent code (OS threads vs green threads) but these are orthogonal to the idea of async APIs, which are one way of modelling concurrent programming flows. For example, futures in Rust can be used with both OS threads and lightweight tasks. Tokio is mostly agnostic about the choice of the executor. It definitely takes some time to grok async Rust (even if you come from C#/JS), but I think it really shines once you get to know it, similar to the benefits you get from learning about iterators & higher level functions as opposed to plain loops. For instance, I've recently implemented the Raft protocol as part of a distributed algorithms course. Using Tokio and a single threaded executor made the implementation fairly readable, mostly relying on few async constructs(futures, tasks, channels and select loops) to model fairly complex behavior (bidirectional messaging, multiple states, timeouts, etc..) Doing so in a more traditional callback oriented style would've required maintaining a very complex state machine(in addition to the state machine of the algorithm itself) Also, Async/Await originated in C#, a language which already supports threads(and many other concurrency models), not JavaScript which historically relied on callbacks