6 ms·

Is Heavy Use of Async/Await a Fad?

13 points by tabtab 7y ago

The multi-user design of web servers already provides "natural" parallel processing to distribute the processing load. If that's the case, then why the common suggestion to add async/await to all the libraries and even domain code? I'm not an expert on the "guts" of web servers (such as IIS and Apache), but when I ask in related expert forums, it stirs up controversy. According to some the "need" for such is only for heavily used web-sites such that async/await are wasted on roughly 99% of sites. Second, it's allegedly only a performance improvement because of the particular implementation of the common web servers and/or OS's. If the implementation of the web servers is reworked in the future, the benefits of async/await might just go away. Some say "adding async/await still doesn't hurt", but it can clutter up code and make certain debugging harder. Thus, having it scattered about is not a free lunch.

nostrademons 7y ago
That's not the point of async/await, or of any concurrency strategy in general. The point is backend fan-out and the associated latency that goes with it. Most modern webservices - ones that do useful things at least - need to make a number of network calls to backend services to retrieve results. If you single-thread the webserver, these get executed in series, and each one of them blocks the whole webserver until it completes. Even thread-per-request or process-per-request, you're still blocking the individual request until all backend requests complete; you're just sitting there waiting on a database fetch while you have other backends running on other computers that could be doing useful work. That's how you end up with 2s requests where the user's browser just hangs while loading. Worse, each of those threads take up a few megabytes of RAM that can't be freed until the request completes, so if you have a lot of users requesting pages at once, there's a big risk of the server paging to swap (which will absolutely kill performance) or running out of RAM entirely and crashing. The only way to avoid this is to have some sort of concurrency within the request, where you can you fetch your session cookie and user record from Redis at the same time as you check memcached for a cached version of the page at the same time as you kick off a backend request to a custom server at the same time as you read some records out of a database. Different languages have different approaches for this: old-school Java used to use threadpools and message queues, pre-ES6 Node.js would use callbacks, C++ would use state machines, Django or Rails would just single-thread the backend requests and live with the 10 requests/sec throughput. Async/await has gained popularity lately because it's honestly a lot more intuitive than the alternatives: you can at least use normal language mechanisms (variables, arrays, and semicolons) to indicate "these things should happen in series, and these things should happen in parallel" instead of breaking your server up into little function-sized or class-sized chunks that are dictated by when responses come back instead of what data needs to be processed.
- tabtab 7y ago
  Re: If you single-thread the webserver, [network calls] get executed in series Why is the webserver single-threaded then? And even if it were, why can't network calls be fanned out within it? If they made multi-threaded webservers, would that solve the problem? Re: Worse, each of those threads take up a few megabytes of RAM that can't be freed until the request completes Do you mean user page requests, or RAM related to network calls? If the first, that probably will happen no matter what. I'm trying to sketch little timing diagrams to see where the bottleneck could come from, and cannot see any "general" problem, only problems related to specific architecture choices. But again, I'm not a systems-level software expert. Is there a common concept behind the bottleneck, lots of architecture-specific nuances adding up (OS+WebSrvr+Compiler), or something else?
  - nostrademons 7y ago
    Are you familiar with the difference between blocking & non-blocking I/O calls? Also, have you done any distributed systems work, where the algorithm that services a given request might require computation on different machines? That's background knowledge that's important; it's hard to know where to start without knowing your background.
    - tabtab 7y ago
      Yes, but when I ask web-server experts, they can't supply a clear explanation for what happens with a typical web application, such as the famed "shopping cart application". Or if they do, another expert chimes in and says Expert A's explanation is technically wrong, and then Expert B argues over Expert A's knowledge of how web servers/OS's work or could work, and they delve into arguments over minutia about cache size and chip architecture, and/or bad design of C#'s compiler, etc. I open Pandora's Box but can't close it. Thus, I cannot get the bottom of whether heavy async/await usage is to get around quirks in current systems, or solve a general/universal problem regardless of how web-servers/OSs/compilers/chips are built.
      - nostrademons 7y ago

Most experts on expert programming forums are idiots. That includes this one, but I've at least worked on an application (Google Search) that I'm pretty sure you've used. Most large websites (with hundreds of millions of users and dev teams in the thousands) don't run all their code in the webserver. They farm out requests to a bunch of backend services running on different boxes - databases, app servers, caches, etc. The central problem with this is that you don't know when a request is going to come back. You don't want to have your webserver simply wait around and do nothing while each request finishes; it'll spend 90% of its time doing nothing, which means you need 10x as much hardware to service the same number of users. But if you have lots of requests to backends in flight, you need some way to synchronize the responses and ensure that the webserver executes the right piece of code only when all of the data needed for it is present. Different languages have different approaches to this. The way we used to do it in C++ is to hand-write a big state machine: each time a response comes back, dump it into memory, consult the current state, move to a new state that might be something like COOKIE_FETCH_COMPLETED or USER_FETCH_COMPLETED or RESULTS_LOADED, execute the code associated with that state (which may kick off even more backend requests), and then wait for the next response to come back, which then triggers the next edge in the state machine, and so on. Async/await just gives language-level support for that. When you await a future, the language interprets it as "execute the next statement of this function only when the data in this future is present. And you can compose futures together, so you're awaiting "all of the data in this list of promises is present". It's a way to apply familiar programming-language concepts like variables and statements to code that's not all running on one box. Do you need async/await? Not if your entire system runs on one box. If every time you make a blocking call, it's just going to a database that runs on the same physical machine as the webserver, you're not really losing anything because the same computer has to do the work anyway. I'd bet that a number of "experts" have only worked on systems like this: if they're talking about OSes and cache sizes and chip architecture, they're not even thinking in terms where concurrency and non-blocking I/O is useful. Oftentimes it's very reasonable to build an initial version like this, but the set of profitable niches that can be served by things like a "shopping cart application" is rapidly declining. Unfortunately, the failure mode for a single-box server with blocking I/O is "your server locks up", and that tends to happen right when it gets most popular, and then you have to rewrite everything if you want to switch it to non-blocking async code. That's the other reason async/await is getting popular: this is one of those areas where scaling is a cliff, not a slope, and most people want to avoid cliffs in their business.