4 ms·

Yes, but when I ask web-server experts, they can't supply a clear explanation for what happens with a typical web application, such as the famed "shopping cart

• nostrademons 7y ago
  Most experts on expert programming forums are idiots. That includes this one, but I've at least worked on an application (Google Search) that I'm pretty sure you've used. Most large websites (with hundreds of millions of users and dev teams in the thousands) don't run all their code in the webserver. They farm out requests to a bunch of backend services running on different boxes - databases, app servers, caches, etc. The central problem with this is that you don't know when a request is going to come back. You don't want to have your webserver simply wait around and do nothing while each request finishes; it'll spend 90% of its time doing nothing, which means you need 10x as much hardware to service the same number of users. But if you have lots of requests to backends in flight, you need some way to synchronize the responses and ensure that the webserver executes the right piece of code only when all of the data needed for it is present. Different languages have different approaches to this. The way we used to do it in C++ is to hand-write a big state machine: each time a response comes back, dump it into memory, consult the current state, move to a new state that might be something like COOKIE_FETCH_COMPLETED or USER_FETCH_COMPLETED or RESULTS_LOADED, execute the code associated with that state (which may kick off even more backend requests), and then wait for the next response to come back, which then triggers the next edge in the state machine, and so on. Async/await just gives language-level support for that. When you await a future, the language interprets it as "execute the next statement of this function only when the data in this future is present. And you can compose futures together, so you're awaiting "all of the data in this list of promises is present". It's a way to apply familiar programming-language concepts like variables and statements to code that's not all running on one box. Do you need async/await? Not if your entire system runs on one box. If every time you make a blocking call, it's just going to a database that runs on the same physical machine as the webserver, you're not really losing anything because the same computer has to do the work anyway. I'd bet that a number of "experts" have only worked on systems like this: if they're talking about OSes and cache sizes and chip architecture, they're not even thinking in terms where concurrency and non-blocking I/O is useful. Oftentimes it's very reasonable to build an initial version like this, but the set of profitable niches that can be served by things like a "shopping cart application" is rapidly declining. Unfortunately, the failure mode for a single-box server with blocking I/O is "your server locks up", and that tends to happen right when it gets most popular, and then you have to rewrite everything if you want to switch it to non-blocking async code. That's the other reason async/await is getting popular: this is one of those areas where scaling is a cliff, not a slope, and most people want to avoid cliffs in their business.
  • tabtab 7y ago
    Re: You don't want to have your webserver simply wait around and do nothing while each request [of another server] finishes; it'll spend 90% of its time doing nothing, which means you need 10x as much hardware to service the same number of users. No, it's processing other users' requests. We are talking about high-traffic sites, right? Let me see if I can present a scenario to explain: R-I-A-I-B-I-C-I-H // typical processing steps R = request received (get or post) H = HTML/HTTP output (response) A,B,C = requests for services from/to other servers I = Internal processing (regular app logic) Even if there is a lot of waiting for A, B, and C, there's still more than plenty of "I" for the web server(s) to work on, because each user request will be at a different stage typically. If say B is slow, it's going to be the bottleneck regardless of whether async/await is used or not. They'll all be waiting on B (at least those requests requiring the same resources). Now maybe you are implying one rewrite the application so that R-I-A-I-B-I-C-I-H's steps don't have to be done in the order shown. But that's NOT what async/await does (at least not how it's typically presented/promoted).
    • nostrademons 7y ago
      You lose the internal concurrency within the request (hoping this formatting goes through): W--- C-H / X \ / |/ \ \ R-I--I--I---I--I-I-I-I-H \/\ / \ /|\ / / A B U \ Y / Z----- R = request I = internal processing C = cache server H = HTML output X = early exit A = Advertising server B = Session cookie server U = User record W,X,Y,Z = application backends Compare vs sequential processing: R-C-I--A-H?-I-B--I-U---I--W-----I-X-I-Y--I--Z--------I-H Note the user-perceived latency. For realistic sites, you may be looking at 400ms in parallel vs 5s or worse in series, which is a big difference in how long your users wait and how likely they are to bounce. This also has knock-on effects in RAM usage. Say that your webserver is running at 100% CPU utilization. If a request takes 10x as long, it will have 10x requests in flight at any given time. They won't be consuming CPU, but they will be consuming RAM, as the server needs to keep all the context around that it'll need later. RAM can often be the bottleneck for many servers. > Now maybe you are implying one rewrite the application so that R-I-A-I-B-I-C-I-H's steps don't have to be done in the order shown. But that's NOT what async/await does (at least not how it's typically presented/promoted). It is if you're doing it right. That's what Promise.all()/race() does in ES6, or asyncio.gather()/wait() in Python3, or Future::join in Rust. If you're not doing this you're basically gaining nothing from async/await.