4 ms·

Obviously the async framework introduces some overhead, but that bit of overhead is probably a lot less than the 3 billion cpu cycles you'll waste waiting 1000m

Obviously the async framework introduces some overhead, but that bit of overhead is probably a lot less than the 3 billion cpu cycles you'll waste waiting 1000ms for an external service. Waiting for I/O does usually not waste any CPU cycles, the thread is not spinning in a loop waiting for a response, the operating system will just not schedule the thread until the I/O request completed.

toxik 6y ago
Sigh. Async is somewhat orthogonal to parallel. You are making dinner. You start to boil water for the potatoes. While that happens, you prepare the beef. Async. You and your girlfriend are making dinner. You do the potatoes, she does the beef. Parallel. You can perhaps see how you could have asynchronous and parallel execution at the same time. In the context of a Web server, a request is handled by a single Python process (so don’t give me that “OS scheduler can do other things”). Async matters here because your request turnover can be higher, even if the requests/sec remains the same. In the cooking example, each request gets a single cook. If that cook is able to do things asynchronously, he will finish a single meal faster. If it were only parallel, you could have more cooks - because they would be less demanding - but they would each be slower.
- danbruc 6y ago
  I do not disagree with that, my point was just that you are not wasting clock cycles, you may however, as you pointed out, be wasting time while waiting for I/O to complete which you could potentially make better use of by using some more clock cycles while the I/O operation is in progress to do more work which is not dependent on the I/O result.
  - toxik 6y ago
    I didn’t mean to disagree with you, I just wanted to put my take on it out there
- stock_toaster 6y ago
  > In the cooking example, each request gets a single cook. If that cook is able to do things asynchronously, he will finish a single meal faster. There is a bit of nuance here, in that the async-chef would make any individual meal slower than a sync-chef, once the number of outstanding requests is large. The sync-chef would indeed have overall higher wait times, but each meal would process just as fast as normal (eg. more like a checkout line at a grocery store). I prefer the grocery store checkout line metaphor for this reason. If a single clerk was "async" and checking out multiple people at once, all the people in a line would have an average wait time of roughly the same for a small line size. A "sync" clerk would have a longer line with people overall waiting longer, but each individual checkout would take the same amount of time once the customer managed to reached the clerk. This is pertinent when considering the resources utilized during the job. If an sync clerk only ever holds a single database connection, while an async clerk holds one for every customer they try to check out at the same time, the sync clerk will be far more friendly to the database (but less friendly to the customers, when there aren't too many customers at once).
  - toxik 6y ago
    I think you managed to miss the point: the async chef is doing other stuff necessary to fulfill a single order when he can, i.e., while the potatoes are boiling. The sync chef has to wait for the potatoes to boil, only when those are done can he start to fry the beef. The sync chef doesn't occupy the frying pan when he's boiling potatoes, so in some sense he only really does as much as he can. Having hundreds of sync chefs would likely be more efficient in terms of order volume, _but not order latency._
maxmalysh 6y ago
https://en.wikipedia.org/wiki/C10k_problem https://en.wikipedia.org/wiki/C10k_problem

toxik 6y ago

Sigh. Async is somewhat orthogonal to parallel. You are making dinner. You start to boil water for the potatoes. While that happens, you prepare the beef. Async. You and your girlfriend are making dinner. You do the potatoes, she does the beef. Parallel. You can perhaps see how you could have asynchronous and parallel execution at the same time. In the context of a Web server, a request is handled by a single Python process (so don’t give me that “OS scheduler can do other things”). Async matters here because your request turnover can be higher, even if the requests/sec remains the same. In the cooking example, each request gets a single cook. If that cook is able to do things asynchronously, he will finish a single meal faster. If it were only parallel, you could have more cooks - because they would be less demanding - but they would each be slower.

danbruc 6y ago
I do not disagree with that, my point was just that you are not wasting clock cycles, you may however, as you pointed out, be wasting time while waiting for I/O to complete which you could potentially make better use of by using some more clock cycles while the I/O operation is in progress to do more work which is not dependent on the I/O result.
- toxik 6y ago
  I didn’t mean to disagree with you, I just wanted to put my take on it out there
stock_toaster 6y ago
> In the cooking example, each request gets a single cook. If that cook is able to do things asynchronously, he will finish a single meal faster. There is a bit of nuance here, in that the async-chef would make any individual meal slower than a sync-chef, once the number of outstanding requests is large. The sync-chef would indeed have overall higher wait times, but each meal would process just as fast as normal (eg. more like a checkout line at a grocery store). I prefer the grocery store checkout line metaphor for this reason. If a single clerk was "async" and checking out multiple people at once, all the people in a line would have an average wait time of roughly the same for a small line size. A "sync" clerk would have a longer line with people overall waiting longer, but each individual checkout would take the same amount of time once the customer managed to reached the clerk. This is pertinent when considering the resources utilized during the job. If an sync clerk only ever holds a single database connection, while an async clerk holds one for every customer they try to check out at the same time, the sync clerk will be far more friendly to the database (but less friendly to the customers, when there aren't too many customers at once).
- toxik 6y ago
  I think you managed to miss the point: the async chef is doing other stuff necessary to fulfill a single order when he can, i.e., while the potatoes are boiling. The sync chef has to wait for the potatoes to boil, only when those are done can he start to fry the beef. The sync chef doesn't occupy the frying pan when he's boiling potatoes, so in some sense he only really does as much as he can. Having hundreds of sync chefs would likely be more efficient in terms of order volume, _but not order latency._

maxmalysh 6y ago

https://en.wikipedia.org/wiki/C10k_problem https://en.wikipedia.org/wiki/C10k_problem