5 ms·

Not versed enough in Python and asyncio to replicate or understand his benchmark, however from my simplistic view async (or any concurrent framework really) sho

by heipei 6y ago

Not versed enough in Python and asyncio to replicate or understand his benchmark, however from my simplistic view async (or any concurrent framework really) should almost always be faster with any modern kind of application. Let me give an example: If you run a web-service, chances are that you're gonna make some network call as part of processing a request, be it database, network, search index, etc. With async, these calls free up your application to work on another requests until the call returns. I'd say that many modern web-services are just a stitching-together of external calls (get-user-info-from-db, retrieve-user-items-from-db), so the only real work left for the web application to handle is to wait and parse/encode some JSON to client. If you can't max out your bandwidth in terms of JSON performance with one core you just start as many async processes as you have cores. The big advantage over sync processes then is that your async process can also handle insanely-long-running background calls while still crunching through the other requests. Someone please explain to me if I'm missing something here.

twic 6y ago
This isn't correct at all. If you have threads, then when one of them does something blocking, it is suspended, which frees up the processor for other threads to do work. It's not like the computer is sitting there idle waiting for IO to happen. If you're writing synchronous code and not using threads, then yes, your analysis is right, but that would be a daft thing to do! The difference between sync and async is mostly whether the state associated with a task (like an incoming HTTP request being serviced) is kept on a dedicated native thread stack (as it is with sync), or in some sort of coroutine structure (as with async). Thread stacks may be somewhat more efficient, but you have to allocate a whole stack upfront for each thread, so if you want to have lots of threads, you need to dedicate a lot of memory to that, and that goes badly. For applications with small numbers of tasks in flight at once, we shouldn't expect a lot of difference between sync and async code. But for tasks with huge numbers of tasks (chat servers are the classic example, but high-traffic webservers with lots of blocking calls in the backend are another), async code should keep chugging on where sync code just falls over. tl;dr async is about the number of tasks you can handle at once, not the speed with which you handle each task.
- heipei 6y ago
  Yeah no, I get that, but what I'm saying is that with sync I can handle as many requests concurrently as I have processes / OS threads running, which is usually the number of cores in my system. With async, I can have thousands of requests in flight, all of them just waiting for the response of the backend, and all I have to do is to start a single OS thread.
  - arethuza 6y ago
    In a purely sync world you can have tens of threads per processes without much difficulty - so definitely much heavier resource wise than async but not that bad.
    - barrkel 6y ago
      10s? There are over 2000 threads running on my Linux laptop, with a couple of database servers, an IDE and two browsers open. Firefox has 323 threads, Chrome 420, and Slack 69. Async code with continuations and no second execution of continuations is isomorphic to threaded code. The chief differences are that in threaded code, the pointer to the continuation function and its closure are on the stack instead of on the heap, and context switches are done by the OS (pro: fairness, con: context switching overhead). Stack allocation and deallocation is generally faster than heap allocation, but because it's contiguous you need to pay for the high water mark. Even then, that's not expensive unless you have loads of recursion or locally bound state.
      - arethuza 6y ago
        Yeah - I knew it was large numbers but didn't have them handy so used 10s :-)
  - twic 6y ago
    I think what you're missing is that you can have a lot more threads than cores. A lot more! The practical limit is due to the amount of memory needed for each thread stack.
arethuza 6y ago
I don't know the details of async works in Python - but in other platforms non-async is usually thread-per-request (usually with a pool of threads) so its not like your whole application stops when processing makes an outgoing connection to a database etc.
DasIch 6y ago
Your database is realistically postgres or mysql and can really only run as many queries in parallel as you have cores. You can run more queries concurrently but then the database just switches between those, which usually means worse latency and not potentially worse throughput. As you wrote yourself an application handles request probably by doing a few database queries. As a consequence regardless of how many requests you can handle concurrently, they are probably all bottlenecked by the number of CPUs on your database server. All of this is probably true regardless of whether you are using async or sync io. However with async io you have some overhead. Of course, if you also happen to be doing other kinds of IO, say HTTP requests to other services in addition to or instead of those database queries. This might look a bit differently. tl;dr: It really depends on what your application is doing.

by heipei 6y ago

twic 6y ago

This isn't correct at all. If you have threads, then when one of them does something blocking, it is suspended, which frees up the processor for other threads to do work. It's not like the computer is sitting there idle waiting for IO to happen. If you're writing synchronous code and not using threads, then yes, your analysis is right, but that would be a daft thing to do! The difference between sync and async is mostly whether the state associated with a task (like an incoming HTTP request being serviced) is kept on a dedicated native thread stack (as it is with sync), or in some sort of coroutine structure (as with async). Thread stacks may be somewhat more efficient, but you have to allocate a whole stack upfront for each thread, so if you want to have lots of threads, you need to dedicate a lot of memory to that, and that goes badly. For applications with small numbers of tasks in flight at once, we shouldn't expect a lot of difference between sync and async code. But for tasks with huge numbers of tasks (chat servers are the classic example, but high-traffic webservers with lots of blocking calls in the backend are another), async code should keep chugging on where sync code just falls over. tl;dr async is about the number of tasks you can handle at once, not the speed with which you handle each task.

heipei 6y ago
Yeah no, I get that, but what I'm saying is that with sync I can handle as many requests concurrently as I have processes / OS threads running, which is usually the number of cores in my system. With async, I can have thousands of requests in flight, all of them just waiting for the response of the backend, and all I have to do is to start a single OS thread.
- arethuza 6y ago
  In a purely sync world you can have tens of threads per processes without much difficulty - so definitely much heavier resource wise than async but not that bad.
  - barrkel 6y ago
    10s? There are over 2000 threads running on my Linux laptop, with a couple of database servers, an IDE and two browsers open. Firefox has 323 threads, Chrome 420, and Slack 69. Async code with continuations and no second execution of continuations is isomorphic to threaded code. The chief differences are that in threaded code, the pointer to the continuation function and its closure are on the stack instead of on the heap, and context switches are done by the OS (pro: fairness, con: context switching overhead). Stack allocation and deallocation is generally faster than heap allocation, but because it's contiguous you need to pay for the high water mark. Even then, that's not expensive unless you have loads of recursion or locally bound state.
    - arethuza 6y ago
      Yeah - I knew it was large numbers but didn't have them handy so used 10s :-)
- twic 6y ago
  I think what you're missing is that you can have a lot more threads than cores. A lot more! The practical limit is due to the amount of memory needed for each thread stack.

arethuza 6y ago

I don't know the details of async works in Python - but in other platforms non-async is usually thread-per-request (usually with a pool of threads) so its not like your whole application stops when processing makes an outgoing connection to a database etc.

DasIch 6y ago

Your database is realistically postgres or mysql and can really only run as many queries in parallel as you have cores. You can run more queries concurrently but then the database just switches between those, which usually means worse latency and not potentially worse throughput. As you wrote yourself an application handles request probably by doing a few database queries. As a consequence regardless of how many requests you can handle concurrently, they are probably all bottlenecked by the number of CPUs on your database server. All of this is probably true regardless of whether you are using async or sync io. However with async io you have some overhead. Of course, if you also happen to be doing other kinds of IO, say HTTP requests to other services in addition to or instead of those database queries. This might look a bit differently. tl;dr: It really depends on what your application is doing.