5 ms·

Great article. But I think this part may need as second look: If your views involve heavy-lifting calculations or long-running network calls to be done as

Great article. But I think this part may need as second look: If your views involve heavy-lifting calculations or long-running network calls to be done as part of the request path, it’s a great use case for using async views. That seems true for long-running network calls (IO). But for heavy-lifting calculations? I thought that was the canonical example of situations async won't improve. CPU bound and memory bound, after all.

Znafon 6y ago
You are correct, async will only help for long-running network calls, which happens when calling another service or querying a database. When doing a long computation the CPU is not idle so there is no free compute power to use for something else. Finally, when doing IO calls in Python so GIL is usually released so the kernel can already schedule another thread while waiting for IO, so it is not sure that converting to async will yield improvement and should be benchmarked if you plan on converting an existing program.
- pdonis 6y ago
  > when doing IO calls in Python so GIL is usually released so the kernel can already schedule another thread while waiting for IO This is true, but scheduling another thread through the kernel can have higher overhead since it requires context switches. Running multiple threads also has other potential issues with lock contention; how problematic they are will depend on the use case. The potential advantage of scheduling another thread is, of course, that it can do CPU bound work; but in Python, unfortunately, doing that means the GIL doesn't get released so that thread will prevent any further network I/O while it's running, the same as would happen in an async framework if a worker did a lot of CPU work. So Python doesn't really let you realize the advantages of threads in this context.
  - Znafon 6y ago
    > doing that means the GIL doesn't get released so that thread will prevent any further network I/O while it's running, the same as would happen in an async framework if a worker did a lot of CPU work. So Python doesn't really let you realize the advantages of threads in this context. I don't think that's true, the GIL get released for many computing intensive or IO bound tasks in Python, for example when reading from a socket the GIL gets released at https://github.com/python/cpython/blob/e822e37946f27c09953bb5733acf3b07c2db690f/Modules/socketmodule.c#L899 https://github.com/python/cpython/blob/e822e37946f27c09953bb...
    - pdonis 6y ago
      > the GIL get released for many computing intensive or IO bound tasks in Python I/O bound, yes, since that requires system calls, and system calls (reading from a socket is an example of a system call) release the GIL. Computing intensive, no. Code that is doing a CPU intensive computation but makes no system calls will never release the GIL.
      - jashmatthews 6y ago
        The GIL gets released whenever it gets released. C extensions like zlib release the GIL while (de)compressing requests/responses. https://docs.python.org/3/c-api/init.html#releasing-the-gil-from-extension-code https://docs.python.org/3/c-api/init.html#releasing-the-gil-...
        pdonis 6y ago
        > C extensions like zlib release the GIL while (de)compressing requests/responses Yes, you're right, my statement was too broad.
      - Znafon 6y ago
        > Computing intensive, no. Code that is doing a CPU intensive computation but makes no system calls will never release the GIL. Any code that does not involve Python objects can release the GIL, no matter whether it makes system call or not. For example, NumPy the most popular scientific computation package in Python, on which many other popular packages like Pandas are based, releases the GIL when doing operation on matrix. This is documented at https://numpy.org/doc/stable/reference/internals.code-explanations.html#function-call https://numpy.org/doc/stable/reference/internals.code-explan...: > If NPY_ALLOW_THREADS is defined during compilation, then as long as no object arrays are involved, the Python Global Interpreter Lock (GIL) is released prior to calling the loops. It is re-acquired if necessary to handle error conditions. To do so it uses the same macro used by the socket module when doing system calls: https://github.com/numpy/numpy/blob/18a6e3e505ee416ddfc617f3e9afdff5a031c2c2/numpy/core/include/numpy/ndarraytypes.h#L959 https://github.com/numpy/numpy/blob/18a6e3e505ee416ddfc617f3...
  - zzzeek 6y ago
    there is an advantage to threads in the CPU bound case which is that the work of other threads will not be blocked for a CPU-intense operation. With an IO-event based scheduler, your CPU bound task will not context switch leading to network logic elsewhere to simply time out. A particularly acute example is something like a network library logging into the MySQL database which gives the client a ten second window to respond to the initial security challenge. It was both an extremely difficult bug for me to diagnose as well as helpful for my role at work that I was able to track that one down in Openstack :).
ghostwriter 6y ago
Perhaps they meant that heavy long-running calculations could be offloaded to a worker pool with a help of concurrent futures and run_in_executor() - https://docs.python.org/3/library/concurrent.futures.html https://docs.python.org/3/library/concurrent.futures.html - https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor https://docs.python.org/3/library/asyncio-eventloop.html#asy...
- pdonis 6y ago
  This will only help if the workers are separate processes. Thread workers will hold the GIL in Python and prevent network I/O while they are doing CPU bound tasks.
  - ghostwriter 6y ago
    sure, the pool only cares about concurrent.futures.Executor interface, an implementation could be processes or cloud resources.
  - dr_zoidberg 6y ago
    > Thread workers will hold the GIL in Python and prevent network I/O while they are doing CPU bound tasks. Using cython: with nogil: # whatever you need to do, as long as it # doesn't touch a python object If you're doing heavy calculations from python you should at least be considering cython.
dec0dedab0de 6y ago
I thought the only reason to use ASGI is to use web sockets, and the only reason to use web sockets is to avoid making multiple requests for things that don't matter if a particular message is lost.

Znafon 6y ago

You are correct, async will only help for long-running network calls, which happens when calling another service or querying a database. When doing a long computation the CPU is not idle so there is no free compute power to use for something else. Finally, when doing IO calls in Python so GIL is usually released so the kernel can already schedule another thread while waiting for IO, so it is not sure that converting to async will yield improvement and should be benchmarked if you plan on converting an existing program.

pdonis 6y ago
> when doing IO calls in Python so GIL is usually released so the kernel can already schedule another thread while waiting for IO This is true, but scheduling another thread through the kernel can have higher overhead since it requires context switches. Running multiple threads also has other potential issues with lock contention; how problematic they are will depend on the use case. The potential advantage of scheduling another thread is, of course, that it can do CPU bound work; but in Python, unfortunately, doing that means the GIL doesn't get released so that thread will prevent any further network I/O while it's running, the same as would happen in an async framework if a worker did a lot of CPU work. So Python doesn't really let you realize the advantages of threads in this context.
- Znafon 6y ago
  > doing that means the GIL doesn't get released so that thread will prevent any further network I/O while it's running, the same as would happen in an async framework if a worker did a lot of CPU work. So Python doesn't really let you realize the advantages of threads in this context. I don't think that's true, the GIL get released for many computing intensive or IO bound tasks in Python, for example when reading from a socket the GIL gets released at https://github.com/python/cpython/blob/e822e37946f27c09953bb5733acf3b07c2db690f/Modules/socketmodule.c#L899 https://github.com/python/cpython/blob/e822e37946f27c09953bb...
  - pdonis 6y ago
    > the GIL get released for many computing intensive or IO bound tasks in Python I/O bound, yes, since that requires system calls, and system calls (reading from a socket is an example of a system call) release the GIL. Computing intensive, no. Code that is doing a CPU intensive computation but makes no system calls will never release the GIL.
    - jashmatthews 6y ago
      The GIL gets released whenever it gets released. C extensions like zlib release the GIL while (de)compressing requests/responses. https://docs.python.org/3/c-api/init.html#releasing-the-gil-from-extension-code https://docs.python.org/3/c-api/init.html#releasing-the-gil-...
      - pdonis 6y ago
        > C extensions like zlib release the GIL while (de)compressing requests/responses Yes, you're right, my statement was too broad.
    - Znafon 6y ago
      > Computing intensive, no. Code that is doing a CPU intensive computation but makes no system calls will never release the GIL. Any code that does not involve Python objects can release the GIL, no matter whether it makes system call or not. For example, NumPy the most popular scientific computation package in Python, on which many other popular packages like Pandas are based, releases the GIL when doing operation on matrix. This is documented at https://numpy.org/doc/stable/reference/internals.code-explanations.html#function-call https://numpy.org/doc/stable/reference/internals.code-explan...: > If NPY_ALLOW_THREADS is defined during compilation, then as long as no object arrays are involved, the Python Global Interpreter Lock (GIL) is released prior to calling the loops. It is re-acquired if necessary to handle error conditions. To do so it uses the same macro used by the socket module when doing system calls: https://github.com/numpy/numpy/blob/18a6e3e505ee416ddfc617f3e9afdff5a031c2c2/numpy/core/include/numpy/ndarraytypes.h#L959 https://github.com/numpy/numpy/blob/18a6e3e505ee416ddfc617f3...
- zzzeek 6y ago
  there is an advantage to threads in the CPU bound case which is that the work of other threads will not be blocked for a CPU-intense operation. With an IO-event based scheduler, your CPU bound task will not context switch leading to network logic elsewhere to simply time out. A particularly acute example is something like a network library logging into the MySQL database which gives the client a ten second window to respond to the initial security challenge. It was both an extremely difficult bug for me to diagnose as well as helpful for my role at work that I was able to track that one down in Openstack :).

ghostwriter 6y ago

Perhaps they meant that heavy long-running calculations could be offloaded to a worker pool with a help of concurrent futures and run_in_executor() - https://docs.python.org/3/library/concurrent.futures.html https://docs.python.org/3/library/concurrent.futures.html - https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor https://docs.python.org/3/library/asyncio-eventloop.html#asy...

pdonis 6y ago
This will only help if the workers are separate processes. Thread workers will hold the GIL in Python and prevent network I/O while they are doing CPU bound tasks.
- ghostwriter 6y ago
  sure, the pool only cares about concurrent.futures.Executor interface, an implementation could be processes or cloud resources.
- dr_zoidberg 6y ago
  > Thread workers will hold the GIL in Python and prevent network I/O while they are doing CPU bound tasks. Using cython: with nogil: # whatever you need to do, as long as it # doesn't touch a python object If you're doing heavy calculations from python you should at least be considering cython.

dec0dedab0de 6y ago

I thought the only reason to use ASGI is to use web sockets, and the only reason to use web sockets is to avoid making multiple requests for things that don't matter if a particular message is lost.