3 ms·

Asyncio is not ment for CPU Bound Task but for IO Bound Tasks. You should have used multiprocessing. The problem you describe is exactly what asyncio is used fo

Asyncio is not ment for CPU Bound Task but for IO Bound Tasks. You should have used multiprocessing. The problem you describe is exactly what asyncio is used for: saturate your bandwidth better. maybe the right aproach would have been a Threadpool? Plus you don't have to refractor the task. Just let it run sync, but you can make it also async

JosephRedfern 4y ago
Depending on the specifics, a threadpool proabably wouldn't help with a CPU bound task due to GIL limitations. `multiprocess.Pool` might (though process pools aren' without their overheads)
matsemann 4y ago
In other languages the async paradigm work for multiple kind of workflows, not just heavily IO bound ones.
- number6 4y ago
  Can you give me an example which languages do this?
  - matsemann 4y ago
    In Kotlin for instance you can execute your coroutines with different "dispatchers", which have different behaviors. You can for instance use a dispatcher with multiple threads in a threadpool, run it in the main thread, a specific thread, in threads with low/high priority etc. Basically allowing you to write code using the async/coroutine paradigm, but then control its execution from the outside. So I've also used coroutines in Kotlin for CPU bound tasks with a nice speedup, where multiple sub-tasks get executed in parallel and then gathered when needed to produce new tasks etc., in a granular way that would be very intrusive doing with threadpools. Here I basically took the existing coroutine code and slapped some more threads on it. With that said, I'm not entirely sold on Kotlin's way either. Both Kotlin and Python have the "what color is your function" problem.
    - tmh88j 4y ago
      > In Kotlin for instance you can execute your coroutines with different "dispatchers", which have different behaviors. You can for instance use a dispatcher with multiple threads in a threadpool, run it in the main thread, a specific thread, in threads with low/high priority etc. Basically allowing you to write code using the async/coroutine paradigm, but then control its execution from the outside. I'm not very experienced with Kotlin, but that sounds similar to python's run_in_executor [1] [1] https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor https://docs.python.org/3/library/asyncio-eventloop.html#asy...
  - throwaway81523 4y ago
    Look at seastar.io which is an async concurrency library for C++. It is painful in that it resembles node.js but it is very performant. I'd be interested in seeing a version adapted to use C++20 coroutines.
- qwertox 4y ago
  But it only does so because you can move out that extra work from the async thread into a thread pool. In Python there's no benefit in doing this due to the GIL, unless you're using a module which implements its own multithreading (for example in C). Python is not alone with this issue. In other languages you're basically stepping out of the async paradigm in order to use threading in parallel. You can wait for result of the thread in the async loop without blocking. I really enjoy using asyncio in Python for things where I have to do a lot of stuff in parallel, like executing remote scripts in a dozen of servers in parallel via AsyncSSH or for low workload servers which query databases. In any case, what keeps me hooked on Python like a junkie is the `reload(module)` which I invoke via an inotify hook every time a file/module changes: server.py server_handler.py reloader.py server.py loads reloader.py (which sets up inotify) and that one takes care of reloading server_handler.py whenever it changes. server.py defers all the request/response handling to server_handler.py which can be edited on the fly so that the next request executes the new code. Instant hot reloading. This also works very nice with asyncio (like aiohttp) and one ends up with very readable code. If you need performance, then use Java, Rust, Go or C/C++, but for prototyping and tooling I absolutely love this approach with Python.
dekhn 4y ago
any concurrency system that is properly designed supports both types of tasks.
- number6 4y ago
  I only know C#: https://learn.microsoft.com/en-us/dotnet/csharp/async#recognize-cpu-bound-and-io-bound-work https://learn.microsoft.com/en-us/dotnet/csharp/async#recogn... There you also have to act differently based the task
  - dekhn 4y ago
    The docs on that page explicitly say you just have to provide a different parameter (basically, to force it to use a thread-like instead of select-like approach). The docs on that page also point you to a different task library which does what I'd expect: "If the work is appropriate for concurrency and parallelism, also consider using the Task Parallel Library." I checked those docs and that is a framework that makes sense.
    - phs2501 4y ago
      Not sure how that's different than Python asyncio's "run blocking stuff with `run_in_executor`, the result (future) of which integrates into the async event loop?
      - dekhn 4y ago
        it's threaded with the option to use 1:M, N:N or N:M. It's a mental model that includes both asyncio and threads, while async will always only be for IO-blocking calls that can be multiplexed.
jerf 4y ago
"Asyncio is not ment for CPU Bound Task but for IO Bound Tasks." You can amplify this; pure Python is not meant for CPU-bound tasks. If you have a pure Python task, you've CPU optimized it somewhat, and it's still too slow, the answer is, get out of pure Python. There's half-a-dozen options that still leave you essentially in Python land (just not pure Python anymore) like cython or NumPy, and of course dozens of other languages that leave Python entirely. To accelerate a pure Python task to the speed that you can get out of a single thread of a more efficient language, you have to perfectly parallelize your task across upwards of 40 CPUs (conservatively!), because that's how slow pure Python is. asyncio isn't a solution to this, but neither is any sort of multiprocessing of the pure Python either. This is not criticism. This is just engineering reality. Pure Python is a fun language, but a very slow one.

JosephRedfern 4y ago

Depending on the specifics, a threadpool proabably wouldn't help with a CPU bound task due to GIL limitations. `multiprocess.Pool` might (though process pools aren' without their overheads)

matsemann 4y ago

In other languages the async paradigm work for multiple kind of workflows, not just heavily IO bound ones.

number6 4y ago
Can you give me an example which languages do this?
- matsemann 4y ago
  In Kotlin for instance you can execute your coroutines with different "dispatchers", which have different behaviors. You can for instance use a dispatcher with multiple threads in a threadpool, run it in the main thread, a specific thread, in threads with low/high priority etc. Basically allowing you to write code using the async/coroutine paradigm, but then control its execution from the outside. So I've also used coroutines in Kotlin for CPU bound tasks with a nice speedup, where multiple sub-tasks get executed in parallel and then gathered when needed to produce new tasks etc., in a granular way that would be very intrusive doing with threadpools. Here I basically took the existing coroutine code and slapped some more threads on it. With that said, I'm not entirely sold on Kotlin's way either. Both Kotlin and Python have the "what color is your function" problem.
  - tmh88j 4y ago
    > In Kotlin for instance you can execute your coroutines with different "dispatchers", which have different behaviors. You can for instance use a dispatcher with multiple threads in a threadpool, run it in the main thread, a specific thread, in threads with low/high priority etc. Basically allowing you to write code using the async/coroutine paradigm, but then control its execution from the outside. I'm not very experienced with Kotlin, but that sounds similar to python's run_in_executor [1] [1] https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor https://docs.python.org/3/library/asyncio-eventloop.html#asy...
- throwaway81523 4y ago
  Look at seastar.io which is an async concurrency library for C++. It is painful in that it resembles node.js but it is very performant. I'd be interested in seeing a version adapted to use C++20 coroutines.
qwertox 4y ago
But it only does so because you can move out that extra work from the async thread into a thread pool. In Python there's no benefit in doing this due to the GIL, unless you're using a module which implements its own multithreading (for example in C). Python is not alone with this issue. In other languages you're basically stepping out of the async paradigm in order to use threading in parallel. You can wait for result of the thread in the async loop without blocking. I really enjoy using asyncio in Python for things where I have to do a lot of stuff in parallel, like executing remote scripts in a dozen of servers in parallel via AsyncSSH or for low workload servers which query databases. In any case, what keeps me hooked on Python like a junkie is the `reload(module)` which I invoke via an inotify hook every time a file/module changes: server.py server_handler.py reloader.py server.py loads reloader.py (which sets up inotify) and that one takes care of reloading server_handler.py whenever it changes. server.py defers all the request/response handling to server_handler.py which can be edited on the fly so that the next request executes the new code. Instant hot reloading. This also works very nice with asyncio (like aiohttp) and one ends up with very readable code. If you need performance, then use Java, Rust, Go or C/C++, but for prototyping and tooling I absolutely love this approach with Python.

dekhn 4y ago

any concurrency system that is properly designed supports both types of tasks.

number6 4y ago
I only know C#: https://learn.microsoft.com/en-us/dotnet/csharp/async#recognize-cpu-bound-and-io-bound-work https://learn.microsoft.com/en-us/dotnet/csharp/async#recogn... There you also have to act differently based the task
- dekhn 4y ago
  The docs on that page explicitly say you just have to provide a different parameter (basically, to force it to use a thread-like instead of select-like approach). The docs on that page also point you to a different task library which does what I'd expect: "If the work is appropriate for concurrency and parallelism, also consider using the Task Parallel Library." I checked those docs and that is a framework that makes sense.
  - phs2501 4y ago
    Not sure how that's different than Python asyncio's "run blocking stuff with `run_in_executor`, the result (future) of which integrates into the async event loop?
    - dekhn 4y ago
      it's threaded with the option to use 1:M, N:N or N:M. It's a mental model that includes both asyncio and threads, while async will always only be for IO-blocking calls that can be multiplexed.

jerf 4y ago

"Asyncio is not ment for CPU Bound Task but for IO Bound Tasks." You can amplify this; pure Python is not meant for CPU-bound tasks. If you have a pure Python task, you've CPU optimized it somewhat, and it's still too slow, the answer is, get out of pure Python. There's half-a-dozen options that still leave you essentially in Python land (just not pure Python anymore) like cython or NumPy, and of course dozens of other languages that leave Python entirely. To accelerate a pure Python task to the speed that you can get out of a single thread of a more efficient language, you have to perfectly parallelize your task across upwards of 40 CPUs (conservatively!), because that's how slow pure Python is. asyncio isn't a solution to this, but neither is any sort of multiprocessing of the pure Python either. This is not criticism. This is just engineering reality. Pure Python is a fun language, but a very slow one.