3 ms·
Perhaps they meant that heavy long-running calculations could be offloaded to a worker pool with a help of concurrent futures and run_in_executor() - https://d
by ghostwriter 6y ago
Perhaps they meant that heavy long-running calculations could be offloaded to a worker pool with a help of concurrent futures and run_in_executor()
- https://docs.python.org/3/library/concurrent.futures.html https://docs.python.org/3/library/concurrent.futures.html
- https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor https://docs.python.org/3/library/asyncio-eventloop.html#asy...
- pdonis 6y agoThis will only help if the workers are separate processes. Thread workers will hold the GIL in Python and prevent network I/O while they are doing CPU bound tasks.
- ghostwriter 6y agosure, the pool only cares about concurrent.futures.Executor interface, an implementation could be processes or cloud resources.
- dr_zoidberg 6y ago> Thread workers will hold the GIL in Python and prevent network I/O while they are doing CPU bound tasks. Using cython: with nogil: # whatever you need to do, as long as it # doesn't touch a python object If you're doing heavy calculations from python you should at least be considering cython.