3 ms·
You may be interested in the concurrent.futures library, available for over a decade now. It keeps you from shooting yourself in the foot like that. https://do
by empthought 3y ago
You may be interested in the concurrent.futures library, available for over a decade now. It keeps you from shooting yourself in the foot like that.
https://docs.python.org/3/library/concurrent.futures.html https://docs.python.org/3/library/concurrent.futures.html
- KolenCh 3y agoWhy do you think it would help? It provides a nice interface but is using multiprocessing or multi threading under the hood depending on which executioner you use: > The ProcessPoolExecutor class is an Executor subclass that uses a pool of processes to execute calls asynchronously. ProcessPoolExecutor uses the multiprocessing module, which allows it to side-step the Global Interpreter Lock but also means that only picklable objects can be executed and returned.
- empthought 3y agoYour trouble seems to involve not understanding how to set up signal handlers, which ProcessPoolExecutor handles for you and exposes via a BrokenProcessPool exception.
- wiseowise 3y ago> Derived from BrokenExecutor (formerly RuntimeError), this exception class is raised when one of the workers of a ProcessPoolExecutor has terminated in a non-clean fashion (for example, if it was killed from the outside). What if it hangs?
- empthought 3y agoThat isn’t the scenario originally described, but there is a timeout parameter in future.result().