4 ms·

So, for me, replacing imap with pool.imap is the easy part. The hard part is dealing with things like handling exceptions, catching keyboard interrupts, and so

• hyperion2010 10y ago
  I have a hack for dealing with KeyboardInterrupt's on a ProcessPoolExecutor: https://github.com/tgbugs/desc/blob/master/util/process_fixed.py https://github.com/tgbugs/desc/blob/master/util/process_fixe.... I used this in concert with asyncio run_in_executor which helps with some of the exception handling.
  • lqdc13 10y ago
    That's a nice hack when you have only a few elements in the iterable, but Futures are pretty heavyweight, so this is unfortunately a lot slower than multiprocessing.Pool when you have many elements. The other thing is that this won't work on Windows. Efficient multiprocessing always a pain in Python in my experience.