4 ms·

Those those who wonder how it can make it faster since it's using threads and Python has the GIL, numpy releases the gil while doing computation, so thread can

• kzrdude 5y ago
  For this code we can also easily try the process executor instead of the thread executor - then we avoid the GIL. In this case, it has an approximate 10% performance penalty, probably since we have to serialize data back and forth, can't share memory.
  • BiteCode_dev 5y ago
    You can since 3.8 :) https://docs.python.org/3/library/multiprocessing.shared_memory.html https://docs.python.org/3/library/multiprocessing.shared_mem... If you use a data structure that supports the buffer protocol (such as a numpy array), you won't even pay a serialization cost. Python has made a LOT of progress, concurrency wise. Between asyncio, process pools, threads releasing the GIL and shared memory, you can get much perfs with the stdlib alone. Then of course you have Dask if you need more later.