3 ms·
> Here's what that looks like in Python. Although python multithreading outside of IO tasks or dropping into C code that drops the GIL isn't going to paralleli
by bpicolo 11y ago
> Here's what that looks like in Python.
Although python multithreading outside of IO tasks or dropping into C code that drops the GIL isn't going to parallelize, the latter of which is the only way you're going to burn up cores. The API does make it seem simple though.
- delroth 11y agoThe example is using multiprocessing.pool.ThreadPool which does not have the GIL problem you mention (at the cost of making it harder to share data).
- someguy1234 11y agoThreadPool would lock the GIL, wouldn't it? I think multiprocessing.Pool uses processes and that's where you can avoid the GIL (each process gets its own) at the cost of making data sharing harder. Even with the GIL, ThreadPools are still fine performance-wise for tasks that release the GIL such as I/O or a number of functions in numpy and scipy.
- aftbit 11y agoYou're thinking multiprocessing.Pool. I think ThreadPool actually uses threads and does hit the GIL.