3 ms·
This seems like a good place to spruik something I made, a Python package for profiling how much the GIL is held: https://github.com/chrisjbillington/gil_load
by cjbillington 9y ago
This seems like a good place to spruik something I made, a Python package for profiling how much the GIL is held:
https://github.com/chrisjbillington/gil_load https://github.com/chrisjbillington/gil_load
In my experience, the GIL is not held for nearly as high a proportion of the time as people think it is, because properly written C extensions and blocking io always releases the GIL. So long as the proportion of time the GIL is held is not approaching 100%, then you can still get gains from threading. This is almost always the case in numerically heavy code that uses numpy or scipy, since the extensions release the GIL. Threads work almost just as well at speeding up this code as in any GIL-free interpreter.
And usually long before you consider multithreaded code, you'll want to move the bottlenecks of your code over into Cython or something, since that can give speedup factors much larger than multithreading. In which case all you need is a "with nogil:" around the the meaty bit of you Cython code, and then it too will be able to get speedups from multithreading.