3 ms·
gil or no gil is mostly irrelevant for numpy, an extension that does most work with gil released.
by guipsp 4y ago
gil or no gil is mostly irrelevant for numpy, an extension that does most work with gil released.
- cdavid 4y agoRelying on the cpython gil and releasing the gil are two different thing: 1. releasing the gil means multithreading is opt-in for a given code section in NumPy. Only very specific parts of the code need to be threadsafe. 2. not relying on a gil in cpython runtime means multithreading becomes opt out. Now all the code by default needs to be threadsafe, including the libs you depend on. A lot of C/C++/Fortran scientific code is not thread safe, and the whole scientific python ecosystem depends heavily on those codebases.
- guipsp 4y agoI understand what you are saying, but I don't see how it's relevant? Numpy doesn't rely on the gil, which is why it releases it.
- cdavid 4y agoIt releases the GIL only in very specific sections. Most of the Numpy C code runs under the gil. A quick check on master shows only 10-20 calls using NPY_BEGIN_ALLOW_THREADS (which is an alias to Py_BEGIN_ALLOW_THREADS). A lot of the NumPy code manipulates python runtime objects, and doing so without thread safety would likely break everywhere. A lot of efforts would be needed to gradually make large C extension thread safe.