4 ms·
Relying on the cpython gil and releasing the gil are two different thing: 1. releasing the gil means multithreading is opt-in for a given code section in Num
by cdavid 4y ago
Relying on the cpython gil and releasing the gil are two different thing:
1. releasing the gil means multithreading is opt-in for a given code section in NumPy. Only very specific parts of the code need to be threadsafe.
2. not relying on a gil in cpython runtime means multithreading becomes opt out. Now all the code by default needs to be threadsafe, including the libs you depend on.
A lot of C/C++/Fortran scientific code is not thread safe, and the whole scientific python ecosystem depends heavily on those codebases.
- guipsp 4y agoI understand what you are saying, but I don't see how it's relevant? Numpy doesn't rely on the gil, which is why it releases it.
- cdavid 4y agoIt releases the GIL only in very specific sections. Most of the Numpy C code runs under the gil. A quick check on master shows only 10-20 calls using NPY_BEGIN_ALLOW_THREADS (which is an alias to Py_BEGIN_ALLOW_THREADS). A lot of the NumPy code manipulates python runtime objects, and doing so without thread safety would likely break everywhere. A lot of efforts would be needed to gradually make large C extension thread safe.