4 ms·
> Most C extensions and embeddings will require major structural changes or even rewrites. Sam claims otherwise: "Most C API extensions don’t require any chang
by js2 3y ago
> Most C extensions and embeddings will require major structural changes or even rewrites.
Sam claims otherwise: "Most C API extensions don’t require any changes, and for those that do require changes, the changes are small. For example, for “nogil” Python I’m providing binary wheels for ~35 extensions that are slow or difficult to build from source and only about seven projects required code changes (PyTorch, pybind11, Cython, numpy, scikit-learn, viztracer, pyo3). For four of those projects, the code changes have already been contributed and adopted upstream. For comparison, many of those same projects also frequently require changes to support minor releases of CPython."
https://discuss.python.org/t/pep-703-making-the-global-interpreter-lock-optional/22606/44 https://discuss.python.org/t/pep-703-making-the-global-inter...
In the following talk, he shows the scikit-learn changes that were needed:
https://youtu.be/9OOJcTp8dqE?t=1958 https://youtu.be/9OOJcTp8dqE?t=1958
I'd also like to point out that the GIL does not inherently make C extensions thread safe. It only protects the interpreter itself. For example, calling in to the C API can already release the GIL as I explained in reply to /u/bjourne here:
https://news.ycombinator.com/item?id=36649769 https://news.ycombinator.com/item?id=36649769