3 ms·
You've been able to release GIL for a long time though in the C ABI (and C++ via pybind) and that hasn't removed the need for nogil. The reason is that a singl
by alfalfasprout 3y ago
You've been able to release GIL for a long time though in the C ABI (and C++ via pybind) and that hasn't removed the need for nogil.
The reason is that a single thread can invoke multicore c/c++ code (even calling into specialized accelerators if needed) but having python objects that are shared between threads is extremely clunky. And multiprocessing results in a lot of communication overhead. Worse-- python code cannot proceed in another thread. Mixing python and C++ is very common in ML and scientific computing workloads.
> And I think trying to do both gil and nogil interpreters, rather than committing to one or the other, the python leadership will sign us up for untold inconveniences around packaging. We already see this in the move to async and it will only be worse with threading.
How has packaging been affected by async? As long as you have a compatible python version what issue do you run into?
- dekhn 3y agoboto's migration to async has caused no end of package incompatibility problems for me. So that's really more on the aiobotocore developers than python in that case. https://github.com/aio-libs/aiobotocore/issues/890 https://github.com/aio-libs/aiobotocore/issues/890 OK not a great example but also not worth debating about.