4 ms·
I saw this article, enjoyed it, and then said to myself "Now let me go read the inevitable well-meaning but misguided comments about the GIL." And sure enough,
by superbatfish 11y ago
I saw this article, enjoyed it, and then said to myself "Now let me go read the inevitable well-meaning but misguided comments about the GIL." And sure enough, the very top comment is a well-meaning but misguided comment about the GIL.
The author of this article is talking about CPU-intensive tasks that involve C-extensions. Such extensions typically release the GIL during their execution, which means that you can effectively parallelize such workloads in a ThreadPool. (He explicitly mentions this about Numpy/Scipy, but most other CPU-intensive extension libraries behave the same way.) Heavy users of Numpy use Thread-level parallelism very frequently.
- hyperion2010 11y agoYep. Thought I was missing something since he was taking about cpu bound tasks and threads, if you are using c extensions then it is wasteful to spin up additional interpreters and you can also get fun bugs like numpy reusing the same random seed in child processes!