3 ms·
About the fact this talk is from 2015: The GIL is released by many libraries that aren't about data science, so was and is relevant to many other types of appli
by quietbritishjim 6y ago
About the fact this talk is from 2015: The GIL is released by many libraries that aren't about data science, so was and is relevant to many other types of application. In the Python standard library, for example, zipfile (which can operate on files or in-memory buffers) and hashlib come to mind. As I mentioned in another comment, hashlib calls have released the GIL at least as far back as 2009, well before this talk, and releasing the GIL was possible all the way back in 1998. Many (most?) Python libraries are implemented in C, and almost all of those that are will release the GIL.
About appreciating both points of view: I'm sorry if I'm making it sound like I think Python developers should always use threads and never multiprocessing. I don't believe that. And I don't believe the speaker should have reformulated their whole talk to use threads instead of processes. Just that it's such a pervasive myth that the GIL prevents all thread-based parallelism that they should have been careful to avoid reinforcing that. The speaker showed an example where the GIL actually did prevent parallelism; all I wanted was for them to add verbally that, by the way, if you were calling almost any CPU-bound library function rather than one you wrote yourself then this wouldn't be a problem.