3 ms·
If you're hitting the GIL then yeah, that could be a big win. Though I suspect that the amount people actually hitting the GIL is significantly lower than the a
by dec0dedab0de 5y ago
If you're hitting the GIL then yeah, that could be a big win. Though I suspect that the amount people actually hitting the GIL is significantly lower than the amount of people talking about it.
- woodruffw 5y agoIt's been a while since I've written a Python extension, but my understanding is that every single use of `PY_INCREF` or `PY_DECREF` (or any transitive use) hits the GIL, since those reference counts need to be locked. Almost every Python extension that I've ever read is littered with those calls (macros?), which I'd expect adds up to a decent amount of contention.
- dec0dedab0de 5y agoYou're right, but it's only an issue when the application programmer is using threads. So basically for anyone that is doing CPU intensive work, that can be broken up into chunks, and needs to share memory, and doesn't want to take the time to write their own extension to do that part. Practically speaking, I/O is usually the bigger bottleneck. Or you can change your algorithm to not share memory, then use multiprocessing, or a task scheduler like celery to spread it over multiple machines. There are a ton of people who hear about someone running into GIL issues and they think it is affecting them, when they're not doing anything that would be helped by removing it. A JIT compiler on the other hand, would speed up every Python program.
- klyrs 5y agoI only talk about the GIL when I try and write some parallel code in Python and get a -10% performance benefit. As a corollary, I only use profanity when speaking of the GIL. But I don't do that often, because I usually skip straight to C++ extensions for code that wants parallelism.