10 ms·

Python GIL removal question

85 points by spahl 15y ago

runT1ME 15y ago
>That is, if one processor writes to memory in a cache-line shared by another processor, they must stop whatever they are doing to synchronize the dirty cache lines with RAM. Thus, updating reference counts would flood the memory bus with traffic and be much worse than the GIL. I dont' understand. Isn't this going to happen if you have multiple threads running even if the GIL is blocking them from running? I'm not a hardware expert, but I'm not sure how constant locking would prevent cache synchronization just because they weren't truly running in parallel. I am fairly certain that constant synchronization(lock) because of the GIL would negatively impact cache performance, especially since well designed multithreaded applications avoid locking for as long as possible.
- meastham 15y ago
  I believe his argument is that it would reduce thrashing between the caches. With the GIL ownership of a cache line containing the reference count for any given object will only have to be transfered at most once per timeslice. If multiple threads were concurrently accessing a python object it would be ping-ponging back and forth between caches much more frequently. EDIT: Also, "stop whatever they are doing to synchronize the dirty cache lines with RAM," is not a very good way to describe what is going on, often times you don't have to hit RAM at all, the caches just synchronize between each other. It is still pretty bad for performance though.
  - runT1ME 15y ago
    >I believe his argument is that it would reduce thrashing between the caches. With the GIL ownership of a cache line containing the reference count for any given object will only have to be transfered at most once per timeslice. Ah. Makes sense. >just synchronize between each other Yes, but that's bad because that cache line is 'stuck' for all processors while the synchronization is occurring, if I'm not mistaken...
    - meastham 15y ago
      In general at least the two processors with the conflict will either have to block for a bit or switch to another hardware thread when write conflicts are occurring. There are lots of architecture tricks people pull to try to mitigate the impact but the reality of the matter is frequently mutating shared state (e.g. reference counts) makes it extremely difficult to have good performance with threads running in parallel.
- lambda_cube 15y ago
  > I dont' understand. Isn't this going to happen if you have multiple threads running even if the GIL is blocking them from running? I don't think it would. If there is a GIL (Global Interpreter Lock) only one thread of the process can be scheduled to run at any time. As the poster (Sturla) says, Python threads are native OS threads so they should be scheduled by the OS kernel (right?). A good scheduler would use affinity scheduling and schedule all threads of the Python program on the same processor/core every time to get benefits from cached data and code. I believe modern kernels (Linux, MacOS, Solaris, probably Windows as well) use this kind of affinity scheduling, so if we're lucky the Python process gets scheduled on the same processor every time and there will be no need any cache synchronization. > I'm not a hardware expert, but I'm not sure how constant locking would prevent cache synchronization just because they weren't truly running in parallel. I'm not sure if you misunderstood the mail. The constant locking would only be used if they were running in parallel. Anyway, if you have a GIL you don't need that kind of locking described in the mail. You only need to do explicit locking on shared data structures when you read or update the contents of those data structures. If you have reference counting, threads that run in parallel and no GIL you would have to lock even if you are just assigning a reference to such a data structure to a new variable. If you have a GIL you are certain that only one thread at a time are updating the reference count. That is indeed what the GIL is, one coarse lock for all data (and the interpreter) instead of fine grained locks for every data structure. (I don't know Python very well, I just answer from general knowledge of computer architecture and language implementation. But I've read about the Python GIL several times, since it's the most discussed GIL of any language.)
  - runT1ME 15y ago
    I don't think there's anything to prevent more than one thread from being scheduled at any time. They just block when trying to run concurrently because of the GIL. >I'm not sure if you misunderstood the mail. The constant locking would only be used if they were running in parallel. No, I'm saying that the GIL is constant locking. You still have two threads being run concurrently on (possibly) two separate cores accessing the same cache lines. They just cannot actually run in parallel. I have no idea how the GIL time slices between the two threads, so what i'm saying is completely possible. However, below my original post meastham correctly pointed out the GIL does prevent cache thrashing where updates to shared memory might go back and fourth multiple times unnecessarily. So it's not as bad as I was imagining.
kev009 15y ago
Sounds like typical denial a la Firefox memory usage or MySQL's early lacking that have to get eaten down the line. Not that the case isn't well argued, but to claim that GIL isn't a fundamental limitation and a bad thing is silly. A few years from now it will be like, "Oh, yeah.. that".
- deleted 15y ago
  [deleted]
- meastham 15y ago
  I don't think that he is arguing the the GIL isn't a limitation, just that the fundamental limitation it imposes can't be removed without also changing the threading model or the garbage collector. It's really impossible run threads in parallel with any sort of performance when they're all constantly generating a huge amount of cache coherency traffic by updating reference counts.
- jerf 15y ago
  "Nobody likes the GIL, ..." That's a quote. What more do you want? The question is not and never has been "Does the GIL have undesirable characteristics?" It has always been "can someone produce an implementation that is missing the GIL and actually better, while meeting all the needs CPython has?" So far, the answer is no, despite rather a lot of smart people trying. (Also note that many people have succeeded by dropping the second clause. Many non-CPython Python implementations don't have a GIL. But they aren't CPython, which in particular means that extensions written for CPython don't work in them, which is really the key thing that distinguishes CPython from just generic "Python".)
- deleted 15y ago
  [deleted]
  - irahul 15y ago
    GIL is a problem - that has been acknowledged every time it has been brought up. There has been past attempts to remove the GIL - that slowed down the interpreter and the patch wasn't merged. Removing GIL is massive work and will make the interpreter complex. Meanwhile, you have gevent, multiprocessing, c extensions... to work out the limitations.
thadeus_venture 15y ago
The most frustrating thing about Python is its community's complete denial about what a joke their concurrency situation is. Truth is python is not truly multi-threaded, and no, claiming that multi-process is the way to do parallel computation across the board is not a sane argument at all. It's religious zeal. My company is currently using it for web apps, and that's proving a pain (i.e. having to use proxy servers for database access to minimize connections across all the python process instances). Using python for anything more serious, like a message queuing system for example is even more prohibitive. People in charge should wake up and start taking serious steps about it. I guess PyPy is the biggest hope. Meanwhile in the JVM world..
- dman 15y ago
  The most frustrating thing about the ferrari community is that they are in complete denial about what a joke their affordable practical sedan story is.
  - thadeus_venture 15y ago
    Have you ever heard Guido speak about the issue? He and a number of others don't think it's one worth solving. Really. Yea it may be a lot of work to create a new GC implementation and change the threading model, but if you want the language to progress that's the way forward.
    - irahul 15y ago
      > Have you ever heard Guido speak about the issue? Yes. > He and a number of others don't think it's one worth solving. Really. Check your claims. http://www.artima.com/weblogs/viewpost.jsp?thread=214235 http://www.artima.com/weblogs/viewpost.jsp?thread=214235 http://docs.python.org/faq/library#can-t-we-get-rid-of-the-global-interpreter-lock http://docs.python.org/faq/library#can-t-we-get-rid-of-the-g...
    - rbanffy 15y ago
      More threads sharing the same data gives rise to all kinds of difficult to track synchronization bugs. The share-nothing multi-process approach Python encourages feels good enough for me.
aklein 15y ago
Can anyone shed light on where the GIL winds up hurting you? According to this paper [1]: "Thus, in all cases, the single global lock semantics seem fundamentally compatible with both lock-based and transactional memory implementations." [1] http://www.usenix.org/event/hotpar09/tech/full_papers/boehm/boehm_html/ http://www.usenix.org/event/hotpar09/tech/full_papers/boehm/...
br1 15y ago
Reference counts could be stored separately from objects and migrated to the thread that modified it. Or several reference counts for the same object could be used. Has this been tried?
pnathan 15y ago
I find it terribly annoying when the solution to a language problem is "Write an extension". It's a cop-out. Java, .NET, various Lisps, and I'm sure other systems that I don't know off the top of my head have solved the problem of true threading.
- wycats 15y ago
  Rubinius (in the current 2.0 betas) has removed the GIL as well. In fact, the only Ruby that still has a GIL is MRI. MacRuby, JRuby and Rubinius (as of 2.0) all have threading without a GIL.
  - pnathan 15y ago
    That's very exciting. Will MRI be fixed "soon" ?
    - mitchty 15y ago
      Just like CPython, its doubtful.
- aklein 15y ago
  "I want to point out one more time that the language doesn't require the GIL -- it's only the CPython virtual machine that has historically been unable to shed it." http://www.artima.com/weblogs/viewpost.jsp?thread=214235 http://www.artima.com/weblogs/viewpost.jsp?thread=214235
  - pnathan 15y ago
    I don't want to get into a software hipster flamewar about the definition of 'mainstream', but CPython is the Standard Python. When I click on "Windows Installer" on python.org, I get CPython.
    - aklein 15y ago
      Agreed, for now, CPython == Python. But PyPy aims to change this. http://morepypy.blogspot.com/2011/06/global-interpreter-lock-or-how-to-kill.html http://morepypy.blogspot.com/2011/06/global-interpreter-lock...
ulrich 15y ago
If you leave aside C-extensions and libraries, CPython is a pretty bad language for number crunching. This is just not the use-case it tries to solve. I am glad they favor simplicity over execution speed. Removing the GIL might be useful for a faster implementation with JIT compiler though.
- dman 15y ago
  CPython has numpy which rocks for number crunching.
  - rbanffy 15y ago
    And has no GIL problems.
    - cdavid 15y ago
      This is not exactly true. We can alleviate the GIL issue by releasing it inside C extensions, true, but it is still there nonetheless. Incidentally, one of the big advantage of the GIL is to make C extensions easier to write.
      - rbanffy 15y ago
        You can then call it The GIL Advantage ;-) As long as you are not creating or destroying the Python objects you expect to give back to the Python side of your program, you don't have to care much about it.
- dagw 15y ago
  CPython is a pretty bad language for number crunching Perhaps, but thanks to numpy, scipy and a host of other amazing libraries it still ties with matlab as the go to language for number crunching among everyone I know who crunches numbers for a living.
yason 15y ago
If the GIL bites you, it's most likely a warning that your program is badly written, independent of the GIL issue. Ah, that old line again. Translation: "We really don't like to even think about changing this crappy design that we started with in the first place, because we can just explain ourselves out of it by coming up with suitable language goals that don't actually require concurrent access to the interpreter. Not accessing the interpreter concurrently is one of our language goals because you can do everything else. So, if you think you still need to get rid of GIL then you're just a bad programmer and your programs are badly written because hey, we just defined the universe you're playing in."
- MostAwesomeDude 15y ago
  Please explain how you would sidestep the GIL, please. We've been waiting for a good solution for over a decade now. :3

85 points by spahl 15y ago

>That is, if one processor writes to memory in a cache-line shared by another processor, they must stop whatever they are doing to synchronize the dirty cache lines with RAM. Thus, updating reference counts would flood the memory bus with traffic and be much worse than the GIL. I dont' understand. Isn't this going to happen if you have multiple threads running even if the GIL is blocking them from running? I'm not a hardware expert, but I'm not sure how constant locking would prevent cache synchronization just because they weren't truly running in parallel. I am fairly certain that constant synchronization(lock) because of the GIL would negatively impact cache performance, especially since well designed multithreaded applications avoid locking for as long as possible.

meastham 15y ago
I believe his argument is that it would reduce thrashing between the caches. With the GIL ownership of a cache line containing the reference count for any given object will only have to be transfered at most once per timeslice. If multiple threads were concurrently accessing a python object it would be ping-ponging back and forth between caches much more frequently. EDIT: Also, "stop whatever they are doing to synchronize the dirty cache lines with RAM," is not a very good way to describe what is going on, often times you don't have to hit RAM at all, the caches just synchronize between each other. It is still pretty bad for performance though.
- runT1ME 15y ago
  >I believe his argument is that it would reduce thrashing between the caches. With the GIL ownership of a cache line containing the reference count for any given object will only have to be transfered at most once per timeslice. Ah. Makes sense. >just synchronize between each other Yes, but that's bad because that cache line is 'stuck' for all processors while the synchronization is occurring, if I'm not mistaken...
  - meastham 15y ago
    In general at least the two processors with the conflict will either have to block for a bit or switch to another hardware thread when write conflicts are occurring. There are lots of architecture tricks people pull to try to mitigate the impact but the reality of the matter is frequently mutating shared state (e.g. reference counts) makes it extremely difficult to have good performance with threads running in parallel.
lambda_cube 15y ago
> I dont' understand. Isn't this going to happen if you have multiple threads running even if the GIL is blocking them from running? I don't think it would. If there is a GIL (Global Interpreter Lock) only one thread of the process can be scheduled to run at any time. As the poster (Sturla) says, Python threads are native OS threads so they should be scheduled by the OS kernel (right?). A good scheduler would use affinity scheduling and schedule all threads of the Python program on the same processor/core every time to get benefits from cached data and code. I believe modern kernels (Linux, MacOS, Solaris, probably Windows as well) use this kind of affinity scheduling, so if we're lucky the Python process gets scheduled on the same processor every time and there will be no need any cache synchronization. > I'm not a hardware expert, but I'm not sure how constant locking would prevent cache synchronization just because they weren't truly running in parallel. I'm not sure if you misunderstood the mail. The constant locking would only be used if they were running in parallel. Anyway, if you have a GIL you don't need that kind of locking described in the mail. You only need to do explicit locking on shared data structures when you read or update the contents of those data structures. If you have reference counting, threads that run in parallel and no GIL you would have to lock even if you are just assigning a reference to such a data structure to a new variable. If you have a GIL you are certain that only one thread at a time are updating the reference count. That is indeed what the GIL is, one coarse lock for all data (and the interpreter) instead of fine grained locks for every data structure. (I don't know Python very well, I just answer from general knowledge of computer architecture and language implementation. But I've read about the Python GIL several times, since it's the most discussed GIL of any language.)
- runT1ME 15y ago
  I don't think there's anything to prevent more than one thread from being scheduled at any time. They just block when trying to run concurrently because of the GIL. >I'm not sure if you misunderstood the mail. The constant locking would only be used if they were running in parallel. No, I'm saying that the GIL is constant locking. You still have two threads being run concurrently on (possibly) two separate cores accessing the same cache lines. They just cannot actually run in parallel. I have no idea how the GIL time slices between the two threads, so what i'm saying is completely possible. However, below my original post meastham correctly pointed out the GIL does prevent cache thrashing where updates to shared memory might go back and fourth multiple times unnecessarily. So it's not as bad as I was imagining.

kev009 15y ago

Sounds like typical denial a la Firefox memory usage or MySQL's early lacking that have to get eaten down the line. Not that the case isn't well argued, but to claim that GIL isn't a fundamental limitation and a bad thing is silly. A few years from now it will be like, "Oh, yeah.. that".

deleted 15y ago
[deleted]
meastham 15y ago
I don't think that he is arguing the the GIL isn't a limitation, just that the fundamental limitation it imposes can't be removed without also changing the threading model or the garbage collector. It's really impossible run threads in parallel with any sort of performance when they're all constantly generating a huge amount of cache coherency traffic by updating reference counts.
jerf 15y ago
"Nobody likes the GIL, ..." That's a quote. What more do you want? The question is not and never has been "Does the GIL have undesirable characteristics?" It has always been "can someone produce an implementation that is missing the GIL and actually better, while meeting all the needs CPython has?" So far, the answer is no, despite rather a lot of smart people trying. (Also note that many people have succeeded by dropping the second clause. Many non-CPython Python implementations don't have a GIL. But they aren't CPython, which in particular means that extensions written for CPython don't work in them, which is really the key thing that distinguishes CPython from just generic "Python".)
deleted 15y ago
[deleted]
- irahul 15y ago
  GIL is a problem - that has been acknowledged every time it has been brought up. There has been past attempts to remove the GIL - that slowed down the interpreter and the patch wasn't merged. Removing GIL is massive work and will make the interpreter complex. Meanwhile, you have gevent, multiprocessing, c extensions... to work out the limitations.

thadeus_venture 15y ago

The most frustrating thing about Python is its community's complete denial about what a joke their concurrency situation is. Truth is python is not truly multi-threaded, and no, claiming that multi-process is the way to do parallel computation across the board is not a sane argument at all. It's religious zeal. My company is currently using it for web apps, and that's proving a pain (i.e. having to use proxy servers for database access to minimize connections across all the python process instances). Using python for anything more serious, like a message queuing system for example is even more prohibitive. People in charge should wake up and start taking serious steps about it. I guess PyPy is the biggest hope. Meanwhile in the JVM world..

dman 15y ago
The most frustrating thing about the ferrari community is that they are in complete denial about what a joke their affordable practical sedan story is.
- thadeus_venture 15y ago
  Have you ever heard Guido speak about the issue? He and a number of others don't think it's one worth solving. Really. Yea it may be a lot of work to create a new GC implementation and change the threading model, but if you want the language to progress that's the way forward.
  - irahul 15y ago
    > Have you ever heard Guido speak about the issue? Yes. > He and a number of others don't think it's one worth solving. Really. Check your claims. http://www.artima.com/weblogs/viewpost.jsp?thread=214235 http://www.artima.com/weblogs/viewpost.jsp?thread=214235 http://docs.python.org/faq/library#can-t-we-get-rid-of-the-global-interpreter-lock http://docs.python.org/faq/library#can-t-we-get-rid-of-the-g...
  - rbanffy 15y ago
    More threads sharing the same data gives rise to all kinds of difficult to track synchronization bugs. The share-nothing multi-process approach Python encourages feels good enough for me.

aklein 15y ago

Can anyone shed light on where the GIL winds up hurting you? According to this paper [1]: "Thus, in all cases, the single global lock semantics seem fundamentally compatible with both lock-based and transactional memory implementations." [1] http://www.usenix.org/event/hotpar09/tech/full_papers/boehm/boehm_html/ http://www.usenix.org/event/hotpar09/tech/full_papers/boehm/...

br1 15y ago

Reference counts could be stored separately from objects and migrated to the thread that modified it. Or several reference counts for the same object could be used. Has this been tried?

pnathan 15y ago

I find it terribly annoying when the solution to a language problem is "Write an extension". It's a cop-out. Java, .NET, various Lisps, and I'm sure other systems that I don't know off the top of my head have solved the problem of true threading.

wycats 15y ago
Rubinius (in the current 2.0 betas) has removed the GIL as well. In fact, the only Ruby that still has a GIL is MRI. MacRuby, JRuby and Rubinius (as of 2.0) all have threading without a GIL.
- pnathan 15y ago
  That's very exciting. Will MRI be fixed "soon" ?
  - mitchty 15y ago
    Just like CPython, its doubtful.
aklein 15y ago
"I want to point out one more time that the language doesn't require the GIL -- it's only the CPython virtual machine that has historically been unable to shed it." http://www.artima.com/weblogs/viewpost.jsp?thread=214235 http://www.artima.com/weblogs/viewpost.jsp?thread=214235
- pnathan 15y ago
  I don't want to get into a software hipster flamewar about the definition of 'mainstream', but CPython is the Standard Python. When I click on "Windows Installer" on python.org, I get CPython.
  - aklein 15y ago
    Agreed, for now, CPython == Python. But PyPy aims to change this. http://morepypy.blogspot.com/2011/06/global-interpreter-lock-or-how-to-kill.html http://morepypy.blogspot.com/2011/06/global-interpreter-lock...

ulrich 15y ago

If you leave aside C-extensions and libraries, CPython is a pretty bad language for number crunching. This is just not the use-case it tries to solve. I am glad they favor simplicity over execution speed. Removing the GIL might be useful for a faster implementation with JIT compiler though.

dman 15y ago
CPython has numpy which rocks for number crunching.
- rbanffy 15y ago
  And has no GIL problems.
  - cdavid 15y ago
    This is not exactly true. We can alleviate the GIL issue by releasing it inside C extensions, true, but it is still there nonetheless. Incidentally, one of the big advantage of the GIL is to make C extensions easier to write.
    - rbanffy 15y ago
      You can then call it The GIL Advantage ;-) As long as you are not creating or destroying the Python objects you expect to give back to the Python side of your program, you don't have to care much about it.
dagw 15y ago
CPython is a pretty bad language for number crunching Perhaps, but thanks to numpy, scipy and a host of other amazing libraries it still ties with matlab as the go to language for number crunching among everyone I know who crunches numbers for a living.

yason 15y ago

If the GIL bites you, it's most likely a warning that your program is badly written, independent of the GIL issue. Ah, that old line again. Translation: "We really don't like to even think about changing this crappy design that we started with in the first place, because we can just explain ourselves out of it by coming up with suitable language goals that don't actually require concurrent access to the interpreter. Not accessing the interpreter concurrently is one of our language goals because you can do everything else. So, if you think you still need to get rid of GIL then you're just a bad programmer and your programs are badly written because hey, we just defined the universe you're playing in."

MostAwesomeDude 15y ago
Please explain how you would sidestep the GIL, please. We've been waiting for a good solution for over a decade now. :3