5 ms·

Is there anything inherent to the language that makes this global lock thing difficult to be without in Python, but not in other languages?

dalke 10y ago
Inherent? No. Neither Jython nor Iron Python have a GIL.
jerf 10y ago
It was ground in to CPython at an early date and every Python C extension ends up with it ground in, too, because it's in the C API. Some things are very difficult to retrofit on to an existing system. [1] If you create a new Python implementation from scratch it doesn't need to be there. It is not intrinsic to Python; it is intrinsic to the specific CPython implementation. It is also, broadly speaking, easy to remove the GIL. However the bar the CPython developers have set is that they will not accept a GIL-removal patch that harms single-threaded performance. This is the bar that has proved difficult to hurdle. It is also fair to point out that while it's a perennial topic in the Python community, it is not the only scripting language implementation with a GIL or moral equivalent in it. [1]: For similar reasons, none of the core implementations of the 1990s-era dynamic scripting languages have a very good true multi-threading story. (Some of the alternates can do it, like Jython.) They don't all even have implementations, and last I knew, none of them have anything that you ought to use in production. I don't think this is a fundamental limitation of any of the languages in question, it's just that it's really hard to retrofit threading onto a non-threaded code base after ~ten very heavy years of development. There's a set of other things that are hard to retrofit in to an existing code base if they don't start there from day one, like "unit testability" or "proper string handling".
- eloff 10y ago
  This is not true - it is intrinsic to Python - not just CPython. Part of it is the garbage collector, reference counting is expensive to do without the GIL - but that can be changed. IronPython, Jython, some translations of PyPy all attest to that. The other part is C methods, which includes all operations on Python collections like lists and dicts, are atomic, making those collections safe to share and modify across threads. Most python code in the wild that utilizes threads makes use of those guarantees. So you either use fine-grained locking, or lock-free data structures, or STM, etc to protect ALL collection instances, or you've basically made a Python 3000 that breaks compatibility with all existing Python code. So every Python implementation that removes the GIL chooses to go the first route of pessimistically making everything threadsafe - which was tried in Java way back when and was and is a miserable failure. You give up too much performance for it to be worthwhile. So the problem in Python will never be fixed in my opinion - the problem is Python and there's no saving it. Without breaking backwards compatibility it will always be better to use processes instead of threads and keep the GIL.
  - fatbird 10y ago
    You should watch the video. I won't try to repeat it all here, but Hastings identifies the performance problem areas and lays out some plausible directions for tackling it, and a plausible implementation/changeover path that doesn't create another big breaking change.
  - jerf 10y ago
    If you carefully read what I said, I think you'll find there's less disagreement between what you posted and what I said than your tone suggests you think. I did, for instance, mention C methods, but you seem to imply that that matters to all Python implementations. It only matters to CPython. Barring what last I knew was some fairly experimental stuff from PyPy, Python extensions are not generally compatible between the implementations. (Where you see libraries that appear to work on multiple implementations, AIUI you're actually seeing multiple implementations of the binding code.) Personally my feeling is that if you care enough about performance to be threading in Python, you care enough about performance to not be using Python anyhow. It's a nice language, it's by far my favorite of the scripting languages of its family, but even with all the PyPy JIT and other magic tech words you can throw at it, it's still a slow language. When that doesn't matter, great; when it does, you eventually reach the point where you're stuck. (I've never been overwhelmed by the "implement it in C then" argument, for many reasons, and nowadays to an increasing degree if you're going to implement your "core" in another language for speed, there's a language you can choose that is both nice to use and already fast for your task, and that's only going to get more true as time goes on.)
    - eloff 10y ago
      The C methods do unfortunately matter to all implementations. They gave thread safety and almost all Python code in the wild relies on that property, so it's become an undocumented property of the Python language specification and an intrinsic flaw of the language itself.
  - andreasvc 10y ago
    Reference counting and the C API are properties of the CPython implementation, not Python the language. Other Python implementations do NOT use reference counting and don't offer the C API, but do make guarantees about atomic operations. I think the problem with backward compatibility is with wanting to preserve the C API and the existing extension modules implemented in C, not necessarily with Python the language.
fatbird 10y ago
Nothing inherent, no. Hastings discusses this is the talk: short version is that having a GIL gets python a fairly simple, straightforward implementation (also for extension authors); not having it requires a lot of work to achieve comparable performance. This is why so many interpreted languages use a GIL (like Ruby). No language requires it, it just makes the language much easier to develop and extend. Implementations like Jython that don't have one, don't because the hard work to not have one is already done in the VM. It only took him about a week to remove the GIL (and the talk includes the steps); all this work is making a GIL-less python work as well as one with a GIL.
ubernostrum 10y ago
Not really. Other languages have a GIL or GIL-like. The backstory of Python's GIL is a tradeoff made nearly a quarter-century ago. Python came from the Unix world, where forking processes was the common way of having multiple lines of execution, but in the 90s threading was starting to take off as an alternative (insert standard anecdote about Java adopting threading since it had to run on set-top TV boxes that couldn't handle multi-process concurrency). So Python needed to gain support for threading, but also needed to not horribly break a lot of existing code -- particularly extensions written in C -- and not destroy performance. Which was an issue since Python's memory management and garbage collection internals were not thread-safe, and it would've been both a major slowdown and a major breaking change to just say "all right, we'll do this in the theoretically best possible way". The trade-off solution was to introduce this global lock which needed to be acquired and released. The side effects of the lock are twofold: 1. Only a single thread can be executing at a time within a given Python interpreter process, and 2. The bookkeeping around the lock introduces overhead. This overhead is negligible in some cases but ruinous in others. The tradeoff seemed reasonable at the time, though. For one thing, hardware capable of exposing the single-thread limit as a relevant limit wasn't exactly common in 1992. For another, the overhead of the GIL mainly bites you in CPU-bound workloads, and at the time people wanted threading to write I/O-bound applications like networking daemons. Here in 2016, of course, everybody has a multi-core computer in their pocket and we see plenty of CPU-bound workloads that we'd like to parallelize. Which is why people complain about the GIL. But now it's even more of a breaking change to try to remove it, and still has performance implications.
the_mitsuhiko 10y ago
> Is there anything inherent to the language that makes this global lock thing difficult to be without in Python, but not in other languages? Both language and interpreter design. I was wanting to give a talk or at least write about some of the internals of the interpreter and language and how to do it better next time. In case there is interest for this I might actually get around doing that for once. (I have played around with a different from of GIL-less execution a few years back that was based on independent interpreters and message passing thread bound objects but I ran into so many issues with the interpreter :()
- eric_bullington 10y ago
  > In case there is interest for this I might actually get around doing that for once. Yes, I'm very interested in your thoughts on this, particularly wrt interpreter design.
- bakery2k 10y ago
  I would certainly be interested to learn about better language and interpreter design. Do you think anyone will ever use the lessons learned about interpreters and write a "better next time" implementation of Python, or do you only see an improved runtime appearing alongside a significantly different language?
pizlonator 10y ago
The biggest problem is the use of reference counting. Consider this simple statement: tmp = p.f; o.f = tmp; In a garbage-collected language that supports concurrency and has no GIL, this is a wait-free operations with no locks. You load p.f. You store o.f. Maybe there is a GC barrier, but that fast-paths to a load-branch combo 99% of the time. It's all easy and fast. In a reference-counted language that supports concurrency and has no GIL, you're in a world of hurt. First you have to make sure that you atomically increment the reference count on 'tmp'. This is guaranteed to have to store to the memory that 'tmp' points to. Note that in the GC world, no such store happened. Already here you have a problem: you're storing to memory much more often. Stores are the things that parallel CPUs hate, because the hardware now has to do work to propagate the value you stored to the other CPUs. But wait, there's more. You also have to atomically decrement the reference count of the object that 'o.f' used to point to. That's another store that will have to be propagated. Now your algorithm for "o.f = tmp" looks like this: 1. Atomically increment the reference count of 'tmp'. 2. Load the old value of o.f and atomically decrement the reference count. If it drops to zero, delete the object. 3. Store 'tmp' into 'o.f'. But wait, you're not done yet. You also have to handle the dec race on 'o.f'. If two threads simultaneously store to 'o.f', in which case you now have a race in steps 2 and 3. You'd think that you can just use an atomic swap in (2), but then you'd be wrong: you don't want two threads to both load the same old object from (lets that one 'oldfart') and dec it. Here's an example: Thread #1: o.f = tmp; Thread #2: o.f = thingy; If the interleaving is Thread #1 executes step (2) then Thread #2 executes step (2) then Thread #1 executes step (3) then Thread #2 executes step (3), then you will dec 'oldfart' twice, inc 'tmp' once, and inc 'thingy' once. This means that you've over-released 'oldfart' and over-retained 'tmp'. Bad news! So, you have to put a lock around steps (2) and (3). Basically every field in memory has to have a lock around it. That sucks! It's kind of funny to me that people who are stuck with reference counting say things like "yeah, garbage collection can be made to be as fast as reference counting". That makes me lol. Garbage collection is so much faster than reference counting, particularly in concurrency scenarios, because: In a GC: "o.f = p.f" is a load and a store, with maybe an extra load-branch if you have a barrier. That's it. There's no locks! There's no atomics! Notably, the only thing being stored to is "o.f", just as the user intended. In reference counting: "o.f = p.f" means atomic ops on 'tmp', a lock around "o.f", and atomic ops on 'oldfart'. The lock will require one or two atomic ops. There will be atomic stores to the cache line of "o.f", the cache line of "tmp", and the cache line of "oldfart". What a mess! TL;DR. Garbage collection is a slam dunk of overwhelming awesomeness.

dalke 10y ago

Inherent? No. Neither Jython nor Iron Python have a GIL.

jerf 10y ago

It was ground in to CPython at an early date and every Python C extension ends up with it ground in, too, because it's in the C API. Some things are very difficult to retrofit on to an existing system. [1] If you create a new Python implementation from scratch it doesn't need to be there. It is not intrinsic to Python; it is intrinsic to the specific CPython implementation. It is also, broadly speaking, easy to remove the GIL. However the bar the CPython developers have set is that they will not accept a GIL-removal patch that harms single-threaded performance. This is the bar that has proved difficult to hurdle. It is also fair to point out that while it's a perennial topic in the Python community, it is not the only scripting language implementation with a GIL or moral equivalent in it. [1]: For similar reasons, none of the core implementations of the 1990s-era dynamic scripting languages have a very good true multi-threading story. (Some of the alternates can do it, like Jython.) They don't all even have implementations, and last I knew, none of them have anything that you ought to use in production. I don't think this is a fundamental limitation of any of the languages in question, it's just that it's really hard to retrofit threading onto a non-threaded code base after ~ten very heavy years of development. There's a set of other things that are hard to retrofit in to an existing code base if they don't start there from day one, like "unit testability" or "proper string handling".

eloff 10y ago
This is not true - it is intrinsic to Python - not just CPython. Part of it is the garbage collector, reference counting is expensive to do without the GIL - but that can be changed. IronPython, Jython, some translations of PyPy all attest to that. The other part is C methods, which includes all operations on Python collections like lists and dicts, are atomic, making those collections safe to share and modify across threads. Most python code in the wild that utilizes threads makes use of those guarantees. So you either use fine-grained locking, or lock-free data structures, or STM, etc to protect ALL collection instances, or you've basically made a Python 3000 that breaks compatibility with all existing Python code. So every Python implementation that removes the GIL chooses to go the first route of pessimistically making everything threadsafe - which was tried in Java way back when and was and is a miserable failure. You give up too much performance for it to be worthwhile. So the problem in Python will never be fixed in my opinion - the problem is Python and there's no saving it. Without breaking backwards compatibility it will always be better to use processes instead of threads and keep the GIL.
- fatbird 10y ago
  You should watch the video. I won't try to repeat it all here, but Hastings identifies the performance problem areas and lays out some plausible directions for tackling it, and a plausible implementation/changeover path that doesn't create another big breaking change.
- jerf 10y ago
  If you carefully read what I said, I think you'll find there's less disagreement between what you posted and what I said than your tone suggests you think. I did, for instance, mention C methods, but you seem to imply that that matters to all Python implementations. It only matters to CPython. Barring what last I knew was some fairly experimental stuff from PyPy, Python extensions are not generally compatible between the implementations. (Where you see libraries that appear to work on multiple implementations, AIUI you're actually seeing multiple implementations of the binding code.) Personally my feeling is that if you care enough about performance to be threading in Python, you care enough about performance to not be using Python anyhow. It's a nice language, it's by far my favorite of the scripting languages of its family, but even with all the PyPy JIT and other magic tech words you can throw at it, it's still a slow language. When that doesn't matter, great; when it does, you eventually reach the point where you're stuck. (I've never been overwhelmed by the "implement it in C then" argument, for many reasons, and nowadays to an increasing degree if you're going to implement your "core" in another language for speed, there's a language you can choose that is both nice to use and already fast for your task, and that's only going to get more true as time goes on.)
  - eloff 10y ago
    The C methods do unfortunately matter to all implementations. They gave thread safety and almost all Python code in the wild relies on that property, so it's become an undocumented property of the Python language specification and an intrinsic flaw of the language itself.
- andreasvc 10y ago
  Reference counting and the C API are properties of the CPython implementation, not Python the language. Other Python implementations do NOT use reference counting and don't offer the C API, but do make guarantees about atomic operations. I think the problem with backward compatibility is with wanting to preserve the C API and the existing extension modules implemented in C, not necessarily with Python the language.

fatbird 10y ago

Nothing inherent, no. Hastings discusses this is the talk: short version is that having a GIL gets python a fairly simple, straightforward implementation (also for extension authors); not having it requires a lot of work to achieve comparable performance. This is why so many interpreted languages use a GIL (like Ruby). No language requires it, it just makes the language much easier to develop and extend. Implementations like Jython that don't have one, don't because the hard work to not have one is already done in the VM. It only took him about a week to remove the GIL (and the talk includes the steps); all this work is making a GIL-less python work as well as one with a GIL.

ubernostrum 10y ago

Not really. Other languages have a GIL or GIL-like. The backstory of Python's GIL is a tradeoff made nearly a quarter-century ago. Python came from the Unix world, where forking processes was the common way of having multiple lines of execution, but in the 90s threading was starting to take off as an alternative (insert standard anecdote about Java adopting threading since it had to run on set-top TV boxes that couldn't handle multi-process concurrency). So Python needed to gain support for threading, but also needed to not horribly break a lot of existing code -- particularly extensions written in C -- and not destroy performance. Which was an issue since Python's memory management and garbage collection internals were not thread-safe, and it would've been both a major slowdown and a major breaking change to just say "all right, we'll do this in the theoretically best possible way". The trade-off solution was to introduce this global lock which needed to be acquired and released. The side effects of the lock are twofold: 1. Only a single thread can be executing at a time within a given Python interpreter process, and 2. The bookkeeping around the lock introduces overhead. This overhead is negligible in some cases but ruinous in others. The tradeoff seemed reasonable at the time, though. For one thing, hardware capable of exposing the single-thread limit as a relevant limit wasn't exactly common in 1992. For another, the overhead of the GIL mainly bites you in CPU-bound workloads, and at the time people wanted threading to write I/O-bound applications like networking daemons. Here in 2016, of course, everybody has a multi-core computer in their pocket and we see plenty of CPU-bound workloads that we'd like to parallelize. Which is why people complain about the GIL. But now it's even more of a breaking change to try to remove it, and still has performance implications.

the_mitsuhiko 10y ago

> Is there anything inherent to the language that makes this global lock thing difficult to be without in Python, but not in other languages? Both language and interpreter design. I was wanting to give a talk or at least write about some of the internals of the interpreter and language and how to do it better next time. In case there is interest for this I might actually get around doing that for once. (I have played around with a different from of GIL-less execution a few years back that was based on independent interpreters and message passing thread bound objects but I ran into so many issues with the interpreter :()

eric_bullington 10y ago
> In case there is interest for this I might actually get around doing that for once. Yes, I'm very interested in your thoughts on this, particularly wrt interpreter design.
bakery2k 10y ago
I would certainly be interested to learn about better language and interpreter design. Do you think anyone will ever use the lessons learned about interpreters and write a "better next time" implementation of Python, or do you only see an improved runtime appearing alongside a significantly different language?

pizlonator 10y ago

The biggest problem is the use of reference counting. Consider this simple statement: tmp = p.f; o.f = tmp; In a garbage-collected language that supports concurrency and has no GIL, this is a wait-free operations with no locks. You load p.f. You store o.f. Maybe there is a GC barrier, but that fast-paths to a load-branch combo 99% of the time. It's all easy and fast. In a reference-counted language that supports concurrency and has no GIL, you're in a world of hurt. First you have to make sure that you atomically increment the reference count on 'tmp'. This is guaranteed to have to store to the memory that 'tmp' points to. Note that in the GC world, no such store happened. Already here you have a problem: you're storing to memory much more often. Stores are the things that parallel CPUs hate, because the hardware now has to do work to propagate the value you stored to the other CPUs. But wait, there's more. You also have to atomically decrement the reference count of the object that 'o.f' used to point to. That's another store that will have to be propagated. Now your algorithm for "o.f = tmp" looks like this: 1. Atomically increment the reference count of 'tmp'. 2. Load the old value of o.f and atomically decrement the reference count. If it drops to zero, delete the object. 3. Store 'tmp' into 'o.f'. But wait, you're not done yet. You also have to handle the dec race on 'o.f'. If two threads simultaneously store to 'o.f', in which case you now have a race in steps 2 and 3. You'd think that you can just use an atomic swap in (2), but then you'd be wrong: you don't want two threads to both load the same old object from (lets that one 'oldfart') and dec it. Here's an example: Thread #1: o.f = tmp; Thread #2: o.f = thingy; If the interleaving is Thread #1 executes step (2) then Thread #2 executes step (2) then Thread #1 executes step (3) then Thread #2 executes step (3), then you will dec 'oldfart' twice, inc 'tmp' once, and inc 'thingy' once. This means that you've over-released 'oldfart' and over-retained 'tmp'. Bad news! So, you have to put a lock around steps (2) and (3). Basically every field in memory has to have a lock around it. That sucks! It's kind of funny to me that people who are stuck with reference counting say things like "yeah, garbage collection can be made to be as fast as reference counting". That makes me lol. Garbage collection is so much faster than reference counting, particularly in concurrency scenarios, because: In a GC: "o.f = p.f" is a load and a store, with maybe an extra load-branch if you have a barrier. That's it. There's no locks! There's no atomics! Notably, the only thing being stored to is "o.f", just as the user intended. In reference counting: "o.f = p.f" means atomic ops on 'tmp', a lock around "o.f", and atomic ops on 'oldfart'. The lock will require one or two atomic ops. There will be atomic stores to the cache line of "o.f", the cache line of "tmp", and the cache line of "oldfart". What a mess! TL;DR. Garbage collection is a slam dunk of overwhelming awesomeness.