11 ms·

The Gilectomy – How's It Going [video]

73 points by varunramesh 9y ago

comex 9y ago
If you're the type that prefers to read text, here's LWN's writeup of the linked talk: https://lwn.net/SubscriberLink/723514/f674d4a807264ba1/ https://lwn.net/SubscriberLink/723514/f674d4a807264ba1/
- scribu 9y ago
  I've enjoyed LWN articles in the past, but found this particular writeup very tedious to follow, compared to watching the video. A direct transcript, perhaps with some light editing, would have been more useful, IMO.
chairmanwow 9y ago
Is he being serious when he says he only has one test case? That really doesn't seem like a reasonable thing to do. Furthermore, would a recursive implementation of Fibonacci even benefit from multithreading?
- m_mueller 9y ago
  Basically, his point at the moment is not to construct some benchmarks that show some multithread benefits and then call victory. Instead he wants to reduce its overhead so it won't make existing (thus unoptimised for Gilectomy) programs much slower - which is probably necessary for main line adoption. As such I find his approach really refreshing, interesting and honest: Basically start with the worst case first, get everyone's expectation down and then at the end show that the situation is actually not as hopeless as you might think.
- wulfjack 9y ago
  The goal should be, and is kind of what Larry Hastings is looking for, is that any program should run 8 times faster on a 8-core CPU compared to a 1-core. And as said above Python can basically only use one core b/c of GIL. Actually Python 2.7 multithreading runs much slower on a multicore CPU than on a single core due to locking congestion on the GIL.
  - marvy 9y ago
    What? No! Multithreaded programs should run faster on 8 cores than on one core. That's not very realistic for single-threaded programs, in any language. I could be wrong, but I think Py2.7 is about the same speed on multicore vs 1 core. Where did you get that idea?
    - diek 9y ago
      Python 2.7 has terrible thrashing in the way the GIL is acquired that is exacerbated as more threads are used. Dave Beazley has given great talks with the technical details: http://www.dabeaz.com/GIL/ http://www.dabeaz.com/GIL/
    - alfanerd 9y ago
      Some years ago I wrote pyworks (www.github.com/pylots/pyworks), a proposal for async objects (inspired by ABCL and cooC (Concurrent Object Oriented-C, https://www.researchgate.net/publication/220178380_Concurrent_Object_Oriented_%27C%27_cooC https://www.researchgate.net/publication/220178380_Concurren...) The testcase is 100 threads sending 1000 messages in to each other in a ring. On a 8-core Mac Jython and IronPython performs better than on 1-core, but Python 2.7 performs so badly that it never finishes. The ideal scenario is probably that the CPython interpreter starts one thread per core running as many coroutines in parallel as possible, but that looks like a long way away for Python
  - thomaslee 9y ago
    > The goal should be, and is kind of what Larry Hastings is looking for, is that any program should run 8 times faster on a 8-core CPU compared to a 1-core. A program that's inherently single-threaded it's unlikely to benefit from more CPUs. When you say "any program" here, you mean "any program with >=8 threads", right?
    - alfanerd 9y ago
      I mean the developer (using a highlevel language as Python) ideally should get more performance on an 8-core than on a 1-core CPU. Erlang, which btw. is older than Python, will perform better the more cores you have due to its message oriented nature. Python (2.7) on the other hand performed worse with multithreading on multicore. I was hoping that Python would take the same direction in the future, but unfortunately we are getting the async/await mess, instead of a simple async object model (sorry, my pet peeve)
ars 9y ago
Gilectomy project: the removal of Python's Global Interpreter Lock, or "GIL".
WaxProlix 9y ago
It's funny, I've written a lot of python in quite a few domains and haven't really struggled directly because of the GIL before. Is this more of a 'data scientist' problem? I feel like if I had a huge pile of data to crunch, python wouldn't be my first choice really.
- askvictor 9y ago
  Strangely, python is the choice for heaps of (data and other) scientists, mainly as it's so easy to pick up and tooling (jupyter) - remember that these people aren't native coders. And even then parallelism is handled behind the scenes with libraries such as scipy and numpy. I think the problems come up when you have a high performance application that doesn't play nice with numpy or scipy
- prewett 9y ago
  Servers are where the problem is. The GIL makes python functionally single-threaded, which is a bummer for your server at any kind of scale. So you end up having to have n cores' worth of server processes behind a load balancer, even if you only need one server machine, which is a bummer if a you have a stateful server (such as a game server), as you now have to manage communicating state between processes by storing it in another process (frequently Redis). But python is easy and fun to write code in, and "developer time is expensive, servers are cheap," so there are a lot of python servers which could benefit from a lack of GIL. Never mind that it's fast to write, but difficult to maintain since it is a dynamically typed language and one typo creates runtime errors that any statically typed language will catch at compile time. Or that it is slow. Or that a python process never really releases memory back to the system, just within itself, so the process slowly grows over the course of a few weeks. Or that the Twisted framework you're using for cooperative multitasking because of the GIL is really easy to block on a database query by accident, leading to uncooperative multitasking (= large lags), resulting to forced server restarts, and loss of players (= loss of revenue). So yeah, "developer time is cheap" but it's sort of an expensive cheap. I came to the conclusion that python is unsuitable for servers, but until Go came out, there wasn't a realistic alternative, since C++ and Java are too heavyweight, and Ruby suffers from similar problems (don't know about a GIL).
amelius 9y ago
Can't they use the GC techniques used in other languages? I've heard that Golang has a very efficient concurrent garbage collector.
- peterhunt 9y ago
  Implementing a tracing gc was covered in the video.
  - amelius 9y ago
    Well, I didn't see the video yet, but I noticed they are referencing "The Garbage Collector Handbook", which is from 2011. The people from Golang have had some more recent successes with their concurrent garbage collector, which, as I've heard, is really efficient.
    - poooogles 9y ago
      >The people from Golang have had some more recent successes with their concurrent garbage collector, which, as I've heard, is really efficient. Haven't they just sacrificed throughout for pause time though? Not a GC expert at all, but that's what Ive got the gist of from speaking to people.
- ubernostrum 9y ago
  Before making a comment which begins with: "Why don't they just..." "Can't they just..." "Haven't they heard of..." or other similar phrases, please 1. Read through the linked content (or if it's a video, watch it or find a transcript), and 2. Bonus points: familiarize yourself with the problem domain. Doing (1) will usually answer these questions all on its own. If it doesn't, doing (2) will help. In this case, we're not talking about a random amateur starting fresh on a problem nobody's considered thoroughly yet. Python's global interpreter lock is extremely well-trod ground; the people working on it are not just experienced programmers but experienced with Python; and there's a lot of background out there, available in an easy Google search, explaining what's been tried and what's been ruled out over the history of the problem. In particular, the two problems you'd learn about from following the advice above are: * A lot of important existing Python code consists of modules which are partially or entirely written in C and depend on the documented Python/C API remaining stable. Breaking the C API, and thus forcing all that existing code to rewrite to a new API, is considered an unacceptable solution (this rules out many "why not just use (insert your favorite GC technique here)" approaches). * Prior attempts at removing the global interpreter lock while not breaking existing Python/C code have caused major performance degradation for single-threaded Python code, which is also considered unacceptable.
thomaslee 9y ago
If any Python devs are out there reading: my understanding is that removing the GIL itself isn't the hard part so much as removing the GIL while satisfying certain constraints deemed necessary by GvR and/or the rest of the community. I know some of those constraints relate to compatibility with existing C extensions -- but there must be others too? The reason I ask is Larry's attempt buffered ref counting surely has implications for single-threaded code that maybe relies on the existing semantics -- e.g. a program like this may no longer reliably print "Deallocated!": Python 2.7.13 (default, Mar 5 2017, 00:33:10) [GCC 6.3.0 20170205] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class Foo(object): ... def __del__(self): ... print 'Deallocated!' ... >>> foo = Foo() >>> foo = None Deallocated! >>> A bad example in some ways since in this particular case we could wait for all ref counting operations to be processed before letting the interpreter exit, but hopefully my point is still clear. Similarly, what about multi-threaded Python code that isn't written to operate in a GIL-free environment -- absent locks, atomic reads/writes, etc.? At best, you might expect some bad results. At worst, segfaults. Are these all bridges that need to be crossed once a realistic solution to the core GIL removal issue is proposed? As glad as I am that folks are still thinking hard about this problem, I'm personally sort of pessimistic that the GIL can be killed off without a policy change wrt backward compatibility. Still, I do sort of wonder if some rules of engagement wrt departures from existing semantics might help drive a solution.
- jholman 9y ago
  If I'm understanding you, some or all of these questions are explicitly addressed in the Q&A. My apologies if you got that far and I simply didn't understand you. For example, your first question seems to be asking about whether there's a semantic change coming from a lack of immediacy in when __del__ will run. And the answer is explicitly "yes, and the docs already told you not to count on that". As for multi-threaded Python code... and perhaps also multi-threaded C code in extensions... I think the clear answer is "yes, our whole goal is to remove some guarantees that were previously provided, so if you counted on those guarantees you're in trouble". Again, c.f. the Q&A in case that helps. From the talk, it doesn't look to me like Larry Hastings has a plan for the policy change in question; so maybe "bridges that need to be crossed once [the technical issues are smaller]" is correct?
mrfusion 9y ago
Can Python copy how other languages like golang or java operate without a Gil? Why or why not?
- bbayles 9y ago
  Yes, it can. The purpose of the project is to figure out how to do that while minimizing the number of APIs that have to break.
- chrisseaton 9y ago
  Python has different semantics to those languages. I don't think it's formally specified, but people program in Python expecting the semantics that reference counting provides, and that unsychrnoised concurrent access to data structures will not cause errors. Despite ongoing research, it appears to be hard to continue to provide these semantics without a GIL. Golang (informally I think) and Java (more formally) are not specified to provide reference counting semantics and not specified to guarantee that unsychrnoised concurrent access to data structures will not cause errors. So the languages have different semantics - that's why you can't copy-and-paste the solution from one to another. Some alternative implementations of Python don't follow the above semantics, like Jython, but then some people aren't happy with that. It may not be acceptable to the community to drop those semantics, even if they were never formally given.
  - munin 9y ago
    > and that unsychrnoised concurrent access to data structures will not cause errors Python doesn't ensure that unsynchornized concurrent access to data structures won't cause errors. As I understand and experience Python multithreading, all the GIL ensures is that of the "load - inc - store" stages running amonst N threads, each separate stage will be locked, but not the overall sequence. So you'll still have data races, even with the GIL, and you still need to use mutexes etc in your Python program, which is why they are there.
    - chrisseaton 9y ago
      I mean it won't cause errors within the basic data structure access operations. I'm not talking about composition. In Java a hash table write can fail with an exception if there is a concurrent write that conflicts. A Python dict write is atomic, because it happens within a single instruction as you say and so will not be interrupted and will never fail. That's what you aren't getting in Java. That expectation is very hard to provide without a GIL. Jython does it with blunt fine grained locking, but that's slow.

73 points by varunramesh 9y ago

If you're the type that prefers to read text, here's LWN's writeup of the linked talk: https://lwn.net/SubscriberLink/723514/f674d4a807264ba1/ https://lwn.net/SubscriberLink/723514/f674d4a807264ba1/

scribu 9y ago
I've enjoyed LWN articles in the past, but found this particular writeup very tedious to follow, compared to watching the video. A direct transcript, perhaps with some light editing, would have been more useful, IMO.

chairmanwow 9y ago

Is he being serious when he says he only has one test case? That really doesn't seem like a reasonable thing to do. Furthermore, would a recursive implementation of Fibonacci even benefit from multithreading?

m_mueller 9y ago
Basically, his point at the moment is not to construct some benchmarks that show some multithread benefits and then call victory. Instead he wants to reduce its overhead so it won't make existing (thus unoptimised for Gilectomy) programs much slower - which is probably necessary for main line adoption. As such I find his approach really refreshing, interesting and honest: Basically start with the worst case first, get everyone's expectation down and then at the end show that the situation is actually not as hopeless as you might think.
wulfjack 9y ago
The goal should be, and is kind of what Larry Hastings is looking for, is that any program should run 8 times faster on a 8-core CPU compared to a 1-core. And as said above Python can basically only use one core b/c of GIL. Actually Python 2.7 multithreading runs much slower on a multicore CPU than on a single core due to locking congestion on the GIL.
- marvy 9y ago
  What? No! Multithreaded programs should run faster on 8 cores than on one core. That's not very realistic for single-threaded programs, in any language. I could be wrong, but I think Py2.7 is about the same speed on multicore vs 1 core. Where did you get that idea?
  - diek 9y ago
    Python 2.7 has terrible thrashing in the way the GIL is acquired that is exacerbated as more threads are used. Dave Beazley has given great talks with the technical details: http://www.dabeaz.com/GIL/ http://www.dabeaz.com/GIL/
  - alfanerd 9y ago
    Some years ago I wrote pyworks (www.github.com/pylots/pyworks), a proposal for async objects (inspired by ABCL and cooC (Concurrent Object Oriented-C, https://www.researchgate.net/publication/220178380_Concurrent_Object_Oriented_%27C%27_cooC https://www.researchgate.net/publication/220178380_Concurren...) The testcase is 100 threads sending 1000 messages in to each other in a ring. On a 8-core Mac Jython and IronPython performs better than on 1-core, but Python 2.7 performs so badly that it never finishes. The ideal scenario is probably that the CPython interpreter starts one thread per core running as many coroutines in parallel as possible, but that looks like a long way away for Python
- thomaslee 9y ago
  > The goal should be, and is kind of what Larry Hastings is looking for, is that any program should run 8 times faster on a 8-core CPU compared to a 1-core. A program that's inherently single-threaded it's unlikely to benefit from more CPUs. When you say "any program" here, you mean "any program with >=8 threads", right?
  - alfanerd 9y ago
    I mean the developer (using a highlevel language as Python) ideally should get more performance on an 8-core than on a 1-core CPU. Erlang, which btw. is older than Python, will perform better the more cores you have due to its message oriented nature. Python (2.7) on the other hand performed worse with multithreading on multicore. I was hoping that Python would take the same direction in the future, but unfortunately we are getting the async/await mess, instead of a simple async object model (sorry, my pet peeve)

ars 9y ago

Gilectomy project: the removal of Python's Global Interpreter Lock, or "GIL".

WaxProlix 9y ago

It's funny, I've written a lot of python in quite a few domains and haven't really struggled directly because of the GIL before. Is this more of a 'data scientist' problem? I feel like if I had a huge pile of data to crunch, python wouldn't be my first choice really.

askvictor 9y ago
Strangely, python is the choice for heaps of (data and other) scientists, mainly as it's so easy to pick up and tooling (jupyter) - remember that these people aren't native coders. And even then parallelism is handled behind the scenes with libraries such as scipy and numpy. I think the problems come up when you have a high performance application that doesn't play nice with numpy or scipy
prewett 9y ago
Servers are where the problem is. The GIL makes python functionally single-threaded, which is a bummer for your server at any kind of scale. So you end up having to have n cores' worth of server processes behind a load balancer, even if you only need one server machine, which is a bummer if a you have a stateful server (such as a game server), as you now have to manage communicating state between processes by storing it in another process (frequently Redis). But python is easy and fun to write code in, and "developer time is expensive, servers are cheap," so there are a lot of python servers which could benefit from a lack of GIL. Never mind that it's fast to write, but difficult to maintain since it is a dynamically typed language and one typo creates runtime errors that any statically typed language will catch at compile time. Or that it is slow. Or that a python process never really releases memory back to the system, just within itself, so the process slowly grows over the course of a few weeks. Or that the Twisted framework you're using for cooperative multitasking because of the GIL is really easy to block on a database query by accident, leading to uncooperative multitasking (= large lags), resulting to forced server restarts, and loss of players (= loss of revenue). So yeah, "developer time is cheap" but it's sort of an expensive cheap. I came to the conclusion that python is unsuitable for servers, but until Go came out, there wasn't a realistic alternative, since C++ and Java are too heavyweight, and Ruby suffers from similar problems (don't know about a GIL).

amelius 9y ago

Can't they use the GC techniques used in other languages? I've heard that Golang has a very efficient concurrent garbage collector.

peterhunt 9y ago
Implementing a tracing gc was covered in the video.
- amelius 9y ago
  Well, I didn't see the video yet, but I noticed they are referencing "The Garbage Collector Handbook", which is from 2011. The people from Golang have had some more recent successes with their concurrent garbage collector, which, as I've heard, is really efficient.
  - poooogles 9y ago
    >The people from Golang have had some more recent successes with their concurrent garbage collector, which, as I've heard, is really efficient. Haven't they just sacrificed throughout for pause time though? Not a GC expert at all, but that's what Ive got the gist of from speaking to people.
ubernostrum 9y ago
Before making a comment which begins with: "Why don't they just..." "Can't they just..." "Haven't they heard of..." or other similar phrases, please 1. Read through the linked content (or if it's a video, watch it or find a transcript), and 2. Bonus points: familiarize yourself with the problem domain. Doing (1) will usually answer these questions all on its own. If it doesn't, doing (2) will help. In this case, we're not talking about a random amateur starting fresh on a problem nobody's considered thoroughly yet. Python's global interpreter lock is extremely well-trod ground; the people working on it are not just experienced programmers but experienced with Python; and there's a lot of background out there, available in an easy Google search, explaining what's been tried and what's been ruled out over the history of the problem. In particular, the two problems you'd learn about from following the advice above are: * A lot of important existing Python code consists of modules which are partially or entirely written in C and depend on the documented Python/C API remaining stable. Breaking the C API, and thus forcing all that existing code to rewrite to a new API, is considered an unacceptable solution (this rules out many "why not just use (insert your favorite GC technique here)" approaches). * Prior attempts at removing the global interpreter lock while not breaking existing Python/C code have caused major performance degradation for single-threaded Python code, which is also considered unacceptable.

thomaslee 9y ago

If any Python devs are out there reading: my understanding is that removing the GIL itself isn't the hard part so much as removing the GIL while satisfying certain constraints deemed necessary by GvR and/or the rest of the community. I know some of those constraints relate to compatibility with existing C extensions -- but there must be others too? The reason I ask is Larry's attempt buffered ref counting surely has implications for single-threaded code that maybe relies on the existing semantics -- e.g. a program like this may no longer reliably print "Deallocated!": Python 2.7.13 (default, Mar 5 2017, 00:33:10) [GCC 6.3.0 20170205] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class Foo(object): ... def __del__(self): ... print 'Deallocated!' ... >>> foo = Foo() >>> foo = None Deallocated! >>> A bad example in some ways since in this particular case we could wait for all ref counting operations to be processed before letting the interpreter exit, but hopefully my point is still clear. Similarly, what about multi-threaded Python code that isn't written to operate in a GIL-free environment -- absent locks, atomic reads/writes, etc.? At best, you might expect some bad results. At worst, segfaults. Are these all bridges that need to be crossed once a realistic solution to the core GIL removal issue is proposed? As glad as I am that folks are still thinking hard about this problem, I'm personally sort of pessimistic that the GIL can be killed off without a policy change wrt backward compatibility. Still, I do sort of wonder if some rules of engagement wrt departures from existing semantics might help drive a solution.

jholman 9y ago
If I'm understanding you, some or all of these questions are explicitly addressed in the Q&A. My apologies if you got that far and I simply didn't understand you. For example, your first question seems to be asking about whether there's a semantic change coming from a lack of immediacy in when __del__ will run. And the answer is explicitly "yes, and the docs already told you not to count on that". As for multi-threaded Python code... and perhaps also multi-threaded C code in extensions... I think the clear answer is "yes, our whole goal is to remove some guarantees that were previously provided, so if you counted on those guarantees you're in trouble". Again, c.f. the Q&A in case that helps. From the talk, it doesn't look to me like Larry Hastings has a plan for the policy change in question; so maybe "bridges that need to be crossed once [the technical issues are smaller]" is correct?

mrfusion 9y ago

Can Python copy how other languages like golang or java operate without a Gil? Why or why not?

bbayles 9y ago
Yes, it can. The purpose of the project is to figure out how to do that while minimizing the number of APIs that have to break.
chrisseaton 9y ago
Python has different semantics to those languages. I don't think it's formally specified, but people program in Python expecting the semantics that reference counting provides, and that unsychrnoised concurrent access to data structures will not cause errors. Despite ongoing research, it appears to be hard to continue to provide these semantics without a GIL. Golang (informally I think) and Java (more formally) are not specified to provide reference counting semantics and not specified to guarantee that unsychrnoised concurrent access to data structures will not cause errors. So the languages have different semantics - that's why you can't copy-and-paste the solution from one to another. Some alternative implementations of Python don't follow the above semantics, like Jython, but then some people aren't happy with that. It may not be acceptable to the community to drop those semantics, even if they were never formally given.
- munin 9y ago
  > and that unsychrnoised concurrent access to data structures will not cause errors Python doesn't ensure that unsynchornized concurrent access to data structures won't cause errors. As I understand and experience Python multithreading, all the GIL ensures is that of the "load - inc - store" stages running amonst N threads, each separate stage will be locked, but not the overall sequence. So you'll still have data races, even with the GIL, and you still need to use mutexes etc in your Python program, which is why they are there.
  - chrisseaton 9y ago
    I mean it won't cause errors within the basic data structure access operations. I'm not talking about composition. In Java a hash table write can fail with an exception if there is a concurrent write that conflicts. A Python dict write is atomic, because it happens within a single instruction as you say and so will not be interrupted and will never fail. That's what you aren't getting in Java. That expectation is very hard to provide without a GIL. Jython does it with blunt fine grained locking, but that's slow.