5 ms·

Presentation on the new Python GIL

77 points by voxcogitatio 17y ago

sagarm 17y ago
Does anyone know why multiple kernel threads are used at all? If only one thread can run at a time, why not just use a single kernel thread?
- kingkilr 17y ago
  Because CPython doesn't want to invent it's own scheduler (and various other things that your operating system already does for you), further multiple threads can be active at the same time if they're doing any operations that release the GIL (such as I/O), Python can't know whether your thread will do any I/O before you create it.
  - scott_s 17y ago
    However, the approach explained in this presentation does just that. The solution looks like a crude process operating systems scheduler.
    - kingkilr 17y ago
      It's not really, it's simply threads asking the main thread to drop every 5 ms, and then relying on the OS to schedule them. If Python had it's own scheduler it would do things like assigning priorities and actually deciding which thread would take over when the GIL was dropped.
- haberman 17y ago
  Just to expand on the previous reply: a lot of C libraries have a blocking model where you call a function that blocks until it is complete. The simplest example is fread(). If the interpreter doesn't support multiple threads, you have to do a lot of gymnastics to make sure you never block the interpreter in an extension. It can be done (this is how Ruby pre-1.9 works), but people complain about it a lot. Matz's rationale for why he added native OS threads to Ruby 1.9 was "people seem to like them." In response to my fread() example you may be tempted to say that you can just put the underlying fd in non-blocking mode with fcntl(2). This is unfortunately unsafe (as I recall), though I cannot find the reference right now. Accessing the fd directly with read() and write() is non-blocking mode is of course safe, but then you have to do your own buffering. Also, while poll() and select() give you most of what you need to make your I/O non-blocking, I recall a case where a pipe will be write-ready, but if you try to write too much data at the same time the write will block anyway, instead of returning a short count of bytes written. This was on RHEL3 Linux, so things may have changed since then.
  - neilc 17y ago
    I recall a case where a pipe will be write-ready, but if you try to write too much data at the same time the write will block anyway, instead of returning a short count of bytes written That just sounds like a bug.
    - ajross 17y ago
      No, it's the specified behavior of a Unix pipe or socket[1]. If you want it not to block, you should set the O_NONBLOCK flag using fcntl(). Being "write ready" just means that it can take at least one byte in a buffer; it can't possibly be a promise not to block if you feed it a 2G buffer. [1] But not disk files. Local disk storage isn't considered "blocking" if it simply needs to do a disk seek before returning the data. Most unix geeks end up finding this out experimentally at some point in their careers and using strong language. No, I don't know what they were thinking either...
      - haberman 17y ago
        In the SuS description of poll(), it says that POLLOUT means "Normal data may be written without blocking." However it does not say how much data. It is unexpected that a write() call would block instead of returning a short count; for example, if you feed it a 2G buffer, it could return 4096 indicating that only 4k of the 2G was actually written.
        ajross 17y ago
        It could, but it doesn't and never has. The standard was written to admit a broader set of behavior than real systems actually implement. And in this case, there's absolutely nothing "surprising" to my eyes about a blocking (!) socket blocking on overflow. Again: there is already a mechanism in place to support non-blocking behavior, and it's not this one.
      - neilc 17y ago
        If you want it not to block, you should set the O_NONBLOCK flag using fcntl(). Certainly, I had assumed that the OP was talking about poll over non-blocking pipes.
st3fan 17y ago
"""Does it work? Yes it's better: Sequential 23.5 seconds. Threaded 24.0 seconds.""" How is that better? It is an improvement over earlier versions of Python. Better would be if the threaded version would actually be twice as fast.
- jnoller 17y ago
  An improvement over the old implementation is better. "Fixed and Perfect" would be free-threading, where it would be twice as fast with threads. I'll take incremental, concrete improvements anytime.
- Luyt 17y ago
  If you really want to speed up your program, you have to parallelize and distribute the workload over multiple cores. Python has a module in the standard library for this, it's called 'multiprocessing'. See http://docs.python.org/library/multiprocessing.html http://docs.python.org/library/multiprocessing.html
  - rbanffy 17y ago
    It won't help if you have to do stuff over a single shared data structure.
    - st3fan 17y ago
      Well you can of course. It is just a pain to do it.
      - rbanffy 17y ago
        That's a good point. I rarely had to do it and, when I could, I opted to distribute the data. Still, Python needs better threading.
- mitchellh 17y ago
  If you look at the code he used to test this, its completely CPU-bound, meaning that it requires the GIL to run. And since the GIL is, well, a "GIL," that means that only one can run at any single time. Therefore, the expected behavior of running these two threads would be to have identical runtimes. That would be the "perfect" case with a GIL. So with this "better" GIL its actually near-perfect in this _simple_ case. (Like the presenter said, there needs to be more tests to study the behavior under heavier tasks) What you're describing, with the threaded being "twice as fast" assumes that the threads will allow two python-instruction-bound tasks to run concurrently, which would require having no GIL.
fauigerzigerk 17y ago
I have a slightly weird and probably completely unworkable "idea" that has nothing to do with this particular GIL improvement but with the broader GIL issue. The problem with removing the GIL seems to be garbage collection. I wonder why it's not possible to introduce a new type of object reference that exempts referenced objects from being garbage collected. Then multiple Python interpreters could be started in separate threads, each with its own unmodified GIL, and the only thing they could access would be their own thread local data and those special shared objects. What this amounts to is basically an implementation of a multi process architecture on top of a multi threaded architecture. The crucial difference is that the memory shared among the interpreters could hold pointers and thus proper in memory data structures and not just a BLOB into which everything has to be serialized as in the case of conventional shared memory. Of course the shared objects would have to be manually deleted. One issue I see is that when such a special object is created, all the objects it creates recursively would also have to be allocated in that pool of special objects. But I think it should be possible to use some kind of global flag to special case the allocator. Well, there is probably a huge number of issues with this kind of trickery and it's definately not a long term solution. But I'd love to use Python much more than I do and the GIL issue is what prevents that for me at the moment.
- wmf 17y ago
  I think Erlang and MS Singularity use this technique to some extent. http://doi.acm.org/10.1145/1029873.1029875 http://doi.acm.org/10.1145/1029873.1029875 http://doi.acm.org/10.1145/1146809.1146813 http://doi.acm.org/10.1145/1146809.1146813 http://research.microsoft.com/apps/pubs/default.aspx?id=67482 http://research.microsoft.com/apps/pubs/default.aspx?id=6748...
  - viraptor 17y ago
    People write (http://www.julmar.com/blog/mark/PermaLink,guid,3670d081-0276-48e6-b97d-1b644093b52e.aspx http://www.julmar.com/blog/mark/PermaLink,guid,3670d081-0276...) that even .NET itself has a similar kind of GC (partially concurrent, only address fixup stops the world). Unfortunately I can't check how it compares to Singularity.
  - 17y ago
ShabbyDoo 17y ago
Knowing little of Python, is it the case that the GIL was not initially deemed a bad scheme because multi-processor x86 machines were a rarity in the mid-90's (when Python was young)? I presume that Sun desired Java to be multiprocessor friendly because it had multi-cpu SPARC machines at the time of Java's design? And, for those using Python for high-volume production systems, how do you cope? Do you run one process (say a Zope instance) per core and load balance?
- jart 17y ago
  Keep in mind the GIL is only a problem when you a) use threads and b) perform CPU-bound operations (unless you need soft real time) When writing CPU-bound code, there are many libraries like NumPy that can do your heavy lifting in native code (thus avoiding the GIL) So it really isn't as bad as it seems. The reason this patch is good, is because it prevents performance from falling apart if one of your threads takes up too much CPU.
- zepolen 17y ago
  Yep, a process per core. The beauty is that since your app has to be written to be multiprocess capable then you can also extend past one computer when the need arises. You pay a memory tax overhead though.

77 points by voxcogitatio 17y ago

Does anyone know why multiple kernel threads are used at all? If only one thread can run at a time, why not just use a single kernel thread?

kingkilr 17y ago
Because CPython doesn't want to invent it's own scheduler (and various other things that your operating system already does for you), further multiple threads can be active at the same time if they're doing any operations that release the GIL (such as I/O), Python can't know whether your thread will do any I/O before you create it.
- scott_s 17y ago
  However, the approach explained in this presentation does just that. The solution looks like a crude process operating systems scheduler.
  - kingkilr 17y ago
    It's not really, it's simply threads asking the main thread to drop every 5 ms, and then relying on the OS to schedule them. If Python had it's own scheduler it would do things like assigning priorities and actually deciding which thread would take over when the GIL was dropped.
haberman 17y ago
Just to expand on the previous reply: a lot of C libraries have a blocking model where you call a function that blocks until it is complete. The simplest example is fread(). If the interpreter doesn't support multiple threads, you have to do a lot of gymnastics to make sure you never block the interpreter in an extension. It can be done (this is how Ruby pre-1.9 works), but people complain about it a lot. Matz's rationale for why he added native OS threads to Ruby 1.9 was "people seem to like them." In response to my fread() example you may be tempted to say that you can just put the underlying fd in non-blocking mode with fcntl(2). This is unfortunately unsafe (as I recall), though I cannot find the reference right now. Accessing the fd directly with read() and write() is non-blocking mode is of course safe, but then you have to do your own buffering. Also, while poll() and select() give you most of what you need to make your I/O non-blocking, I recall a case where a pipe will be write-ready, but if you try to write too much data at the same time the write will block anyway, instead of returning a short count of bytes written. This was on RHEL3 Linux, so things may have changed since then.
- neilc 17y ago
  I recall a case where a pipe will be write-ready, but if you try to write too much data at the same time the write will block anyway, instead of returning a short count of bytes written That just sounds like a bug.
  - ajross 17y ago
    No, it's the specified behavior of a Unix pipe or socket[1]. If you want it not to block, you should set the O_NONBLOCK flag using fcntl(). Being "write ready" just means that it can take at least one byte in a buffer; it can't possibly be a promise not to block if you feed it a 2G buffer. [1] But not disk files. Local disk storage isn't considered "blocking" if it simply needs to do a disk seek before returning the data. Most unix geeks end up finding this out experimentally at some point in their careers and using strong language. No, I don't know what they were thinking either...
    - haberman 17y ago
      In the SuS description of poll(), it says that POLLOUT means "Normal data may be written without blocking." However it does not say how much data. It is unexpected that a write() call would block instead of returning a short count; for example, if you feed it a 2G buffer, it could return 4096 indicating that only 4k of the 2G was actually written.
      - ajross 17y ago
        It could, but it doesn't and never has. The standard was written to admit a broader set of behavior than real systems actually implement. And in this case, there's absolutely nothing "surprising" to my eyes about a blocking (!) socket blocking on overflow. Again: there is already a mechanism in place to support non-blocking behavior, and it's not this one.
    - neilc 17y ago
      If you want it not to block, you should set the O_NONBLOCK flag using fcntl(). Certainly, I had assumed that the OP was talking about poll over non-blocking pipes.

st3fan 17y ago

"""Does it work? Yes it's better: Sequential 23.5 seconds. Threaded 24.0 seconds.""" How is that better? It is an improvement over earlier versions of Python. Better would be if the threaded version would actually be twice as fast.

jnoller 17y ago
An improvement over the old implementation is better. "Fixed and Perfect" would be free-threading, where it would be twice as fast with threads. I'll take incremental, concrete improvements anytime.
Luyt 17y ago
If you really want to speed up your program, you have to parallelize and distribute the workload over multiple cores. Python has a module in the standard library for this, it's called 'multiprocessing'. See http://docs.python.org/library/multiprocessing.html http://docs.python.org/library/multiprocessing.html
- rbanffy 17y ago
  It won't help if you have to do stuff over a single shared data structure.
  - st3fan 17y ago
    Well you can of course. It is just a pain to do it.
    - rbanffy 17y ago
      That's a good point. I rarely had to do it and, when I could, I opted to distribute the data. Still, Python needs better threading.
mitchellh 17y ago
If you look at the code he used to test this, its completely CPU-bound, meaning that it requires the GIL to run. And since the GIL is, well, a "GIL," that means that only one can run at any single time. Therefore, the expected behavior of running these two threads would be to have identical runtimes. That would be the "perfect" case with a GIL. So with this "better" GIL its actually near-perfect in this _simple_ case. (Like the presenter said, there needs to be more tests to study the behavior under heavier tasks) What you're describing, with the threaded being "twice as fast" assumes that the threads will allow two python-instruction-bound tasks to run concurrently, which would require having no GIL.

fauigerzigerk 17y ago

I have a slightly weird and probably completely unworkable "idea" that has nothing to do with this particular GIL improvement but with the broader GIL issue. The problem with removing the GIL seems to be garbage collection. I wonder why it's not possible to introduce a new type of object reference that exempts referenced objects from being garbage collected. Then multiple Python interpreters could be started in separate threads, each with its own unmodified GIL, and the only thing they could access would be their own thread local data and those special shared objects. What this amounts to is basically an implementation of a multi process architecture on top of a multi threaded architecture. The crucial difference is that the memory shared among the interpreters could hold pointers and thus proper in memory data structures and not just a BLOB into which everything has to be serialized as in the case of conventional shared memory. Of course the shared objects would have to be manually deleted. One issue I see is that when such a special object is created, all the objects it creates recursively would also have to be allocated in that pool of special objects. But I think it should be possible to use some kind of global flag to special case the allocator. Well, there is probably a huge number of issues with this kind of trickery and it's definately not a long term solution. But I'd love to use Python much more than I do and the GIL issue is what prevents that for me at the moment.

wmf 17y ago
I think Erlang and MS Singularity use this technique to some extent. http://doi.acm.org/10.1145/1029873.1029875 http://doi.acm.org/10.1145/1029873.1029875 http://doi.acm.org/10.1145/1146809.1146813 http://doi.acm.org/10.1145/1146809.1146813 http://research.microsoft.com/apps/pubs/default.aspx?id=67482 http://research.microsoft.com/apps/pubs/default.aspx?id=6748...
- viraptor 17y ago
  People write (http://www.julmar.com/blog/mark/PermaLink,guid,3670d081-0276-48e6-b97d-1b644093b52e.aspx http://www.julmar.com/blog/mark/PermaLink,guid,3670d081-0276...) that even .NET itself has a similar kind of GC (partially concurrent, only address fixup stops the world). Unfortunately I can't check how it compares to Singularity.
- 17y ago

ShabbyDoo 17y ago

Knowing little of Python, is it the case that the GIL was not initially deemed a bad scheme because multi-processor x86 machines were a rarity in the mid-90's (when Python was young)? I presume that Sun desired Java to be multiprocessor friendly because it had multi-cpu SPARC machines at the time of Java's design? And, for those using Python for high-volume production systems, how do you cope? Do you run one process (say a Zope instance) per core and load balance?

jart 17y ago
Keep in mind the GIL is only a problem when you a) use threads and b) perform CPU-bound operations (unless you need soft real time) When writing CPU-bound code, there are many libraries like NumPy that can do your heavy lifting in native code (thus avoiding the GIL) So it really isn't as bad as it seems. The reason this patch is good, is because it prevents performance from falling apart if one of your threads takes up too much CPU.
zepolen 17y ago
Yep, a process per core. The beauty is that since your app has to be written to be multiprocess capable then you can also extend past one computer when the need arises. You pay a memory tax overhead though.