3 ms·

>Something as "rah-rah" this, which goes so far as to basically call the GIL a source of unicorns and rainbows The GIL is definitely no source of unicorns and

>Something as "rah-rah" this, which goes so far as to basically call the GIL a source of unicorns and rainbows The GIL is definitely no source of unicorns and rainbows, but I think the case against it is usually overstated. There are numerous ways of sidestepping it (multiprocessing, pypy, C extensions, etc.), and it does serve a useful purpose.

w0utert 12y ago
>> The GIL is definitely no source of unicorns and rainbows, but I think the case against it is usually overstated. There are numerous ways of sidestepping it (multiprocessing, pypy, C extensions, etc.), and it does serve a useful purpose. That's definitely all true, but the article brushes it's implications for multithreaded Python off as if they don't exist, which is what one of the posters above me was probably referring to. Yes you can do multiprocessing, but if I have lots of shared, volatile state that's not what I want. Yes you can use PyPy, but if that doesn't work for some python framework I use, or if I can't control my deployment environment and it only has CPython I can't use PyPy. Obviously you can write C extensions for about any language, using that as an arguments why the GIL is not a problem for multithreaded Python is disingenuous. Maybe I don't know or don't like to program in C? Green threads are not a substitute for multi-threading either, as they still don't allow full utilization of multiple cores and are really only a solution for I/O bound processing. Of the 'numerous ways to sidestep the GIL' none are satisfactory if you have a CPU bound problem operating on shared state, that lends itself well to parallel execution, which are many. I wouldn't use Python to write a video codec or to do DNA sequence processing for example. It's not a fatal flaw for Python-as-a-language, but it's a flaw of CPython nonetheless, and not an insignificant one.
- crdoconnor 12y ago
  I wouldn't write a video codec in python either, nor would I write code that requires a huge amount of shared volatile state, but these things are not common coding tasks in general, particularly not in enterprisey-type programming. > Maybe I don't know or don't like to program in C? If you want to write a video codec or highly performant multithreaded code, you should probably give it a go. >Of the 'numerous ways to sidestep the GIL' none are satisfactory if you have a CPU bound problem operating on shared state, that lends itself well to parallel execution, which are many. You mean exactly like the matrix calculations done in the C extensions of numpy?
  - Lofkin 12y ago
    Also numba. It is a multithreaded JIT compiler for a subset of python code. Just need a decorator.
  - deleted 12y ago
    [deleted]
halflings 12y ago
PyPy also has a GIL. Their proposal to get rid of it (PyPy STM) is far from being ready for production and comes with a pretty important overhead: http://pypy.readthedocs.org/en/latest/stm.html#introduction http://pypy.readthedocs.org/en/latest/stm.html#introduction (but it's a really promising project that all Python developers should follow!)
jrochkind1 12y ago
You can deal with the GIL, it's not the death sentance some people think it is, I agree. But this guy actually DOES try to argue it's a _positive_ somehow! He really is claiming it's all unicorns and rainbows, he doesn't mention _any_ negative to it, he says it doesn't effect concurrency at all, look at generators and deferreds that proves it (not true, a misdirection), he even tries to say it's a positive, the "GIL is a performance optimization for most use cases of Python, and a development ease optimization for virtually all CPython code. The GIL makes it much easier to use OS threads" I think either this is an intellectually dishonest argument meant to confuse less-technical people, or the guy doesn't understand what he's talking about. And I still think python is probably fine for 'enterprise', although as always it depends on what you're doing with it!
- dragonwriter 12y ago
  The GIL absolutely makes it easier to use OS threads in terms of development ease, but the "performance optimization" piece I can't really see, except insofar as the development ease aspect means that developers are less likely to avoid using threads where they would provide a performance benefit.
  - slantedview 12y ago
    "The GIL absolutely makes it easier to use OS threads in terms of development ease" Yes, but pretending that you can have the ease of not worrying about thread safety while still having the performance benefits of multi-threading (real OS threads, mind you) is ridiculous. You can't have your cake and eat it too.
    - dragonwriter 12y ago
      You can have the ease with some of the performance benefits with the GIL (since the GIL is released in certain low level operations) -- because, for the things where you get the performance benefits, someone else has dealt with the pain of not having the ease.
  - gdwatson 12y ago

w0utert 12y ago

>> The GIL is definitely no source of unicorns and rainbows, but I think the case against it is usually overstated. There are numerous ways of sidestepping it (multiprocessing, pypy, C extensions, etc.), and it does serve a useful purpose. That's definitely all true, but the article brushes it's implications for multithreaded Python off as if they don't exist, which is what one of the posters above me was probably referring to. Yes you can do multiprocessing, but if I have lots of shared, volatile state that's not what I want. Yes you can use PyPy, but if that doesn't work for some python framework I use, or if I can't control my deployment environment and it only has CPython I can't use PyPy. Obviously you can write C extensions for about any language, using that as an arguments why the GIL is not a problem for multithreaded Python is disingenuous. Maybe I don't know or don't like to program in C? Green threads are not a substitute for multi-threading either, as they still don't allow full utilization of multiple cores and are really only a solution for I/O bound processing. Of the 'numerous ways to sidestep the GIL' none are satisfactory if you have a CPU bound problem operating on shared state, that lends itself well to parallel execution, which are many. I wouldn't use Python to write a video codec or to do DNA sequence processing for example. It's not a fatal flaw for Python-as-a-language, but it's a flaw of CPython nonetheless, and not an insignificant one.

crdoconnor 12y ago
I wouldn't write a video codec in python either, nor would I write code that requires a huge amount of shared volatile state, but these things are not common coding tasks in general, particularly not in enterprisey-type programming. > Maybe I don't know or don't like to program in C? If you want to write a video codec or highly performant multithreaded code, you should probably give it a go. >Of the 'numerous ways to sidestep the GIL' none are satisfactory if you have a CPU bound problem operating on shared state, that lends itself well to parallel execution, which are many. You mean exactly like the matrix calculations done in the C extensions of numpy?
- Lofkin 12y ago
  Also numba. It is a multithreaded JIT compiler for a subset of python code. Just need a decorator.
- deleted 12y ago
  [deleted]

halflings 12y ago

PyPy also has a GIL. Their proposal to get rid of it (PyPy STM) is far from being ready for production and comes with a pretty important overhead: http://pypy.readthedocs.org/en/latest/stm.html#introduction http://pypy.readthedocs.org/en/latest/stm.html#introduction (but it's a really promising project that all Python developers should follow!)

jrochkind1 12y ago

You can deal with the GIL, it's not the death sentance some people think it is, I agree. But this guy actually DOES try to argue it's a _positive_ somehow! He really is claiming it's all unicorns and rainbows, he doesn't mention _any_ negative to it, he says it doesn't effect concurrency at all, look at generators and deferreds that proves it (not true, a misdirection), he even tries to say it's a positive, the "GIL is a performance optimization for most use cases of Python, and a development ease optimization for virtually all CPython code. The GIL makes it much easier to use OS threads" I think either this is an intellectually dishonest argument meant to confuse less-technical people, or the guy doesn't understand what he's talking about. And I still think python is probably fine for 'enterprise', although as always it depends on what you're doing with it!

dragonwriter 12y ago
The GIL absolutely makes it easier to use OS threads in terms of development ease, but the "performance optimization" piece I can't really see, except insofar as the development ease aspect means that developers are less likely to avoid using threads where they would provide a performance benefit.
- slantedview 12y ago
  "The GIL absolutely makes it easier to use OS threads in terms of development ease" Yes, but pretending that you can have the ease of not worrying about thread safety while still having the performance benefits of multi-threading (real OS threads, mind you) is ridiculous. You can't have your cake and eat it too.
  - dragonwriter 12y ago
    You can have the ease with some of the performance benefits with the GIL (since the GIL is released in certain low level operations) -- because, for the things where you get the performance benefits, someone else has dealt with the pain of not having the ease.
- gdwatson 12y ago