4 ms·

I'm the original author (and didn't post this) but I fail to see how the article is FUD. In fact, the article makes the exact same point as your fourth paragrap

by jknupp 13y ago

I'm the original author (and didn't post this) but I fail to see how the article is FUD. In fact, the article makes the exact same point as your fourth paragraph ("Novices would..."). Controlling access to shared data often does lead to issues for many programmers, and just throwing threading at a problem is rarely a good idea. I'm sorry you didn't find the article useful, but perhaps you're not in the article's target audience. I simply wanted to give Python novices some information and background about the GIL. That said, the fact that other implementations do not have a GIL isn't relevant to the article; it specifically refers to the cPython implementation. And your observation that multiprocessing avoids the GIL is explicitly mentioned in the article. To say "blocking on I/O gives up the GIL" is true in a very narrow sense but not very interesting. Import any third party package using C extensions and you now need to worry about how well the author manages the GIL.

markdown 13y ago
As a newcomer to Python (and programming in general), your article was very informative. Thank you for writing it.
goostavos 13y ago
Ignore the negativity train on Hacker News as of late. I thought it was a very well written and interesting article. I was once one of the very noobies that your article describes. In fact, I even made the same Stack Overflow post asking why threading made my program slow to a crawl. I did then (thanks to a friendly answer) learn about multiprocessing rather than threading, but I never actually got around to learning -- or even thinking about!-- what exactly the GIL is and why it makes threading terrible for CPU bound tasks. Point being, I enjoyed it.
- Luyt 13y ago
  "I never actually got around to learning -- or even thinking about!-- what exactly the GIL is" David Beazley does a good job on explaining the GIL. There are multiple videos of his GIL talks on YouTube. This one, for instance: http://www.youtube.com/watch?v=Obt-vMVdM8s http://www.youtube.com/watch?v=Obt-vMVdM8s He has a page about this at http://www.dabeaz.com/GIL/ http://www.dabeaz.com/GIL/
rdtsc 13y ago
There has been talk of criticism on HN about how comments are negative and that is true. So I hope this doesn't add to it and I am trying to honest here and say how I see things. First of all thanks for putting the effort into writing that. There is good information in the article describing how the GIL works and how threads work and things to watch out for. But unfortunately I have to agree with the gp as well, a part of it is FUD. The FUD is not in what was said (most what was said about the GIL is true) the FUD is in what was not said, and that is that threads are there for a reason, and they do provide good speedup in a large number of applications, namely those that are IO bound. I don't think that fact is too hard for novices to grasp so it should be me mentioned. The perception otherwise is that the creators have simply gone insane and decide to add threading to the language but you should never use it because it doesn't work (so why didn't they just remove it then?). Well it does work in large class of problems. Think about nodejs. Like it or not it has become popular recently. It doesn't by default support or take advantage of multiple cores yet it is often found to be performant enough to handle a decent number of concurrent clients connecting. Granted it doesn't pretend to have threads in the first place but it is an example of a modern, useful ecosystem that does not take advantage of all the cores. > To say "blocking on I/O gives up the GIL" is true in a very narrow sense Not it is not. It is true in a very general and wide sense. Try it out! Spawn 50 threads and try to download 50 different pages, you'll notice a speedup relative to doing it all sequentially in a while loop. > Import any third party package using C extensions and you now need to worry about how well the author manages the GIL. Also misrepresents the facts a bit. GIL actually is supposed to make it easier to write C extensions. If the C extension doesn't mess with the GIL it is straight forward to write because the GIL is there. If the GIL wasn't there calling C extensions and returning or having a callback in Python from C would be a pretty complicated affair. Now you can play with the GIL in C and release it to achieve parallel speedup, and I have done that, but that was in a few cases already over the years.

by jknupp 13y ago

markdown 13y ago

As a newcomer to Python (and programming in general), your article was very informative. Thank you for writing it.

goostavos 13y ago

Ignore the negativity train on Hacker News as of late. I thought it was a very well written and interesting article. I was once one of the very noobies that your article describes. In fact, I even made the same Stack Overflow post asking why threading made my program slow to a crawl. I did then (thanks to a friendly answer) learn about multiprocessing rather than threading, but I never actually got around to learning -- or even thinking about!-- what exactly the GIL is and why it makes threading terrible for CPU bound tasks. Point being, I enjoyed it.

Luyt 13y ago
"I never actually got around to learning -- or even thinking about!-- what exactly the GIL is" David Beazley does a good job on explaining the GIL. There are multiple videos of his GIL talks on YouTube. This one, for instance: http://www.youtube.com/watch?v=Obt-vMVdM8s http://www.youtube.com/watch?v=Obt-vMVdM8s He has a page about this at http://www.dabeaz.com/GIL/ http://www.dabeaz.com/GIL/

rdtsc 13y ago

There has been talk of criticism on HN about how comments are negative and that is true. So I hope this doesn't add to it and I am trying to honest here and say how I see things. First of all thanks for putting the effort into writing that. There is good information in the article describing how the GIL works and how threads work and things to watch out for. But unfortunately I have to agree with the gp as well, a part of it is FUD. The FUD is not in what was said (most what was said about the GIL is true) the FUD is in what was not said, and that is that threads are there for a reason, and they do provide good speedup in a large number of applications, namely those that are IO bound. I don't think that fact is too hard for novices to grasp so it should be me mentioned. The perception otherwise is that the creators have simply gone insane and decide to add threading to the language but you should never use it because it doesn't work (so why didn't they just remove it then?). Well it does work in large class of problems. Think about nodejs. Like it or not it has become popular recently. It doesn't by default support or take advantage of multiple cores yet it is often found to be performant enough to handle a decent number of concurrent clients connecting. Granted it doesn't pretend to have threads in the first place but it is an example of a modern, useful ecosystem that does not take advantage of all the cores. > To say "blocking on I/O gives up the GIL" is true in a very narrow sense Not it is not. It is true in a very general and wide sense. Try it out! Spawn 50 threads and try to download 50 different pages, you'll notice a speedup relative to doing it all sequentially in a while loop. > Import any third party package using C extensions and you now need to worry about how well the author manages the GIL. Also misrepresents the facts a bit. GIL actually is supposed to make it easier to write C extensions. If the C extension doesn't mess with the GIL it is straight forward to write because the GIL is there. If the GIL wasn't there calling C extensions and returning or having a callback in Python from C would be a pretty complicated affair. Now you can play with the GIL in C and release it to achieve parallel speedup, and I have done that, but that was in a few cases already over the years.