4 ms·

I'm not following this. Does it mean the GIL is finally gone?

sp332 14y ago
Yes (although so far only in this branch of PyPy). It basically uses fine-grained locking instead of a global lock. I can't explain it simply so I'll just point to recent discussion: https://news.ycombinator.com/item?id=3941642 https://news.ycombinator.com/item?id=3941642 and https://news.ycombinator.com/item?id=3464484 https://news.ycombinator.com/item?id=3464484 and the beginning of this project https://news.ycombinator.com/item?id=2710235 https://news.ycombinator.com/item?id=2710235
axiak 14y ago
This is the first of what hopes to be many releases of pypy using STM for multithreaded safety. Up until this release Armin had a special API for creating and running transactions. Now he has wrapped python's threading module to use STM, see http://morepypy.blogspot.com/2012/05/stm-update-back-to-threads.html http://morepypy.blogspot.com/2012/05/stm-update-back-to-thre... for more information. Note that this is independent of CPython development.
masklinn 14y ago
That is the goal of the effort (see the pypy blog for the history of the effort, http://morepypy.blogspot.com http://morepypy.blogspot.com ). But as noted in TFA, it is currently very, very slow, so no the problem is not solved, creating a very very slow GIL-free interpreter has been possible for a long time, just... very very slow so not acceptable. Also it's not CPython. Technically, if you want the GIL to be gone on Python in general you can already use IronPyton or Jython which do not use a global interpreter lock.
Tobu 14y ago
It is a branch of PyPy (a fast and compliant Python 2.7 interpreter) that behaves as if the GIL was still there — statements are always run atomically, so are the new with atomic blocks — but will actually run multithreaded programs on multiple cores (as long as contention is low). It will also provide a transaction module (implemented with threads and atomic blocks) so that existing reactors can be ported, allowing coroutine-based programs to run on multiple cores too. This isn't faster at the moment, that will become the focus once this is stable. Current documentation: https://bitbucket.org/pypy/pypy/raw/stm-thread/pypy/doc/stm.rst https://bitbucket.org/pypy/pypy/raw/stm-thread/pypy/doc/stm.... The plan: http://pypy.org/tmdonate.html http://pypy.org/tmdonate.html
slurgfest 14y ago
I will get excited when it actually works out to be faster than interpreters with the GIL. I'm a bit disappointed that Python seems to have no interest in standardizing a lightweight-thread (greenlet, coroutine) interface that isn't an awful mess. yield is powerful but coroutine implementations with it are baroque and opaque. Java-style threading has a million gotchas and the interface is just a dog. I want to be able to teach people with a week of experience to write concurrent programs. (What about writing turtle programs with multiple turtles running at the same time?) If Python doesn't do it, then I want someone else to eat their lunch.
- jmtulloss 14y ago
  Stackless works fine for this kind of thing.
- Tobu 14y ago
  Which of the existing coroutine implementations have you tried, and what did you dislike about it?
  - jlgreco 14y ago
    In my humble opinion, the greatest problem with all of them is that there are so many. Most of them are rather fine I think, but it can just be a serious pain in the ass getting your head around all of them.
    - Tobu 14y ago
      I've only needed one (Twisted, which is actually slightly outside the space occupied by gevent et al). You can't stop people from building more, but feel free to bet on a popular project if you don't want to deal with choice.
      - jlgreco 14y ago
        In a world where I only work with my own code, that works. Unfortunately that is not my reality.

sp332 14y ago

Yes (although so far only in this branch of PyPy). It basically uses fine-grained locking instead of a global lock. I can't explain it simply so I'll just point to recent discussion: https://news.ycombinator.com/item?id=3941642 https://news.ycombinator.com/item?id=3941642 and https://news.ycombinator.com/item?id=3464484 https://news.ycombinator.com/item?id=3464484 and the beginning of this project https://news.ycombinator.com/item?id=2710235 https://news.ycombinator.com/item?id=2710235

axiak 14y ago

This is the first of what hopes to be many releases of pypy using STM for multithreaded safety. Up until this release Armin had a special API for creating and running transactions. Now he has wrapped python's threading module to use STM, see http://morepypy.blogspot.com/2012/05/stm-update-back-to-threads.html http://morepypy.blogspot.com/2012/05/stm-update-back-to-thre... for more information. Note that this is independent of CPython development.

masklinn 14y ago

That is the goal of the effort (see the pypy blog for the history of the effort, http://morepypy.blogspot.com http://morepypy.blogspot.com ). But as noted in TFA, it is currently very, very slow, so no the problem is not solved, creating a very very slow GIL-free interpreter has been possible for a long time, just... very very slow so not acceptable. Also it's not CPython. Technically, if you want the GIL to be gone on Python in general you can already use IronPyton or Jython which do not use a global interpreter lock.

Tobu 14y ago

It is a branch of PyPy (a fast and compliant Python 2.7 interpreter) that behaves as if the GIL was still there — statements are always run atomically, so are the new with atomic blocks — but will actually run multithreaded programs on multiple cores (as long as contention is low). It will also provide a transaction module (implemented with threads and atomic blocks) so that existing reactors can be ported, allowing coroutine-based programs to run on multiple cores too. This isn't faster at the moment, that will become the focus once this is stable. Current documentation: https://bitbucket.org/pypy/pypy/raw/stm-thread/pypy/doc/stm.rst https://bitbucket.org/pypy/pypy/raw/stm-thread/pypy/doc/stm.... The plan: http://pypy.org/tmdonate.html http://pypy.org/tmdonate.html

slurgfest 14y ago

I will get excited when it actually works out to be faster than interpreters with the GIL. I'm a bit disappointed that Python seems to have no interest in standardizing a lightweight-thread (greenlet, coroutine) interface that isn't an awful mess. yield is powerful but coroutine implementations with it are baroque and opaque. Java-style threading has a million gotchas and the interface is just a dog. I want to be able to teach people with a week of experience to write concurrent programs. (What about writing turtle programs with multiple turtles running at the same time?) If Python doesn't do it, then I want someone else to eat their lunch.

jmtulloss 14y ago
Stackless works fine for this kind of thing.
Tobu 14y ago
Which of the existing coroutine implementations have you tried, and what did you dislike about it?
- jlgreco 14y ago
  In my humble opinion, the greatest problem with all of them is that there are so many. Most of them are rather fine I think, but it can just be a serious pain in the ass getting your head around all of them.
  - Tobu 14y ago
    I've only needed one (Twisted, which is actually slightly outside the space occupied by gevent et al). You can't stop people from building more, but feel free to bet on a popular project if you don't want to deal with choice.
    - jlgreco 14y ago
      In a world where I only work with my own code, that works. Unfortunately that is not my reality.