5 ms·

Hopefully not for long: http://doc.pypy.org/en/latest/stm.html http://doc.pypy.org/en/latest/stm.html

DasIch 11y ago
STM isn't magic. Transactions can and will silently fail and you lose your parallelism. You need additional tools to identify and debug failed transactions and you'll need to learn how to structure your code to avoid failed transactions. This might turn out to be quite difficult for non-trivial Python applications. STM is exciting but I think it's important to keep in mind that it's an experiment that while promising might still fail in practice.
- ZenoArrow 11y ago
  > "STM isn't magic." What difference does that make. STM will still enable the removal of the GIL. If an applications is coded that doesn't work with PyPy, that's a separate matter.
  - jerf 11y ago
    But first it has to actually work. So far, based on that page, it has suffered the same fate as all the other attempts to remove the GIL; mostly functional, but slower. And the numbers cited on that page put it on the rather slow side for single-threaded code for a GIL removal, too, which is what has always prevented previous efforts from getting merged in. I remain skeptical that STM is a good fit for this problem, as I was when I first heard about this, and after their work, alas, I have to say I see little evidence to make me change my mind on that. STM is something that probably needs to be baked in from day one, possibly all the way into the language itself, not put on the side of a complicated secondary implementation of Python.
    - ZenoArrow 11y ago
      > "So far, based on that page..." From the page... "THIS PAGE IS OLD, THE REST IS ABOUT STMGC-C7 WHEREAS THE CURRENT DEVELOPMENT WORK IS DONE ON STMGC-C8" This page is probably a better starting point for the current PyPy STM work: http://pypy.org/tmdonate2.html http://pypy.org/tmdonate2.html
      - jerf 11y ago
        I didn't see anything newer there to suggest they've made a lot of progress since that page, though. Part of the problem is that they're trying to wedge into a really tight window; for all its flaws, the GIL is not all that expensive (relative to how slow Python in general is, anyhow), and trying to do it on PyPy makes their window even smaller. It's a really hard thing to hit, and I'll believe it when I see it. I'm not saying it's impossible. Just, I'll believe it when I see it, and I don't recommend a high degree of confidence that it will occur. Nor do I recommend that you use it as part of your advocacy... it's nowhere near solid enough to be making promises about it to people outside the community.
        ZenoArrow 11y ago
        > "I'll believe it when I see it". You may be interested in this... "Leysin Winter Sprint (20-27th February 2016)" "STM (Software Transaction Memory), notably: try to come up with benchmarks, and measure them carefully in order to test and improve the conflict reporting tools, and more generally to figure out how practical it is in large projects to avoid conflicts" http://morepypy.blogspot.co.uk/2016/01/leysin-winter-sprint-20-27th-february.html http://morepypy.blogspot.co.uk/2016/01/leysin-winter-sprint-... > "Nor do I recommend that you use it as part of your advocacy." I'm an advocate for F#, but I'll encourage the progress of any open source language, strong competition is good for the field as a whole. The question isn't 'Why can't Python be fast?' but rather 'How fast can Python be?', the difference is a difference in attitude.

STM isn't magic. Transactions can and will silently fail and you lose your parallelism. You need additional tools to identify and debug failed transactions and you'll need to learn how to structure your code to avoid failed transactions. This might turn out to be quite difficult for non-trivial Python applications. STM is exciting but I think it's important to keep in mind that it's an experiment that while promising might still fail in practice.

ZenoArrow 11y ago
> "STM isn't magic." What difference does that make. STM will still enable the removal of the GIL. If an applications is coded that doesn't work with PyPy, that's a separate matter.
- jerf 11y ago
  But first it has to actually work. So far, based on that page, it has suffered the same fate as all the other attempts to remove the GIL; mostly functional, but slower. And the numbers cited on that page put it on the rather slow side for single-threaded code for a GIL removal, too, which is what has always prevented previous efforts from getting merged in. I remain skeptical that STM is a good fit for this problem, as I was when I first heard about this, and after their work, alas, I have to say I see little evidence to make me change my mind on that. STM is something that probably needs to be baked in from day one, possibly all the way into the language itself, not put on the side of a complicated secondary implementation of Python.
  - ZenoArrow 11y ago
    > "So far, based on that page..." From the page... "THIS PAGE IS OLD, THE REST IS ABOUT STMGC-C7 WHEREAS THE CURRENT DEVELOPMENT WORK IS DONE ON STMGC-C8" This page is probably a better starting point for the current PyPy STM work: http://pypy.org/tmdonate2.html http://pypy.org/tmdonate2.html
    - jerf 11y ago
      I didn't see anything newer there to suggest they've made a lot of progress since that page, though. Part of the problem is that they're trying to wedge into a really tight window; for all its flaws, the GIL is not all that expensive (relative to how slow Python in general is, anyhow), and trying to do it on PyPy makes their window even smaller. It's a really hard thing to hit, and I'll believe it when I see it. I'm not saying it's impossible. Just, I'll believe it when I see it, and I don't recommend a high degree of confidence that it will occur. Nor do I recommend that you use it as part of your advocacy... it's nowhere near solid enough to be making promises about it to people outside the community.
      - ZenoArrow 11y ago
        > "I'll believe it when I see it". You may be interested in this... "Leysin Winter Sprint (20-27th February 2016)" "STM (Software Transaction Memory), notably: try to come up with benchmarks, and measure them carefully in order to test and improve the conflict reporting tools, and more generally to figure out how practical it is in large projects to avoid conflicts" http://morepypy.blogspot.co.uk/2016/01/leysin-winter-sprint-20-27th-february.html http://morepypy.blogspot.co.uk/2016/01/leysin-winter-sprint-... > "Nor do I recommend that you use it as part of your advocacy." I'm an advocate for F#, but I'll encourage the progress of any open source language, strong competition is good for the field as a whole. The question isn't 'Why can't Python be fast?' but rather 'How fast can Python be?', the difference is a difference in attitude.