4 ms·

The reality is self-evident from the PyPy home page: "PyPy is a fast, compliant alternative implementation of the Python language (2.7.2)." Why not 3.3? Oh, r

The reality is self-evident from the PyPy home page: "PyPy is a fast, compliant alternative implementation of the Python language (2.7.2)." Why not 3.3? Oh, right, because when the PyPy team started, Python 2.X was the Python language. Now it's not. This has been the case (PyPy has existed and supported 2.X code) since 3.0 was released. I.e., for years. As for PyPy STM, you trot that out like it's a real solution. It is not. It's 5 times slower and doesn't support Python 3.X. Which, of course, is the whole point. But, yes! Thank goodness I can type u'...' again! That will save me years of work! FAIL.

sho_hn 14y ago
> As for PyPy STM, you trot that out like it's a real solution. It is not. It's 5 times slower and doesn't support Python 3.X. This sort of comment doesn't really make me feel like you're qualified to comment on the subtleties of killing the GIL, to be honest. When it comes to removing the GIL, any form of "it's slower" is usually going to refer to single-threaded workloads, and avoiding to be slower for those is going to be nigh on impossible. The idea is that it's an investment to be recouped for various workloads due to the improved ability to exploit concurrency over a GIL-ful interpreter. And really, it's at the proof of concept stage right now, hardly optimized. And there's new CPU hardware with hardware support for transactional memory coming down the pipe.
- masklinn 14y ago
  > The idea is that it's an investment to be recouped for various workloads due to the improved ability to exploit concurrency over a GIL-ful interpreter. This idea has been rejected by the core team though, getting rid of the gil with a major performance hit is not considered acceptable when most existing Python code is not significantly affected by the GIL.
  - sho_hn 14y ago
    Aye, which I think is reasonable, especially in the context of the GIL-removal approaches that were on the table at the time and the effort needed to implement them in CPython. I think the PyPy folks want to get to a place where you can run PyPy in modes tuned for either single-threaded or concurrent workloads.
- dmbaggett 14y ago
  You are making excuses. It should require neither new hardware nor Herculean effort to realize acceptable multicore performance on CPU-bound tasks. And your badgering and down voting people here who disagree with you position doesn't make your argument any more persuasive.
  - sho_hn 14y ago
    Just to come to your aid and make your last sentence not a baseless accusation: Yes, I've been downvoting you. However, it's not because I'm disagreeing with your position, it's because I consider the way you conduct yourself in the debate, well, crappy, because: - First and foremost, your frequent use of ridicule and hyperbole is escalating the tone of the debate. - You prefer to move goal posts instead of addressing arguments (e.g. you still haven't actually gotten around to explaining how the 3.x transition has supposedly set back PyPy by years). - You're not beneath skewing things to fit your narrative, even if it doesn't really make any sense (e.g. pointing out that PyPy's STM tinkering doesn't yet support Python 3.x yet - yeah, duh, given that both are WIP branches that aren't merged into PyPy proper yet, and nevermind that the STM work doesn't really impact the same area of the codebase that adding Python 3.x support does, so there is no cause to anticipate problems with eventually integrating both). - You make bold claims and leave them unsubstantiated (e.g. "It should require neither new hardware nor Herculean effort to realize acceptable multicore performance on CPU-bound tasks." just now - if you've got it all figured out, where's your proposal to eliminate the GIL?). - Those bold claims even miss their mark in the context of the debate (I pointed out that "it's slower" refers to single-threaded workloads; when it comes to multicore performance the STM proof of concept is already showing some benefits). And so on and so on. Simply put, your posts are low quality, that's why I'm hitting that downward arrow.
    - dmbaggett 14y ago
      I still don't see that you are actually challenging the original claims, which were simply that splitting the language in two was a very bad idea, and that the split damaged the Python ecosystem. I stand by my claim that any reasonable general-purpose programming language should be able to run simple CPU-bound threaded tasks properly. I didn't claim arbitrary parallel programming should be easy; just that the GIL causes multi-threaded programs written in standard style in Python to not work properly in CPU-bound contexts, because the GIL prevents these programs from using all the cores available, and, furthermore, because even in single-core contexts it is prone to performance failure modes such as convoying and starvation that induce worse performance when there are more cores available. (cf. very extensive discussions in bugs.python.org on GIL problems, along with David Beazley's, Antoine Pitrou's, and others' attempts to address them). It is a standard, uncontroversial expectation of general purpose languages that offer threading that the threads run in parallel, using the available cores. I don't believe this needs substantiation any more than the claim that general purpose programming languages should run serial programs deterministically. I believe the better course for Python post 2.X would have been to fix the core problems with the language's reference implementation, while maintaining backward compatibility with the very large existing code base, rather than adding features of dubious value, doing nothing to address the GIL problem, and breaking compatibility with the vast majority of 2.X code. While this is certainly my opinion, and yours obviously differs, you can't badger away the limitations of the GIL, and you can't just browbeat people into accepting the 3.X series as a good thing. Look, I don't hate the language -- I use it every day and like it a lot. But don't pretend that there aren't real problems with Python when there clearly are -- the discussions on bugs.python.org are mainly driven by strong proponents of the language and are hardly politicized -- and don't pretend that Python 3 wasn't problematic; it's created years -- yes, years -- of problems and confusion with 2 vs 3 incompatibility, has expanded the footprint of the language and changed its semantics in substantive ways (cf. the many, many accepted PEPs post 2.X) that make implementing the language harder. Every language bump (whether 2.5 to 2.6 or 2.7 to 3.0) has meant additional work for the developers of alternative implementations; this is obvious on the face of it; you need only read these projects' blogs and look at the elapsed calendar time.

sho_hn 14y ago

> As for PyPy STM, you trot that out like it's a real solution. It is not. It's 5 times slower and doesn't support Python 3.X. This sort of comment doesn't really make me feel like you're qualified to comment on the subtleties of killing the GIL, to be honest. When it comes to removing the GIL, any form of "it's slower" is usually going to refer to single-threaded workloads, and avoiding to be slower for those is going to be nigh on impossible. The idea is that it's an investment to be recouped for various workloads due to the improved ability to exploit concurrency over a GIL-ful interpreter. And really, it's at the proof of concept stage right now, hardly optimized. And there's new CPU hardware with hardware support for transactional memory coming down the pipe.

masklinn 14y ago
> The idea is that it's an investment to be recouped for various workloads due to the improved ability to exploit concurrency over a GIL-ful interpreter. This idea has been rejected by the core team though, getting rid of the gil with a major performance hit is not considered acceptable when most existing Python code is not significantly affected by the GIL.
- sho_hn 14y ago
  Aye, which I think is reasonable, especially in the context of the GIL-removal approaches that were on the table at the time and the effort needed to implement them in CPython. I think the PyPy folks want to get to a place where you can run PyPy in modes tuned for either single-threaded or concurrent workloads.
dmbaggett 14y ago
You are making excuses. It should require neither new hardware nor Herculean effort to realize acceptable multicore performance on CPU-bound tasks. And your badgering and down voting people here who disagree with you position doesn't make your argument any more persuasive.
- sho_hn 14y ago
  Just to come to your aid and make your last sentence not a baseless accusation: Yes, I've been downvoting you. However, it's not because I'm disagreeing with your position, it's because I consider the way you conduct yourself in the debate, well, crappy, because: - First and foremost, your frequent use of ridicule and hyperbole is escalating the tone of the debate. - You prefer to move goal posts instead of addressing arguments (e.g. you still haven't actually gotten around to explaining how the 3.x transition has supposedly set back PyPy by years). - You're not beneath skewing things to fit your narrative, even if it doesn't really make any sense (e.g. pointing out that PyPy's STM tinkering doesn't yet support Python 3.x yet - yeah, duh, given that both are WIP branches that aren't merged into PyPy proper yet, and nevermind that the STM work doesn't really impact the same area of the codebase that adding Python 3.x support does, so there is no cause to anticipate problems with eventually integrating both). - You make bold claims and leave them unsubstantiated (e.g. "It should require neither new hardware nor Herculean effort to realize acceptable multicore performance on CPU-bound tasks." just now - if you've got it all figured out, where's your proposal to eliminate the GIL?). - Those bold claims even miss their mark in the context of the debate (I pointed out that "it's slower" refers to single-threaded workloads; when it comes to multicore performance the STM proof of concept is already showing some benefits). And so on and so on. Simply put, your posts are low quality, that's why I'm hitting that downward arrow.
  - dmbaggett 14y ago
    I still don't see that you are actually challenging the original claims, which were simply that splitting the language in two was a very bad idea, and that the split damaged the Python ecosystem. I stand by my claim that any reasonable general-purpose programming language should be able to run simple CPU-bound threaded tasks properly. I didn't claim arbitrary parallel programming should be easy; just that the GIL causes multi-threaded programs written in standard style in Python to not work properly in CPU-bound contexts, because the GIL prevents these programs from using all the cores available, and, furthermore, because even in single-core contexts it is prone to performance failure modes such as convoying and starvation that induce worse performance when there are more cores available. (cf. very extensive discussions in bugs.python.org on GIL problems, along with David Beazley's, Antoine Pitrou's, and others' attempts to address them). It is a standard, uncontroversial expectation of general purpose languages that offer threading that the threads run in parallel, using the available cores. I don't believe this needs substantiation any more than the claim that general purpose programming languages should run serial programs deterministically. I believe the better course for Python post 2.X would have been to fix the core problems with the language's reference implementation, while maintaining backward compatibility with the very large existing code base, rather than adding features of dubious value, doing nothing to address the GIL problem, and breaking compatibility with the vast majority of 2.X code. While this is certainly my opinion, and yours obviously differs, you can't badger away the limitations of the GIL, and you can't just browbeat people into accepting the 3.X series as a good thing. Look, I don't hate the language -- I use it every day and like it a lot. But don't pretend that there aren't real problems with Python when there clearly are -- the discussions on bugs.python.org are mainly driven by strong proponents of the language and are hardly politicized -- and don't pretend that Python 3 wasn't problematic; it's created years -- yes, years -- of problems and confusion with 2 vs 3 incompatibility, has expanded the footprint of the language and changed its semantics in substantive ways (cf. the many, many accepted PEPs post 2.X) that make implementing the language harder. Every language bump (whether 2.5 to 2.6 or 2.7 to 3.0) has meant additional work for the developers of alternative implementations; this is obvious on the face of it; you need only read these projects' blogs and look at the elapsed calendar time.