10 ms·

> These changes are major enough that a fair number of existing Python libraries that work directly with Python’s internals (e.g., Cython) would need to be rewr

> These changes are major enough that a fair number of existing Python libraries that work directly with Python’s internals (e.g., Cython) would need to be rewritten. But the cadence of Python’s release schedule just means such breaking changes would need to be made in a major point release instead of a minor one. Maybe time to rethink? https://www.techrepublic.com/article/programming-languages-why-python-4-0-will-probably-never-arrive-according-to-its-creator/ https://www.techrepublic.com/article/programming-languages-w... If this is as promising as it sounds, it seems Python 4 now has its "thing" and is on the horizon. Or at least may become a serious thing to talk about

dangerbird2 5y ago
I'd wonder if it would be easier to introduce a totally new API along the lines of ruby's ractor API[1] that enables thread parallelism while keeping existing Thread behavior identical as with the GIL. Tons of python code relies on threaded code that is thread-safe under the GIL, but would completely blow up if the GIL was naively replaced. [1] https://docs.ruby-lang.org/en/master/doc/ractor_md.html https://docs.ruby-lang.org/en/master/doc/ractor_md.html
- arthurcolle 5y ago
  Ractors don't offer very good performance yet. Better to have it be awesome right off the bat
  - dangerbird2 5y ago
    Yeah, That's what I thought. I think the greatest barrier now is that most multithreaded python code right now is just barely thread-safe, even with the GIL. I occasionally have to remind colleagues that even though the GIL guarantees instructions are atomic, you need to use mutexes and other synchronization primitives to ensure there is no race condition between multiple instructions. I'd imagine this change would be an optional interpreter feature initially, since removing the GIL would break the vast majority of code out in the wild, and it would be much more difficult to create an automated conversion tool like they did with the syntactic changes between 2.7 and 3
gwking 5y ago
I began using python during the python3.0 betas, and I watched the 2 vs 3 saga from the (unusual?) perspective of a v3 hobbyist with no back-compat requirements. What struck me as most significant was the opportunistic breakage of things not related to the unicode transition. In the many years it took to win people over to v3, they could have marched over all the breaking changes a year at a time. Given that side-by-side installs of python3.x point versions are very functional, with or without venvs, this would have been much more palatable. Perhaps harder than it sounds though. I attempted a couple of 2to3 translations of open source libraries over the years, with varying degrees of success. Every time I found that most of the changes were easy, but debugging the broken bits was hard due to the sheer volume of source changes. If instead I could have done conversions where there was only a single major semantic change at a time, it would be so much easier to figure out what was going wrong at any given step. Furthermore, I imagine that a single-breaking-change mentality would lead to better documentation on how to transition for each version. For this reason, I have become rather suspicious of yearly release schedules. Swift is even more frustrating: the version changes are really just dictated by Apple's yearly PR calendar. Some big things get rushed out for WWDC before they are ready, and smaller fixes can get held back until the next year. I would much rather that the language teams just prioritize one thing at a time, release it when it is ready, and foster a community where staying up-to-date on the latest version is easy and desirable (a more complicated story for Apple than for Python I think, due to ABI, OS version, etc). From past discussions on HN I've gathered that there is such a thing as release fatigue, where developers get irritated when libraries release breaking changes too often. Nevertheless I often wonder if languages and libraries could improve faster by making more breaking changes, one at a time, with robust side-by-side installs to facilitate testing across versions. I wish side-by-side library versions were possible in Python, just to facilitate regression testing. Bringing this all back to the post, I sincerely hope that if Python 4 is a breaking change to the GIL, that it will be only that. I'm curious what others think about all this. Thoughts?
- kevin_thibedeau 5y ago
  > If instead I could have done conversions where there was only a single major semantic change at a time, That was the point of the "from __future__" imports. You could get most of the way toward Python 3 so that 2to3 would be easier to work with and the new semantics could be gradually baked into the code prior to migration. Python 3 had 25 years of cruft to clean up. They won't have to do that again.
klyrs 5y ago
Cython itself should be a relatively simple fix (relative to the difficulty that Cython devs are accustomed to). Libraries that use Cython in a pure way (that is, not fussing with refcounts in hand-written C code) should "just work" after Cython gets updated. It's the poor folk who have done straight C extensions without the benefit of Cython that I'm concerned about.
deleted 5y ago
[deleted]
lpapez 5y ago
A simple solution would be to introduce two new types: ConcurrentThread and ParallelThread. Alias the old Thread to the ConcurrentThread and keep the behaviour. No breaking changes, easy to explain the differrence. People who need it can use the new truly parallel version.

dangerbird2 5y ago

I'd wonder if it would be easier to introduce a totally new API along the lines of ruby's ractor API[1] that enables thread parallelism while keeping existing Thread behavior identical as with the GIL. Tons of python code relies on threaded code that is thread-safe under the GIL, but would completely blow up if the GIL was naively replaced. [1] https://docs.ruby-lang.org/en/master/doc/ractor_md.html https://docs.ruby-lang.org/en/master/doc/ractor_md.html

arthurcolle 5y ago
Ractors don't offer very good performance yet. Better to have it be awesome right off the bat
- dangerbird2 5y ago
  Yeah, That's what I thought. I think the greatest barrier now is that most multithreaded python code right now is just barely thread-safe, even with the GIL. I occasionally have to remind colleagues that even though the GIL guarantees instructions are atomic, you need to use mutexes and other synchronization primitives to ensure there is no race condition between multiple instructions. I'd imagine this change would be an optional interpreter feature initially, since removing the GIL would break the vast majority of code out in the wild, and it would be much more difficult to create an automated conversion tool like they did with the syntactic changes between 2.7 and 3

gwking 5y ago

I began using python during the python3.0 betas, and I watched the 2 vs 3 saga from the (unusual?) perspective of a v3 hobbyist with no back-compat requirements. What struck me as most significant was the opportunistic breakage of things not related to the unicode transition. In the many years it took to win people over to v3, they could have marched over all the breaking changes a year at a time. Given that side-by-side installs of python3.x point versions are very functional, with or without venvs, this would have been much more palatable. Perhaps harder than it sounds though. I attempted a couple of 2to3 translations of open source libraries over the years, with varying degrees of success. Every time I found that most of the changes were easy, but debugging the broken bits was hard due to the sheer volume of source changes. If instead I could have done conversions where there was only a single major semantic change at a time, it would be so much easier to figure out what was going wrong at any given step. Furthermore, I imagine that a single-breaking-change mentality would lead to better documentation on how to transition for each version. For this reason, I have become rather suspicious of yearly release schedules. Swift is even more frustrating: the version changes are really just dictated by Apple's yearly PR calendar. Some big things get rushed out for WWDC before they are ready, and smaller fixes can get held back until the next year. I would much rather that the language teams just prioritize one thing at a time, release it when it is ready, and foster a community where staying up-to-date on the latest version is easy and desirable (a more complicated story for Apple than for Python I think, due to ABI, OS version, etc). From past discussions on HN I've gathered that there is such a thing as release fatigue, where developers get irritated when libraries release breaking changes too often. Nevertheless I often wonder if languages and libraries could improve faster by making more breaking changes, one at a time, with robust side-by-side installs to facilitate testing across versions. I wish side-by-side library versions were possible in Python, just to facilitate regression testing. Bringing this all back to the post, I sincerely hope that if Python 4 is a breaking change to the GIL, that it will be only that. I'm curious what others think about all this. Thoughts?

kevin_thibedeau 5y ago
> If instead I could have done conversions where there was only a single major semantic change at a time, That was the point of the "from __future__" imports. You could get most of the way toward Python 3 so that 2to3 would be easier to work with and the new semantics could be gradually baked into the code prior to migration. Python 3 had 25 years of cruft to clean up. They won't have to do that again.

klyrs 5y ago

Cython itself should be a relatively simple fix (relative to the difficulty that Cython devs are accustomed to). Libraries that use Cython in a pure way (that is, not fussing with refcounts in hand-written C code) should "just work" after Cython gets updated. It's the poor folk who have done straight C extensions without the benefit of Cython that I'm concerned about.

deleted 5y ago

[deleted]

lpapez 5y ago

A simple solution would be to introduce two new types: ConcurrentThread and ParallelThread. Alias the old Thread to the ConcurrentThread and keep the behaviour. No breaking changes, easy to explain the differrence. People who need it can use the new truly parallel version.