5 ms·

The fact you are warned in advance that something bad is going to happen doesn't make it less bad. Just more manageable. The elephant in the room is that the c

The fact you are warned in advance that something bad is going to happen doesn't make it less bad. Just more manageable. The elephant in the room is that the core devs, after Python 2.7, have taken the very bad habit of breaking compat. For such a popular and core tech a Python, this is not responsible. It erodes trust. It also raises the cost of producing good software tremendously. There are so many moving parts in tech now a day you can't spend you time, as an open source dev, tracking everything that break all the time. Nor fixing it. We have limited resources, energy and spare time. And what if I miss it? It's human after all. A project of the scale of Python, that is so central, so essential, should not break on me more than once in a decade, at least willingly. The way the issue is dealt with currently is very short sighted. It's not about the short term cost of having some teams having to pay the price of a rewrite. It's about how bad it is for the Python community, ecosystem and project on the long run.

wirrbel 6y ago
> The elephant in the room is that the core devs, after Python 2.7, have taken the very bad habit of breaking compat. For such a popular and core tech a Python, this is not responsible. It erodes trust. It also raises the cost of producing good software tremendously. You know, the core devs cannot make it right for everyone. For each its-a-breaking-change-complaint there is a similar amount of we-want-shiny-new-things-and-my-teammates-want-to-move-on-to-shiny-new-language-X complaint. So its clear to me that Python in order to stay relevant needs to evolve, add features, break occasionally with new releases. There was once the idea of combining a couple of breaking changes in one release (I would call this the 'once-in-a-decade' breaking change you seem to find acceptable), that was Py2->Py3 and it was a shitshow, in part, because too many changes happened at once. So I am grateful that we now get the occasional breaking change, one change at a time. I think it is the right way if Python aims to be still relevant in 10 years from now.
- BiteCode_dev 6y ago
  See my answer to your sibling. It's not an "or" proposition. The 2 to 3 transition was hard because we did an abrupt cut. We could have had few transitioning versions that supported both worlds, then disabled the old ways but with a switch to put them back on if needed for a while, and allow for a more peaceful transition.
  - uranusjr 6y ago
    It’s very much an “or” situation from a practical standpoint. Maybe not exclusively or, but it’s unrealistic to say one does not significantly affect the other. Python core dev (CPython or otherwise) is already thin on resource for a long time, and you can’t just keep adding things without cleaning up old cruft. Parties most involved in development of the Python language (CPython devs, the Steering Council, etc.) value sustainability most, so it makes sense they decide to drop old stuff when they see no need of it. On the other side, parties that value stability more (e.g. libraries authors) need to actively lobby their goals to the core devs so they can assess the situation correctly and find the right balance between the two. You can’t just sit there and expect people to understand your needs; you have to tell them, which is what pydantic failed to do in this situation.
    - BiteCode_dev 6y ago
      There was an even smaller team in 2.X, and 2.X broke less stuff while still providing new features. So no. It's a matter of product vision here.
      - uranusjr 6y ago
        > 2.X broke less stuff while still providing new features. By putting off all the cleanups to Python 3. I think I like their current approach better.
        BiteCode_dev 6y ago
        Python 3 was necessary for cleaning. Having default new style classes, super() and so on are goodies, not necessities. It was necessary for the things that are were not easy to do without breaking compat such as unicode, fixing comparison bugs, etc. Now none of the stuff that broke compat in 3 after the initial release were things that were necessities. They were goodies. You don't need make from __future__ import annotations as default. You don't need to make async/await keyword.
- mixmastamyk 6y ago
  I don’t agree because it was the Unicode change that made it difficult, every thing else was relatively trivial. No further huge breaks are even being considered. So kind of a moot point anyway.
KptMarchewa 6y ago
Compatibility is great if you're building on solid, or at least good enough foundation. But that's not Python, the foundation is dubious, or even bad. Extensive C API which prevents advanced JIT efforts, tons of dead libraries in standard library. Also, even having solid foundation and big company backing, Java is also regularly deprecating and removing things. Full, unquestionable compatibility is good only if you're committee wizard in your tower and don't care about users beyond big corporate at all, aka C++ model. Oh, and one thing about Python: it's very understaffed. They have like 3 fully paid devs at all? For a project of this impact, it's really bad. Compare this to tens of thousands of man-years spend on JS.
- BiteCode_dev 6y ago
  I agree, and I'm not advocating to freeze the python language, nor don't understand how understaffed the python core devs are. But there are strategies that are in used or have been in used that we could draw inspiration from: - async/await didn't need to become keywords so fast. True/False were not for 15 years. - changing the default encoding is behind env var flag. - changing the way the c api is done is currently made doing the separate HPy project. It's not breaking anything. So there were instances where people intelligently decided to move the language forward, but stopped and remembered Linus wisdom of "we don't breaking userland". And they did improve things without breaking others. Guido used to be very strict on that too. But after so many years, we can't expect him to spend all his energy still policing every aspect of the project, especially after he stepped down as a BDFL.
- mixmastamyk 6y ago
  I looked at the psf budget last normal year, iirc it was like five million in income. They choose to spend most profits on outreach and teaching. That’s their decision but then they also complain about being under resourced, leading to compat breaks on minor versions. I don’t believe it is necessary, compatibility should come before most other activities.
  - williamsmj 6y ago
    1. 3.x to 3.x+1 is referred to as a major version change in Python. It's the same in Go, where 1.x to 1.x+1 is also a major version change (https://golang.org/doc/devel/release.html#policy https://golang.org/doc/devel/release.html#policy). Python does not use semantic versioning, which may be why your expecations do not align with reality. And they do not introduce breaking changes without two major version bumps (i.e. two years) of public warnings about their plans (in that sense they are more conservative than semver). 2. This breaking change has not happened yet. 3. If it does happen, it won't be because the PSF is under-resourced. It will be because a long formal process occurred in back in 2017 to ratify PEP 563, and that processes success started a timer on its incorporation into python 3.9 or later, consistent with Python's deprecation policy. You can argue that this was a bad decision, but "spending most profits on outreach and teaching" is not the reason it happened. 4. The reason resources are a problem is because the original link here is someone encouraging his users to harass the small (too small!) group of uncompensated volunteers who manage python releases, three weeks before the release, behavior for which he has since apologized.
williamsmj 6y ago
> A project of the scale of Python, that is so central, so essential, should not break on me more than once in a decade, at least willingly. They have been unambiguous about their breaking changes policy for twelve years: you get at least two major versions of warning (i.e. two to three years), possibly more. And old major versions are supported (bugfixes, security fixes, and occasional non-breaking backports) for five years. If that's not what you're looking for (which is totally fine!) then python is probably not a good fit for your use case.
- BiteCode_dev 6y ago
  Human being have the right to have an opinion and question the decision of the people designing the product they use. FOSS is not a religion.
  - williamsmj 6y ago
    I didn't say anything about your rights? You are free to continue to use python while criticizing it. p.s. Python is not a "product". Treating it like one is guaranteed to result in disappointment.
WhyNotHugo 6y ago
> The fact you are warned in advance that something bad is going to happen doesn't make it less bad. Just more manageable. Exactly, it was very manageable. They could have avoided writing a whole library (FastAPI) knowing it was going to break inevitably. They could have raised concerns during three years.
- BiteCode_dev 6y ago
  1 - FastAPI was created before the change was announced. 2 - You assume they knew about it. It's a lot of work to create FOSS, it's very easy to miss those things given the huge amount of information you have to check for compat and security. Hell, I certainly don't assume my main dependency, which is a rock solid language no changing the major version and used by millions is going to break my project. I don't read each line of each changelog of each version upgrade of each of my dependency. 3 - Even if they knew about it, they may never have realized it would have broken something before people told them so. 4 - They may even have noticed, 3 years ago, but forgot about it. It was 3 years ago! It's only human. It's FOSS.
  - williamsmj 6y ago
    > 1 - FastAPI was created before the change was announced. No, it wasn't. You might be thinking of pydantic. FastAPI's first commit was December 2018: https://github.com/tiangolo/fastapi/commit/406c092a3bf65bbd4405ce87611a7e0b9c0ae706 https://github.com/tiangolo/fastapi/commit/406c092a3bf65bbd4.... PEP 563 was the first item in the Python 3.7.0 release notes earlier that year.
    - BiteCode_dev 6y ago
      Indeed, my mistake.
  - aflag 6y ago
    It is surprising that they missed this change, though. Given how core type hints are for their projects and how broadly publicised the changes and developments in type hints were. I'd be surprised if there weren't any issues raised against their projects about those. It feels it'd be harder to miss it than it'd be to stumble upon it. Also, weird for them to forget that there is a change that will break the foundations of their libraries. You'd think they would have that in the back of their minds and, being stakeholders, even being part of the discussions. Like you said, it's FOSS and, most importantly, python is a community project.
dragonwriter 6y ago
> And what if I miss it? Irrelevant here, AIUI, since pydantic had bugs for this since 2018 but only raised issues with core 3 weeks before 3.10 feature freeze. They knew it was coming, they were able to test it because it waa available behind a __future__ import, they knew it didn’t work for them, and they sat on it until it waa virtually impossible to resolve well.

wirrbel 6y ago

> The elephant in the room is that the core devs, after Python 2.7, have taken the very bad habit of breaking compat. For such a popular and core tech a Python, this is not responsible. It erodes trust. It also raises the cost of producing good software tremendously. You know, the core devs cannot make it right for everyone. For each its-a-breaking-change-complaint there is a similar amount of we-want-shiny-new-things-and-my-teammates-want-to-move-on-to-shiny-new-language-X complaint. So its clear to me that Python in order to stay relevant needs to evolve, add features, break occasionally with new releases. There was once the idea of combining a couple of breaking changes in one release (I would call this the 'once-in-a-decade' breaking change you seem to find acceptable), that was Py2->Py3 and it was a shitshow, in part, because too many changes happened at once. So I am grateful that we now get the occasional breaking change, one change at a time. I think it is the right way if Python aims to be still relevant in 10 years from now.

BiteCode_dev 6y ago
See my answer to your sibling. It's not an "or" proposition. The 2 to 3 transition was hard because we did an abrupt cut. We could have had few transitioning versions that supported both worlds, then disabled the old ways but with a switch to put them back on if needed for a while, and allow for a more peaceful transition.
- uranusjr 6y ago
  It’s very much an “or” situation from a practical standpoint. Maybe not exclusively or, but it’s unrealistic to say one does not significantly affect the other. Python core dev (CPython or otherwise) is already thin on resource for a long time, and you can’t just keep adding things without cleaning up old cruft. Parties most involved in development of the Python language (CPython devs, the Steering Council, etc.) value sustainability most, so it makes sense they decide to drop old stuff when they see no need of it. On the other side, parties that value stability more (e.g. libraries authors) need to actively lobby their goals to the core devs so they can assess the situation correctly and find the right balance between the two. You can’t just sit there and expect people to understand your needs; you have to tell them, which is what pydantic failed to do in this situation.
  - BiteCode_dev 6y ago
    There was an even smaller team in 2.X, and 2.X broke less stuff while still providing new features. So no. It's a matter of product vision here.
    - uranusjr 6y ago
      > 2.X broke less stuff while still providing new features. By putting off all the cleanups to Python 3. I think I like their current approach better.
      - BiteCode_dev 6y ago
        Python 3 was necessary for cleaning. Having default new style classes, super() and so on are goodies, not necessities. It was necessary for the things that are were not easy to do without breaking compat such as unicode, fixing comparison bugs, etc. Now none of the stuff that broke compat in 3 after the initial release were things that were necessities. They were goodies. You don't need make from __future__ import annotations as default. You don't need to make async/await keyword.
mixmastamyk 6y ago
I don’t agree because it was the Unicode change that made it difficult, every thing else was relatively trivial. No further huge breaks are even being considered. So kind of a moot point anyway.

KptMarchewa 6y ago

Compatibility is great if you're building on solid, or at least good enough foundation. But that's not Python, the foundation is dubious, or even bad. Extensive C API which prevents advanced JIT efforts, tons of dead libraries in standard library. Also, even having solid foundation and big company backing, Java is also regularly deprecating and removing things. Full, unquestionable compatibility is good only if you're committee wizard in your tower and don't care about users beyond big corporate at all, aka C++ model. Oh, and one thing about Python: it's very understaffed. They have like 3 fully paid devs at all? For a project of this impact, it's really bad. Compare this to tens of thousands of man-years spend on JS.

BiteCode_dev 6y ago
I agree, and I'm not advocating to freeze the python language, nor don't understand how understaffed the python core devs are. But there are strategies that are in used or have been in used that we could draw inspiration from: - async/await didn't need to become keywords so fast. True/False were not for 15 years. - changing the default encoding is behind env var flag. - changing the way the c api is done is currently made doing the separate HPy project. It's not breaking anything. So there were instances where people intelligently decided to move the language forward, but stopped and remembered Linus wisdom of "we don't breaking userland". And they did improve things without breaking others. Guido used to be very strict on that too. But after so many years, we can't expect him to spend all his energy still policing every aspect of the project, especially after he stepped down as a BDFL.
mixmastamyk 6y ago
I looked at the psf budget last normal year, iirc it was like five million in income. They choose to spend most profits on outreach and teaching. That’s their decision but then they also complain about being under resourced, leading to compat breaks on minor versions. I don’t believe it is necessary, compatibility should come before most other activities.
- williamsmj 6y ago
  1. 3.x to 3.x+1 is referred to as a major version change in Python. It's the same in Go, where 1.x to 1.x+1 is also a major version change (https://golang.org/doc/devel/release.html#policy https://golang.org/doc/devel/release.html#policy). Python does not use semantic versioning, which may be why your expecations do not align with reality. And they do not introduce breaking changes without two major version bumps (i.e. two years) of public warnings about their plans (in that sense they are more conservative than semver). 2. This breaking change has not happened yet. 3. If it does happen, it won't be because the PSF is under-resourced. It will be because a long formal process occurred in back in 2017 to ratify PEP 563, and that processes success started a timer on its incorporation into python 3.9 or later, consistent with Python's deprecation policy. You can argue that this was a bad decision, but "spending most profits on outreach and teaching" is not the reason it happened. 4. The reason resources are a problem is because the original link here is someone encouraging his users to harass the small (too small!) group of uncompensated volunteers who manage python releases, three weeks before the release, behavior for which he has since apologized.

williamsmj 6y ago

> A project of the scale of Python, that is so central, so essential, should not break on me more than once in a decade, at least willingly. They have been unambiguous about their breaking changes policy for twelve years: you get at least two major versions of warning (i.e. two to three years), possibly more. And old major versions are supported (bugfixes, security fixes, and occasional non-breaking backports) for five years. If that's not what you're looking for (which is totally fine!) then python is probably not a good fit for your use case.

BiteCode_dev 6y ago
Human being have the right to have an opinion and question the decision of the people designing the product they use. FOSS is not a religion.
- williamsmj 6y ago
  I didn't say anything about your rights? You are free to continue to use python while criticizing it. p.s. Python is not a "product". Treating it like one is guaranteed to result in disappointment.

WhyNotHugo 6y ago

> The fact you are warned in advance that something bad is going to happen doesn't make it less bad. Just more manageable. Exactly, it was very manageable. They could have avoided writing a whole library (FastAPI) knowing it was going to break inevitably. They could have raised concerns during three years.

BiteCode_dev 6y ago
1 - FastAPI was created before the change was announced. 2 - You assume they knew about it. It's a lot of work to create FOSS, it's very easy to miss those things given the huge amount of information you have to check for compat and security. Hell, I certainly don't assume my main dependency, which is a rock solid language no changing the major version and used by millions is going to break my project. I don't read each line of each changelog of each version upgrade of each of my dependency. 3 - Even if they knew about it, they may never have realized it would have broken something before people told them so. 4 - They may even have noticed, 3 years ago, but forgot about it. It was 3 years ago! It's only human. It's FOSS.
- williamsmj 6y ago
  > 1 - FastAPI was created before the change was announced. No, it wasn't. You might be thinking of pydantic. FastAPI's first commit was December 2018: https://github.com/tiangolo/fastapi/commit/406c092a3bf65bbd4405ce87611a7e0b9c0ae706 https://github.com/tiangolo/fastapi/commit/406c092a3bf65bbd4.... PEP 563 was the first item in the Python 3.7.0 release notes earlier that year.
  - BiteCode_dev 6y ago
    Indeed, my mistake.
- aflag 6y ago
  It is surprising that they missed this change, though. Given how core type hints are for their projects and how broadly publicised the changes and developments in type hints were. I'd be surprised if there weren't any issues raised against their projects about those. It feels it'd be harder to miss it than it'd be to stumble upon it. Also, weird for them to forget that there is a change that will break the foundations of their libraries. You'd think they would have that in the back of their minds and, being stakeholders, even being part of the discussions. Like you said, it's FOSS and, most importantly, python is a community project.

dragonwriter 6y ago

> And what if I miss it? Irrelevant here, AIUI, since pydantic had bugs for this since 2018 but only raised issues with core 3 weeks before 3.10 feature freeze. They knew it was coming, they were able to test it because it waa available behind a __future__ import, they knew it didn’t work for them, and they sat on it until it waa virtually impossible to resolve well.