4 ms·

It would help a lot if core Python libraries like urllib, NumPy, and others used SemVer. Removing a function is a breaking change. The root of this post is from

by oivey 10mo ago

It would help a lot if core Python libraries like urllib, NumPy, and others used SemVer. Removing a function is a breaking change. The root of this post is from urllib breaking something in a minor release. Get rid of the pseudo SemVer where you can deprecate functions and then break in minor releases. Maybe after that the rest of the community could hope their code will work long term if they don’t increment major versions.

graemep 10mo ago
Python itself is making breaking changes between minor versions, so I think its natural that the ecosystem will follow. I have not had any real problems yet myself, but its worrying.
- zahlman 10mo ago
  Python has never claimed to use semver, despite everyone associating the 3 in front with the 2->3 migration.
  - graemep 10mo ago
    That just means its always been a problem, instead of being a new problem. It does use major.minor.bugfix versioning, but without clarity about when to expect breaking changes. With the pace of 3.x releases it has become more of a problem.
kerkeslager 10mo ago
> Get rid of the pseudo SemVer where you can deprecate functions and then break in minor releases. I agree, but I think there's a bigger, cultural root cause here. This is the result of toxicity in the community. The Python 2 to 3 transition was done properly, with real SemVer, and real tools to aid the transition. For a few years about 25% of my work as a Python dev was transitioning projects from 2 to 3. No project took more than 2 weeks (less than 40 hours of actual work), and most took a day. And unfortunately, the Python team received a ton of hate (including threats) for it. As a natural reaction, it seems that they have a bit of PTSD, and since 3.0 they've been trying to trickle in the breaking changes instead of holding them for a 4.0 release. I don't blame them--it's definitely a worse experience for Python users, but it's probably a better experience for the people working on Python to have the hate and threats trickle in at a manageable rate. I think the solution is for people like us who understand that breaking changes are necessary to pile love on doing it with real SemVer, and try to balance out the hate with support and I had a client who in 2023 still was on 2.7.x, and when I found a few huge security holes in their code and told them I couldn't ethically continue to work on their product if they wouldn't upgrade Python, Django, and a few other packages, and they declined to renew my contract. As far as I know, they're still on 2.7.x. :shrug:
- oivey 10mo ago
  That’s probably true. I do think part of the anger is that a lot of the changes didn’t clearly improve the code around it. The obvious example is the change to print from a statement to a function. It makes the language a little cleaner, but it also breaks existing code for little practical benefit. More insidious was the breaks with strings and byte strings. It was a good and necessary change that also could lead to weird quiet breakages. At least for me, the real blocker was broad package support. Maintainers should think carefully about whether their change induces lots of downstream work for users. Users will be mad if they perceive that maintainers didn’t take that into account.
  - kerkeslager 10mo ago
    > The obvious example is the change to print from a statement to a function. It makes the language a little cleaner, but it also breaks existing code for little practical benefit. To be clear: I literally do not remember a single example of this breaking anything after running 2to3. There was some practical benefit (such as being able to use print in callbacks) and I don't think it breaking existing code is meaningful given how thoroughly automated the fix was. I do get the impression that a lot of the complaints are from people who did not do any upgrades themselves, or if they did, didn't use the automated tools. This is just such an irrelevant critique. This is a quintessential example of bikeshedding: the only reason you're bringing up `print` is because you understand the change, not because it's actually important in any way. > Maintainers should think carefully about whether their change induces lots of downstream work for users. Users will be mad if they perceive that maintainers didn’t take that into account. Sure, but users in this case are blatantly wrong. You can read the discussions on each of the breaking changes, they're public in the PEPs. The dev team is obviously very concerned with causing downstream work for users, and made every effort, very successfully, to avoid such work. If your impression is that maintainers didn't take into account downstream work for users, and your example is print, which frankly did not induce downstream work for users, you're the problem. You're being pretty disrespectful to people who put a lot of work into providing you a free interpreter.
    - 10mo ago
Hizonner 10mo ago
That just encourages people to keep using old, unmaintained, insecure versions of libraries. Then, when they're still on version 2.1.1, and your maintained version is 5.7.3, and somebody finds a major security bug in 2.1, they will come whining at you to release a 2.1.2. Code that is not being maintained is not usually suitable for use, period.
- patmorgan23 10mo ago
  And then you can offer them a support contract to produce an update for an out of support version
- oivey 10mo ago
  Library maintainers have no right to police how people use their open source code, period. Maintainers are also not obligated to backport security fixes. Anything else is effectively against the concept of open source. Notably, even this policing doesn’t fix the whining. The whining will just be about what TFA is whining about. You’re just moving the whining around. It also does nothing to actually force people to upgrade. Instead, people can just cap against the minor version you broke your package on. Instead of being user hostile, why not make the user’s job easier? Correctly following SemVer disincentivizes unnecessary breaking changes. That’s a very good thing for users and ultimately the health of the package. If you don’t want to backport security fixes, users are free to pay, do it themselves, or stop using the library.
zahlman 10mo ago
Numpy does use semver. Lots of people still complained about 2.0.
- oivey 10mo ago
  No, it does not: https://numpy.org/doc/stable/dev/depending_on_numpy.html https://numpy.org/doc/stable/dev/depending_on_numpy.html. > It is important to know that NumPy, like Python itself and most other well known scientific Python projects, does not use semantic versioning. Instead, backwards incompatible API changes require deprecation warnings for at least two releases.
  - zahlman 10mo ago
    Have they made breaking changes to the ABI in other versions?
    - oivey 10mo ago
      I’m not sure how that is relevant. Are you trying to suggest that NumPy in fact does follow SemVer despite their own documentation saying they don’t?

by oivey 10mo ago

graemep 10mo ago

Python itself is making breaking changes between minor versions, so I think its natural that the ecosystem will follow. I have not had any real problems yet myself, but its worrying.

zahlman 10mo ago
Python has never claimed to use semver, despite everyone associating the 3 in front with the 2->3 migration.
- graemep 10mo ago
  That just means its always been a problem, instead of being a new problem. It does use major.minor.bugfix versioning, but without clarity about when to expect breaking changes. With the pace of 3.x releases it has become more of a problem.

kerkeslager 10mo ago

> Get rid of the pseudo SemVer where you can deprecate functions and then break in minor releases. I agree, but I think there's a bigger, cultural root cause here. This is the result of toxicity in the community. The Python 2 to 3 transition was done properly, with real SemVer, and real tools to aid the transition. For a few years about 25% of my work as a Python dev was transitioning projects from 2 to 3. No project took more than 2 weeks (less than 40 hours of actual work), and most took a day. And unfortunately, the Python team received a ton of hate (including threats) for it. As a natural reaction, it seems that they have a bit of PTSD, and since 3.0 they've been trying to trickle in the breaking changes instead of holding them for a 4.0 release. I don't blame them--it's definitely a worse experience for Python users, but it's probably a better experience for the people working on Python to have the hate and threats trickle in at a manageable rate. I think the solution is for people like us who understand that breaking changes are necessary to pile love on doing it with real SemVer, and try to balance out the hate with support and I had a client who in 2023 still was on 2.7.x, and when I found a few huge security holes in their code and told them I couldn't ethically continue to work on their product if they wouldn't upgrade Python, Django, and a few other packages, and they declined to renew my contract. As far as I know, they're still on 2.7.x. :shrug:

oivey 10mo ago
That’s probably true. I do think part of the anger is that a lot of the changes didn’t clearly improve the code around it. The obvious example is the change to print from a statement to a function. It makes the language a little cleaner, but it also breaks existing code for little practical benefit. More insidious was the breaks with strings and byte strings. It was a good and necessary change that also could lead to weird quiet breakages. At least for me, the real blocker was broad package support. Maintainers should think carefully about whether their change induces lots of downstream work for users. Users will be mad if they perceive that maintainers didn’t take that into account.
- kerkeslager 10mo ago
  > The obvious example is the change to print from a statement to a function. It makes the language a little cleaner, but it also breaks existing code for little practical benefit. To be clear: I literally do not remember a single example of this breaking anything after running 2to3. There was some practical benefit (such as being able to use print in callbacks) and I don't think it breaking existing code is meaningful given how thoroughly automated the fix was. I do get the impression that a lot of the complaints are from people who did not do any upgrades themselves, or if they did, didn't use the automated tools. This is just such an irrelevant critique. This is a quintessential example of bikeshedding: the only reason you're bringing up `print` is because you understand the change, not because it's actually important in any way. > Maintainers should think carefully about whether their change induces lots of downstream work for users. Users will be mad if they perceive that maintainers didn’t take that into account. Sure, but users in this case are blatantly wrong. You can read the discussions on each of the breaking changes, they're public in the PEPs. The dev team is obviously very concerned with causing downstream work for users, and made every effort, very successfully, to avoid such work. If your impression is that maintainers didn't take into account downstream work for users, and your example is print, which frankly did not induce downstream work for users, you're the problem. You're being pretty disrespectful to people who put a lot of work into providing you a free interpreter.
  - 10mo ago

Hizonner 10mo ago

That just encourages people to keep using old, unmaintained, insecure versions of libraries. Then, when they're still on version 2.1.1, and your maintained version is 5.7.3, and somebody finds a major security bug in 2.1, they will come whining at you to release a 2.1.2. Code that is not being maintained is not usually suitable for use, period.

patmorgan23 10mo ago
And then you can offer them a support contract to produce an update for an out of support version
oivey 10mo ago
Library maintainers have no right to police how people use their open source code, period. Maintainers are also not obligated to backport security fixes. Anything else is effectively against the concept of open source. Notably, even this policing doesn’t fix the whining. The whining will just be about what TFA is whining about. You’re just moving the whining around. It also does nothing to actually force people to upgrade. Instead, people can just cap against the minor version you broke your package on. Instead of being user hostile, why not make the user’s job easier? Correctly following SemVer disincentivizes unnecessary breaking changes. That’s a very good thing for users and ultimately the health of the package. If you don’t want to backport security fixes, users are free to pay, do it themselves, or stop using the library.

zahlman 10mo ago

Numpy does use semver. Lots of people still complained about 2.0.

oivey 10mo ago
No, it does not: https://numpy.org/doc/stable/dev/depending_on_numpy.html https://numpy.org/doc/stable/dev/depending_on_numpy.html. > It is important to know that NumPy, like Python itself and most other well known scientific Python projects, does not use semantic versioning. Instead, backwards incompatible API changes require deprecation warnings for at least two releases.
- zahlman 10mo ago
  Have they made breaking changes to the ABI in other versions?
  - oivey 10mo ago
    I’m not sure how that is relevant. Are you trying to suggest that NumPy in fact does follow SemVer despite their own documentation saying they don’t?