6 ms·

Why is it necessarily Cryptography’s fault that Python Ansible was taken by surprise? Or any of the other affected parties along the chain? That Cryptography

Why is it necessarily Cryptography’s fault that Python Ansible was taken by surprise? Or any of the other affected parties along the chain? That Cryptography was starting to include Rust in the project was announced on the mailing list by Gaynor last summer. And that email said exactly what future (at that time) release would require a Rust toolchain if the project was to be built from source. The reply to that by the Gentoo guy (who started the Github issue) was that package maintainers cannot follow every single mailing list of every dependency. That is debatable (e.g. maybe you should make an exception for security applications), but let’s take that as a given for now. In that case, what is Cryptography to do? Where should they announce such things in a way that orgs like Gentoo will see it? And also notice it and not just mentally gloss over it as some kind of “spam”? If the Gentoo guy didn’t see it half a year ago, would he have seen an announcement (or the reminders) if it was made five years ago?

klyrs 6y ago
Folks with an eye towards backwards compatibility typically don't implement breaking changes without a year of deprecation warnings emitted by the software. Contrast that to a notice posted in a sub-basement 6 months before instituting a breaking change. The shock and alarm do seem warranted IMO.
- Twirrim 6y ago
  > project mailing lists, Github issue tracker, project IRC channel, documentation (changelog, FAQ), and Twitter channels It's hardly a notice posted in a sub-basement 6 months before instituting a breaking change. They communicated via numerous mechanisms.
  - klyrs 6y ago
    It's a sub-basement from the perspective of folks for whom this is a second-order dependency. And it sounds like there are many more of them than there are folks that caught wind of this. Regardless. The common practice is a year of emitting warnings from the software, that the end-user will see. That's prominent and allows package maintainers enough time to work around the upcoming breakage. Six months notice on a mailing list is simply not prominent enough, and it's half the standard year which puts undue strain downstream.
    - deleted 6y ago
      [deleted]
- detaro 6y ago
  How do you do a deprecation warning for a new language being a build dependency? What does it look like? "your build environment is being deprecated"?
  - tom_mellior 6y ago
    In C you can implement a "your build environment is being deprecated" message like this: #ifndef I_UNDERSTAND_THAT_SOON_RUST_WILL_BE_MANDATORY #error "Starting with version xxx this package will need Rust to compile. Recompile with -DI_UNDERSTAND_THAT_SOON_RUST_WILL_BE_MANDATORY to acknowledge that you understand this warning." #endif Anyone building from source would get notified in a way that's impossible to miss but easy to turn off.
    - detaro 6y ago
      True, that's possible, but quite brutal and I'd expect that would merely have lead to shouting and people going wild over their builds breaking a year earlier.
    - marcinzm 6y ago
      And thousands upon thousands of automated CI jobs and docker container builds fail. You're basically causing massive developer stress to anyone who automatically compiles your package. Most would consider that a bad tradeoff to help a tiny fraction of your users who'd be impacted and also refuse to follow your mailing list.
      - naniwaduni 6y ago
        You're going to do that a few versions down the line anyway. Why not ahead of time? The only downstreams this affects more are the ones who decide to suppress the warning for now and ... put it off until you release the change that actually required breakage.
      - eyelidlessness 6y ago
        > And thousands upon thousands of automated CI jobs and docker container builds fail. You're basically causing massive developer stress to anyone who automatically compiles your package. This isn’t a major source of stress: you include migration instructions and even tooling to automate it. The thing that’s stressful is when your build fails with completely unexpected errors and no indication of what went wrong. Loudly announcing breaking changes is disruptive to some extent, but not doing so either means more disruption or nothing can ever change at all.
macksd 6y ago
I think making a change like this warrants a major version bump. It won't eliminate all the surprise for everyone, and I do have sympathy for the people who did go out of their way to talk about this on the mailing list and then surprised everyone. But it's common to pin yourself to a minor or maintenance release line to automatically pick up security fixes, etc. I expect breakage when changing the major (or even minor), and that's almost always a manual upgrade. And that's when I do read all the release notes, run tests, etc. before committing to the change.
- chrisoverzero 6y ago
  What is a “major version bump”? Before you answer, consider that the library doesn’t use semantic versioning. Before this all blew up, the versioning scheme was this[1]: > Given a version cryptography X.Y.Z, > - X.Y is a decimal number that is incremented for potentially-backwards-incompatible releases. > - - This increases like a standard decimal. In other words, 0.9 is the ninth release, and 1.0 is the tenth (not 0.10). The dividing decimal point can effectively be ignored. > - Z is an integer that is incremented for backward-compatible releases. The system has since changed, but it continues not to be semantic versioning. (It’s effectively the same, in fact, but protects against dependents who think it is semantic.) By that scheme, it was already a “major” (signifying potential backwards-incompatibility) release. [1]: https://cryptography.io/en/latest/api-stability.html#previous-scheme https://cryptography.io/en/latest/api-stability.html#previou...
  - AaronFriel 6y ago
    This reads to me like an argument for semantic versioning, because otherwise I need to internalize the rules of every package and know that some will break compatibility on the Y, some on the Z, ... Etc.
    - chrisoverzero 6y ago
      You know, I had that thought while writing it. I wouldn’t want to force any particular versioning scheme on any particular developer, but maybe the “SemVer façade” versioning scheme they switched to is the best compromise. It has defensive value, at least. Then again, PEP 440 has nothing to say about the semantics of versioning, only requiring: [N!]N(.N)*[{a|b|rc}N][.postN][.devN] PyPA themselves describe various expected versioning schemes, but listing Semantic as preferred[1]. If I squint, I can fit `cryptography`’s previous scheme into “Hybrid”. The biggest lesson I take from this is that if your version scheme isn’t SemVer, work hard to make it look obviously different from SemVer. [1]: https://packaging.python.org/guides/distributing-packages-using-setuptools/#choosing-a-versioning-scheme https://packaging.python.org/guides/distributing-packages-us...
rodgerd 6y ago
> The reply to that by the Gentoo guy (who started the Github issue) was that package maintainers cannot follow every single mailing list of every dependency. It seems reasonable that, if you are the packager for critical packages, that you follow critical dependencies? If the problem is that the distro is supporting so many things that the folks working on it can't keep up - well, that's precisely the author's point: stop pretending that you can support HPPA and MIPS or whatever as well as you can support x86_64. But you don't get to tell a million people that they have to have a less secure Python because 3 people have a toy in a closet they want treated as a first class citizen.
- FridgeSeal 6y ago
  The number of people in the original thread who didn’t appear to be version pinning and then getting upset that a package that they directly relied upon automatically upgraded is eye-watering.
- eyelidlessness 6y ago
  > stop pretending that you can support HPPA and MIPS or whatever as well as you can support x86_64 And then the corresponding uptightness will be “FOO_PROJECT is aligned with the Intel monopoly”, and just as many people will be unhappy. You can see this in many recent threads about Apple not providing free access to M1 documentation for alternative OSes they’re under no obligation to support.
  - rodgerd 6y ago
    That was, of course, a complaint about Linux back in the day. It turned out nobody cared enough to stop Linux development.
    - eyelidlessness 6y ago
      That’s pretty much verbatim the reply I had when people on here were prognosticating it for the M1, other than adding that they also promoted Linux virtualization in the announcement.

klyrs 6y ago

Folks with an eye towards backwards compatibility typically don't implement breaking changes without a year of deprecation warnings emitted by the software. Contrast that to a notice posted in a sub-basement 6 months before instituting a breaking change. The shock and alarm do seem warranted IMO.

Twirrim 6y ago
> project mailing lists, Github issue tracker, project IRC channel, documentation (changelog, FAQ), and Twitter channels It's hardly a notice posted in a sub-basement 6 months before instituting a breaking change. They communicated via numerous mechanisms.
- klyrs 6y ago
  It's a sub-basement from the perspective of folks for whom this is a second-order dependency. And it sounds like there are many more of them than there are folks that caught wind of this. Regardless. The common practice is a year of emitting warnings from the software, that the end-user will see. That's prominent and allows package maintainers enough time to work around the upcoming breakage. Six months notice on a mailing list is simply not prominent enough, and it's half the standard year which puts undue strain downstream.
  - deleted 6y ago
    [deleted]
detaro 6y ago
How do you do a deprecation warning for a new language being a build dependency? What does it look like? "your build environment is being deprecated"?
- tom_mellior 6y ago
  In C you can implement a "your build environment is being deprecated" message like this: #ifndef I_UNDERSTAND_THAT_SOON_RUST_WILL_BE_MANDATORY #error "Starting with version xxx this package will need Rust to compile. Recompile with -DI_UNDERSTAND_THAT_SOON_RUST_WILL_BE_MANDATORY to acknowledge that you understand this warning." #endif Anyone building from source would get notified in a way that's impossible to miss but easy to turn off.
  - detaro 6y ago
    True, that's possible, but quite brutal and I'd expect that would merely have lead to shouting and people going wild over their builds breaking a year earlier.
  - marcinzm 6y ago
    And thousands upon thousands of automated CI jobs and docker container builds fail. You're basically causing massive developer stress to anyone who automatically compiles your package. Most would consider that a bad tradeoff to help a tiny fraction of your users who'd be impacted and also refuse to follow your mailing list.
    - naniwaduni 6y ago
      You're going to do that a few versions down the line anyway. Why not ahead of time? The only downstreams this affects more are the ones who decide to suppress the warning for now and ... put it off until you release the change that actually required breakage.
    - eyelidlessness 6y ago
      > And thousands upon thousands of automated CI jobs and docker container builds fail. You're basically causing massive developer stress to anyone who automatically compiles your package. This isn’t a major source of stress: you include migration instructions and even tooling to automate it. The thing that’s stressful is when your build fails with completely unexpected errors and no indication of what went wrong. Loudly announcing breaking changes is disruptive to some extent, but not doing so either means more disruption or nothing can ever change at all.

macksd 6y ago

I think making a change like this warrants a major version bump. It won't eliminate all the surprise for everyone, and I do have sympathy for the people who did go out of their way to talk about this on the mailing list and then surprised everyone. But it's common to pin yourself to a minor or maintenance release line to automatically pick up security fixes, etc. I expect breakage when changing the major (or even minor), and that's almost always a manual upgrade. And that's when I do read all the release notes, run tests, etc. before committing to the change.

chrisoverzero 6y ago
What is a “major version bump”? Before you answer, consider that the library doesn’t use semantic versioning. Before this all blew up, the versioning scheme was this[1]: > Given a version cryptography X.Y.Z, > - X.Y is a decimal number that is incremented for potentially-backwards-incompatible releases. > - - This increases like a standard decimal. In other words, 0.9 is the ninth release, and 1.0 is the tenth (not 0.10). The dividing decimal point can effectively be ignored. > - Z is an integer that is incremented for backward-compatible releases. The system has since changed, but it continues not to be semantic versioning. (It’s effectively the same, in fact, but protects against dependents who think it is semantic.) By that scheme, it was already a “major” (signifying potential backwards-incompatibility) release. [1]: https://cryptography.io/en/latest/api-stability.html#previous-scheme https://cryptography.io/en/latest/api-stability.html#previou...
- AaronFriel 6y ago
  This reads to me like an argument for semantic versioning, because otherwise I need to internalize the rules of every package and know that some will break compatibility on the Y, some on the Z, ... Etc.
  - chrisoverzero 6y ago
    You know, I had that thought while writing it. I wouldn’t want to force any particular versioning scheme on any particular developer, but maybe the “SemVer façade” versioning scheme they switched to is the best compromise. It has defensive value, at least. Then again, PEP 440 has nothing to say about the semantics of versioning, only requiring: [N!]N(.N)*[{a|b|rc}N][.postN][.devN] PyPA themselves describe various expected versioning schemes, but listing Semantic as preferred[1]. If I squint, I can fit `cryptography`’s previous scheme into “Hybrid”. The biggest lesson I take from this is that if your version scheme isn’t SemVer, work hard to make it look obviously different from SemVer. [1]: https://packaging.python.org/guides/distributing-packages-using-setuptools/#choosing-a-versioning-scheme https://packaging.python.org/guides/distributing-packages-us...

rodgerd 6y ago

> The reply to that by the Gentoo guy (who started the Github issue) was that package maintainers cannot follow every single mailing list of every dependency. It seems reasonable that, if you are the packager for critical packages, that you follow critical dependencies? If the problem is that the distro is supporting so many things that the folks working on it can't keep up - well, that's precisely the author's point: stop pretending that you can support HPPA and MIPS or whatever as well as you can support x86_64. But you don't get to tell a million people that they have to have a less secure Python because 3 people have a toy in a closet they want treated as a first class citizen.

FridgeSeal 6y ago
The number of people in the original thread who didn’t appear to be version pinning and then getting upset that a package that they directly relied upon automatically upgraded is eye-watering.
eyelidlessness 6y ago
> stop pretending that you can support HPPA and MIPS or whatever as well as you can support x86_64 And then the corresponding uptightness will be “FOO_PROJECT is aligned with the Intel monopoly”, and just as many people will be unhappy. You can see this in many recent threads about Apple not providing free access to M1 documentation for alternative OSes they’re under no obligation to support.
- rodgerd 6y ago
  That was, of course, a complaint about Linux back in the day. It turned out nobody cared enough to stop Linux development.
  - eyelidlessness 6y ago
    That’s pretty much verbatim the reply I had when people on here were prognosticating it for the M1, other than adding that they also promoted Linux virtualization in the announcement.