2 ms·

A big part of the problem with SemVer is that "breaking changes" is subjective and there's always the xkcd spacebar heating problem. The flip side of most brea

A big part of the problem with SemVer is that "breaking changes" is subjective and there's always the xkcd spacebar heating problem. The flip side of most breaking changes that are released is some software developer who either wasn't experienced enough to imagine that the change would break someone or else they're dealing with a very hard problem that they're trying to solve and the break change was collateral damage in some edge condition they didn't consider. Many people's breaking changes are someone else's bugfix, and often users fluidly wind up on either side of that condition on different bugs -- screaming about bugs that haven't been fixed for years and then screaming about other bugfixes that broke them.

lordgroff 5y ago
Here's a recent example from sqlalchemy. I'm using a type annotation. Update to new minor version. BAM, annotation no longer works, the classes have moved around and I'm staring at an ugly exception. To be clear: these were not private modules, methods, what have you. So we're not talking about the space bar issue here, it wasn't some undocumented or buggy behavior, things just stopped working as the API got shuffled around. At that point this was literally the third semver breakage in the same month, and yeah, this is unusual, but dependencies have stayed fixed since then, it's a headache and it makes me feel like everything is just built on a pile of shifting sand.
- lamontcg 5y ago
  That's probably because its open source and the Patreon contributions don't compare well to a FAANG staff SDE salary. SemVer is highly constraining and its a PITA to work with when you're getting a 6-figure salary. TANSTAAFL.
xavdid 5y ago
> A big part of the problem with SemVer is that "breaking changes" is subjective I don't think that's true. I've always thought a breaking change is defined in broad terms as "the public API surface, runtime, and output". Given that, each of these would be a SemVer major: * remove a publicly-exported function * public function changes order of arguments * drop support for an old version of the language * a function changes behavior significantly (the issue the OP wrote about) Relying on a specific major version, you should be able to assume none of the above will happen. The following would _not_ be a major version: * renamed/removed private versions * add extra, optional arguments to public function * add support for new version of language * the xkcd example about CPU usage In each of these cases, if you rely on non-guaranteed behavior, you should check every version upgrade closely- package maintainers don't make any assurances about those. I'm not sure how mongoid is developed, but this smells like a bug to me. The `or` behavior started considering a thing it didn't used to, possibly unintentionally. Unfortunately, released mistakes are the one thing SemVer doesn't cover. Testing sufficiently about your own assumptions (in this case, that the user upgraded matches the one found in the query) is the best way to CYA.
- lamontcg 5y ago
  That is a nice neat and tidy way of thinking about changes. The messy details are things like "oh the default options result in a massive security hole, they need to change immediately". There's also a messy medium ground where behavior changes and its just a question as to if anyone is dependent upon that behavior. It was wrong when it was implemented, never documented, but has a currently defined behavior that most people don't want -- unless they happened to find it and use it and work around it and they'll be broken by the switch. This is the realm of things like the undefined behaviors of the C standard which are compiler-dependent. Any sufficiently complicated API will have those all over the place. You can argue that APIs should never have those kinds of undefined standards, but in reality everyone does shoddy work and people accept PRs that they shouldn't (and the flip side of trying to be careful is people yelling about open PRs that have been stalled for months/years trying to sort out a sufficiently robust solution).
  - Xavdidtheshadow 5y ago
    > It was wrong when it was implemented, never documented, but has a currently defined behavior that most people don't want That strikes me as exactly the sort of guarantee that's _not_ made by semver. You can rely on that behavior, but then you have to check against every patch version, because that's exactly the sort of thing that could change out from under you. Maintainers shouldn't worry about breaking this case- they never promised to support it in the first place. I think a lot of those cases should be a major version change, and that's totally fine. We _should_ minimize the number of breaking changes to code people depend on, but sometimes the best fix to the (totally reasonable) situations you outlined above is to make a new major version. Users will have to make changes to upgrade (instead of just bumping the versions), but it won't change out from under them. Some docs about what's changing, why, and how to migrate your code go a long way here.
    - lamontcg 5y ago
      We do major version bumps on a yearly cycle and people get stuck on 8 year old versions and won't upgrade. Its easy to say that integers are free and limitless, but constant major version upgrades are something that people won't tolerate one way or another either. And you have to understand that the people making these decisions make dozens of them for every single release, and when they make category mistakes that is how you get breaking changes released early. Every single bug when looked at in isolation looks obvious what should have happened given hindsight bias. And I just don't think open source software, that isn't corporate backed in one way or another, should ever go 1.0 and it should stay 0.x.y and go ahead and break compat every 3-6 months or so as necessary. The cost of SemVer and trying to get it continuously correct on every single patch is a large tax.

lordgroff 5y ago

Here's a recent example from sqlalchemy. I'm using a type annotation. Update to new minor version. BAM, annotation no longer works, the classes have moved around and I'm staring at an ugly exception. To be clear: these were not private modules, methods, what have you. So we're not talking about the space bar issue here, it wasn't some undocumented or buggy behavior, things just stopped working as the API got shuffled around. At that point this was literally the third semver breakage in the same month, and yeah, this is unusual, but dependencies have stayed fixed since then, it's a headache and it makes me feel like everything is just built on a pile of shifting sand.

lamontcg 5y ago
That's probably because its open source and the Patreon contributions don't compare well to a FAANG staff SDE salary. SemVer is highly constraining and its a PITA to work with when you're getting a 6-figure salary. TANSTAAFL.

xavdid 5y ago

> A big part of the problem with SemVer is that "breaking changes" is subjective I don't think that's true. I've always thought a breaking change is defined in broad terms as "the public API surface, runtime, and output". Given that, each of these would be a SemVer major: * remove a publicly-exported function * public function changes order of arguments * drop support for an old version of the language * a function changes behavior significantly (the issue the OP wrote about) Relying on a specific major version, you should be able to assume none of the above will happen. The following would _not_ be a major version: * renamed/removed private versions * add extra, optional arguments to public function * add support for new version of language * the xkcd example about CPU usage In each of these cases, if you rely on non-guaranteed behavior, you should check every version upgrade closely- package maintainers don't make any assurances about those. I'm not sure how mongoid is developed, but this smells like a bug to me. The `or` behavior started considering a thing it didn't used to, possibly unintentionally. Unfortunately, released mistakes are the one thing SemVer doesn't cover. Testing sufficiently about your own assumptions (in this case, that the user upgraded matches the one found in the query) is the best way to CYA.

lamontcg 5y ago
That is a nice neat and tidy way of thinking about changes. The messy details are things like "oh the default options result in a massive security hole, they need to change immediately". There's also a messy medium ground where behavior changes and its just a question as to if anyone is dependent upon that behavior. It was wrong when it was implemented, never documented, but has a currently defined behavior that most people don't want -- unless they happened to find it and use it and work around it and they'll be broken by the switch. This is the realm of things like the undefined behaviors of the C standard which are compiler-dependent. Any sufficiently complicated API will have those all over the place. You can argue that APIs should never have those kinds of undefined standards, but in reality everyone does shoddy work and people accept PRs that they shouldn't (and the flip side of trying to be careful is people yelling about open PRs that have been stalled for months/years trying to sort out a sufficiently robust solution).
- Xavdidtheshadow 5y ago
  > It was wrong when it was implemented, never documented, but has a currently defined behavior that most people don't want That strikes me as exactly the sort of guarantee that's _not_ made by semver. You can rely on that behavior, but then you have to check against every patch version, because that's exactly the sort of thing that could change out from under you. Maintainers shouldn't worry about breaking this case- they never promised to support it in the first place. I think a lot of those cases should be a major version change, and that's totally fine. We _should_ minimize the number of breaking changes to code people depend on, but sometimes the best fix to the (totally reasonable) situations you outlined above is to make a new major version. Users will have to make changes to upgrade (instead of just bumping the versions), but it won't change out from under them. Some docs about what's changing, why, and how to migrate your code go a long way here.
  - lamontcg 5y ago
    We do major version bumps on a yearly cycle and people get stuck on 8 year old versions and won't upgrade. Its easy to say that integers are free and limitless, but constant major version upgrades are something that people won't tolerate one way or another either. And you have to understand that the people making these decisions make dozens of them for every single release, and when they make category mistakes that is how you get breaking changes released early. Every single bug when looked at in isolation looks obvious what should have happened given hindsight bias. And I just don't think open source software, that isn't corporate backed in one way or another, should ever go 1.0 and it should stay 0.x.y and go ahead and break compat every 3-6 months or so as necessary. The cost of SemVer and trying to get it continuously correct on every single patch is a large tax.