30 ms·

Yeah, semver is great, except nobody follows it. The amount of times I have updated minor or even patch releases of packages with things breaking--I have pretty

Yeah, semver is great, except nobody follows it. The amount of times I have updated minor or even patch releases of packages with things breaking--I have pretty much determined everything in a project stays the same unless there's some pressing need. The worst part is that breakage is often far from obvious, and spending good time wondering why everything breaks when it shouldn't is really really infuriating and makes me feel like the entire world of software is hopelessly broken.

lamontcg 5y ago
A big part of the problem with SemVer is that "breaking changes" is subjective and there's always the xkcd spacebar heating problem. The flip side of most breaking changes that are released is some software developer who either wasn't experienced enough to imagine that the change would break someone or else they're dealing with a very hard problem that they're trying to solve and the break change was collateral damage in some edge condition they didn't consider. Many people's breaking changes are someone else's bugfix, and often users fluidly wind up on either side of that condition on different bugs -- screaming about bugs that haven't been fixed for years and then screaming about other bugfixes that broke them.
- lordgroff 5y ago
  Here's a recent example from sqlalchemy. I'm using a type annotation. Update to new minor version. BAM, annotation no longer works, the classes have moved around and I'm staring at an ugly exception. To be clear: these were not private modules, methods, what have you. So we're not talking about the space bar issue here, it wasn't some undocumented or buggy behavior, things just stopped working as the API got shuffled around. At that point this was literally the third semver breakage in the same month, and yeah, this is unusual, but dependencies have stayed fixed since then, it's a headache and it makes me feel like everything is just built on a pile of shifting sand.
  - lamontcg 5y ago
    That's probably because its open source and the Patreon contributions don't compare well to a FAANG staff SDE salary. SemVer is highly constraining and its a PITA to work with when you're getting a 6-figure salary. TANSTAAFL.
- xavdid 5y ago
  > A big part of the problem with SemVer is that "breaking changes" is subjective I don't think that's true. I've always thought a breaking change is defined in broad terms as "the public API surface, runtime, and output". Given that, each of these would be a SemVer major: * remove a publicly-exported function * public function changes order of arguments * drop support for an old version of the language * a function changes behavior significantly (the issue the OP wrote about) Relying on a specific major version, you should be able to assume none of the above will happen. The following would _not_ be a major version: * renamed/removed private versions * add extra, optional arguments to public function * add support for new version of language * the xkcd example about CPU usage In each of these cases, if you rely on non-guaranteed behavior, you should check every version upgrade closely- package maintainers don't make any assurances about those. I'm not sure how mongoid is developed, but this smells like a bug to me. The `or` behavior started considering a thing it didn't used to, possibly unintentionally. Unfortunately, released mistakes are the one thing SemVer doesn't cover. Testing sufficiently about your own assumptions (in this case, that the user upgraded matches the one found in the query) is the best way to CYA.
  - lamontcg 5y ago
    That is a nice neat and tidy way of thinking about changes. The messy details are things like "oh the default options result in a massive security hole, they need to change immediately". There's also a messy medium ground where behavior changes and its just a question as to if anyone is dependent upon that behavior. It was wrong when it was implemented, never documented, but has a currently defined behavior that most people don't want -- unless they happened to find it and use it and work around it and they'll be broken by the switch. This is the realm of things like the undefined behaviors of the C standard which are compiler-dependent. Any sufficiently complicated API will have those all over the place. You can argue that APIs should never have those kinds of undefined standards, but in reality everyone does shoddy work and people accept PRs that they shouldn't (and the flip side of trying to be careful is people yelling about open PRs that have been stalled for months/years trying to sort out a sufficiently robust solution).
    - Xavdidtheshadow 5y ago
tshaddox 5y ago
When you say “breaking” in that comment are you talking about backwards compatible API changes, or straight up bugs? It kinda sounds like you’re talking about the latter, which of course isn’t within the scope of semver.
- yebyen 5y ago
  That's actually something that is in the author's purview to decide. (Edit: whoops, I misread you. I thought you were saying APIs are not in scope for SemVer. Now I think I realized you meant, you can have bugs that break things and that's not related to semver.) Is the semver contract with respect to the end user activities or the API? For Flux and Helm for example, the APIs are supported with semver code (and the interfaces use Kubernetes API versioning) In Helm, the CLI and user activities are explicitly not part of the semver contract as I understand it, only the API (at least with respect to experimental features like OCI charts for now?)
sofixa 5y ago
> I have pretty much determined everything in a project stays the same unless there's some pressing need Unless you have a way to stay up to date with the security of each and every dependency you have, that seems dangerous. There are static security scanners out there which can parse your dependency tree and alert you, but often they have a high noise ratio ( e.g. npm audit), so that only helps so much.
- lordgroff 5y ago
  Application is airgapped, so it's not as dangerous as it sounds.
- chousuke 5y ago
  The "easy" way to do this is to write your application using whatever is included in a stable Linux distribution. You get free, trivially applied security updates and a much lower chance of breakage. Optionally you can pay for support to have them fix problems in dependencies. Granted, It'll be difficult with languages like Javascript where many popular dependencies aren't likely to be packaged.

lamontcg 5y ago

A big part of the problem with SemVer is that "breaking changes" is subjective and there's always the xkcd spacebar heating problem. The flip side of most breaking changes that are released is some software developer who either wasn't experienced enough to imagine that the change would break someone or else they're dealing with a very hard problem that they're trying to solve and the break change was collateral damage in some edge condition they didn't consider. Many people's breaking changes are someone else's bugfix, and often users fluidly wind up on either side of that condition on different bugs -- screaming about bugs that haven't been fixed for years and then screaming about other bugfixes that broke them.

lordgroff 5y ago
Here's a recent example from sqlalchemy. I'm using a type annotation. Update to new minor version. BAM, annotation no longer works, the classes have moved around and I'm staring at an ugly exception. To be clear: these were not private modules, methods, what have you. So we're not talking about the space bar issue here, it wasn't some undocumented or buggy behavior, things just stopped working as the API got shuffled around. At that point this was literally the third semver breakage in the same month, and yeah, this is unusual, but dependencies have stayed fixed since then, it's a headache and it makes me feel like everything is just built on a pile of shifting sand.
- lamontcg 5y ago
  That's probably because its open source and the Patreon contributions don't compare well to a FAANG staff SDE salary. SemVer is highly constraining and its a PITA to work with when you're getting a 6-figure salary. TANSTAAFL.
xavdid 5y ago
> A big part of the problem with SemVer is that "breaking changes" is subjective I don't think that's true. I've always thought a breaking change is defined in broad terms as "the public API surface, runtime, and output". Given that, each of these would be a SemVer major: * remove a publicly-exported function * public function changes order of arguments * drop support for an old version of the language * a function changes behavior significantly (the issue the OP wrote about) Relying on a specific major version, you should be able to assume none of the above will happen. The following would _not_ be a major version: * renamed/removed private versions * add extra, optional arguments to public function * add support for new version of language * the xkcd example about CPU usage In each of these cases, if you rely on non-guaranteed behavior, you should check every version upgrade closely- package maintainers don't make any assurances about those. I'm not sure how mongoid is developed, but this smells like a bug to me. The `or` behavior started considering a thing it didn't used to, possibly unintentionally. Unfortunately, released mistakes are the one thing SemVer doesn't cover. Testing sufficiently about your own assumptions (in this case, that the user upgraded matches the one found in the query) is the best way to CYA.
- lamontcg 5y ago
  That is a nice neat and tidy way of thinking about changes. The messy details are things like "oh the default options result in a massive security hole, they need to change immediately". There's also a messy medium ground where behavior changes and its just a question as to if anyone is dependent upon that behavior. It was wrong when it was implemented, never documented, but has a currently defined behavior that most people don't want -- unless they happened to find it and use it and work around it and they'll be broken by the switch. This is the realm of things like the undefined behaviors of the C standard which are compiler-dependent. Any sufficiently complicated API will have those all over the place. You can argue that APIs should never have those kinds of undefined standards, but in reality everyone does shoddy work and people accept PRs that they shouldn't (and the flip side of trying to be careful is people yelling about open PRs that have been stalled for months/years trying to sort out a sufficiently robust solution).
  - Xavdidtheshadow 5y ago

tshaddox 5y ago

When you say “breaking” in that comment are you talking about backwards compatible API changes, or straight up bugs? It kinda sounds like you’re talking about the latter, which of course isn’t within the scope of semver.

yebyen 5y ago
That's actually something that is in the author's purview to decide. (Edit: whoops, I misread you. I thought you were saying APIs are not in scope for SemVer. Now I think I realized you meant, you can have bugs that break things and that's not related to semver.) Is the semver contract with respect to the end user activities or the API? For Flux and Helm for example, the APIs are supported with semver code (and the interfaces use Kubernetes API versioning) In Helm, the CLI and user activities are explicitly not part of the semver contract as I understand it, only the API (at least with respect to experimental features like OCI charts for now?)

sofixa 5y ago

> I have pretty much determined everything in a project stays the same unless there's some pressing need Unless you have a way to stay up to date with the security of each and every dependency you have, that seems dangerous. There are static security scanners out there which can parse your dependency tree and alert you, but often they have a high noise ratio ( e.g. npm audit), so that only helps so much.

lordgroff 5y ago
Application is airgapped, so it's not as dangerous as it sounds.
chousuke 5y ago
The "easy" way to do this is to write your application using whatever is included in a stable Linux distribution. You get free, trivially applied security updates and a much lower chance of breakage. Optionally you can pay for support to have them fix problems in dependencies. Granted, It'll be difficult with languages like Javascript where many popular dependencies aren't likely to be packaged.