4 ms·

No, that is not the problem, people are not just holding it wrong. The problem is a mismatch between the Go team's perception of semver usage (or perhaps how t

No, that is not the problem, people are not just holding it wrong. The problem is a mismatch between the Go team's perception of semver usage (or perhaps how they think it should be used) and its real-world usage. Some examples: Major versions are used to communicate more than breaking changes. Project identity doesn't change if a project changes, it is not a new package. Hardly any projects use strict semver in the real world, and other package managers allow a more flexible, looser semver. Just what is a breaking change anyway? Real-world packages make small security changes which break compatibility, sometimes minor changes do too, and that's ok if breakage is very limited in scope. In the current go modules, new major versions are not easily discoverable or announced, and new importers will need to consult docs or know about go get @latest to find the latest version. This could be fixed with tooling if the problem is recognised. There are possible solutions if they want to keep /v2 paths, but we shouldn't pretend there are no problems caused by this change. It makes it easier to produce breaking changes and harder to version increment just for new features (as most projects from linux to rails do). Personally I think it condones breaking changes and will lead to more breaking changes, not fewer - if you're forced to change import paths at higher versions why not just delete that api you don't like? Within Google say there are significant downsides to this and pressure to fix your own breakage, outside, not so much, including in packages people are required to use, like api client packages for popular services. Significant breaking changes introduce real pain for consumers and in an open ecosystem should be avoided at all costs.

kybernetikos 6y ago
> Hardly any projects use strict semver in the real world. Most other package managers don't enforce it. I think strict semver is reasonably possible for things that are libraries rather than products in their own right. But you're right, people want major versions to indicate something significant and new (and occasionally major versions have a contractual significance as well - requiring customers to pay an upgrade fee), rather than small breaking changes. It's even sometimes possible to create a new library that has huge changes in the intended way you use it, its conceptual model and masses of new functionality, but it maintains backwards compatibility - it's perfectly understandable that the package owner wants to indicate this with a major version bump. The other issue I've come across is that what constitutes a breaking change is much more subjective than many people realise. Any change is a breaking change if someone is reading in your library and tweaking bytes in specific locations. Of course, for most libraries they shouldn't be doing that, but that means that if someone is using your library wrong, then you don't worry about breaking them. But the question of if someone is using your library wrong is pretty subjective. At an extreme, that could be considered to be relying on any behaviour not explicitly documented as intended. Ultimately, it comes down to the judgement of the package maintainer, and that doesn't always match up with the judgement of the user. Having version numbers that mean something to machines is very useful when it works though. Perhaps we should just separate those from human-targetted versions rather than go full sentimental versioning: http://sentimentalversioning.org/ http://sentimentalversioning.org/ Maybe something like <human version>.<machine readable api version>.<unique incrementing build number> could work.
- grey-area 6y ago
  I think because of the many problems listed and the significant one you add (Just what is a breaking change? It turns out pretty much any change can be breaking for some user), the tooling needs to recognise that strict semver is unsound and a looser version is required. This isn't just a question of pesky humans with their marketing versions vs unerring machines - there are complexities in the problem space which shouldn't be ignored. Different producers/projects have different expectations for strictness in semver, and that's ok, they can use it differently in a negotiation with their consumers. Also different consumers have different requirements, and most tools provide a way for them to specify that on a per-import basis (upgrade only minor of these pls unattended, upgrade nothing on this one, only breaking changes on that one). In short, loose semver is a feature, not a bug.
jhoechtl 6y ago
I think you conflate marketing with technical change. You would like to announce a new major version with fanfare, bumping a version number. A programmer wants to be sure that his piece of software continues working if there has only been a minor version bump of a library he is using. Did you read the reasoning of Russ Cox in respect to vgo, versioning and the module system in general? It reads very plausible and understandable. Just because so many peopele interpret semver in their sense instead what it really means shouldn't prevent the Go core team to try to do it right.
- grey-area 6y ago
  I think you conflate marketing with technical change. There are multiple reasons given above why strict semantic versioning is not used in real-world software, that is a reductive summary of one of them. Yes, I've read those articles a while back when they were written. In a very real sense, a consumer can never be sure that software keeps running properly if a library changes in any way without extensive checks. Real-world loose semantic versioning is a promise, not a proof, and it works better that way. I don't personally think the current go proposal is terrible or sucks, but I do think it could be improved, and it'll be improved by listening to how people use versions.
sacado2 6y ago
> Just what is a breaking change anyway? It's just an API kind of thing. If a function / constant / var / type was available and is not anymore, it's a breaking change. It's an objective measure, there is nothing left to subjectivity. Now, of course a bug fix is going to "break" things if your code relied on the bogus behavior. An updated algorithm can also break things (I maintain a SAT solver, whenever I update the underlying algorithm, even if it's way faster on average, in some limited cases it can make one very specific problem way slower to solve, which can have bad consequences). But as it has no consequence on the API part of things, it's not a major change in semver's meaning.
- grey-area 6y ago
  There are far more extensive possibilities than what you export - almost every change can be breaking change, often without the producer knowing about it. This is why nobody uses strict semver in practice. https://www.bennadel.com/blog/3501-when-is-a-change-a-breaking-change-for-an-api.htm https://www.bennadel.com/blog/3501-when-is-a-change-a-breaki...
  - brabel 6y ago
    The parent commenter is correct in the sense that API changes can be, and should be, automatically checked for server.. you're just saying that doing that still does not guarantee there's no breaking changes because, besides API changes, there's also semantic changes, and those are difficult, perhaps impossible, to automatically check. Or are they? Check this out: https://www.unisonweb.org/docs/tour#-the-big-technical-idea https://www.unisonweb.org/docs/tour#-the-big-technical-idea
    - grey-area 6y ago
      Thanks for the link, a merkle tree for code sounds interesting.
  - sacado2 6y ago
    Are we talking about semver in general or semver the way it is (supposed to be) used in go mod?
Joker_vD 6y ago
Okay, here's a real world example. Three months ago, stretchr/testify was version 1.3 and supported Go 1.10. Yesterday, we tried to build some project, and suddenly tests don't run because, you see, now testify is version 1.6 and they support only Go 1.13 and up. That's a bloody breaking change, okay? And it happens all the time with Go packages: they're constantly in the process of dropping support for older Go versions without bumping major versions.
- dullgiulio 6y ago
  And is Go so hard to upgrade for your application? Go the language is really best in class when it comes to keeping backwards compatibility.
  - Spivak 6y ago
    But this is still a breaking change though. If I build my app today I should be able to build it again tomorrow without any issue.
  - Joker_vD 6y ago
    Well, in this particular case it was hard. The used version of one our internal libraries didn't support Go 1.13, so we bumped its version, and that required some more dependencies brought in and updated, and then it turned out that the container with Go 1.13 was misconfigured so it literally had no access to that internal library, at all (something with Gitlab deploy keys...), so in a hurry we tried to vendor this internal library wholesale, and then there were some problems with gomod-ed libs and those without go.mod, and some linters had to be explicitly told to skip the 'vendor' directory (wtf), and... And at this point, I just stopped digging down that rabbit hole and instead added TESTIFY := github.com/stretchr/testify and mkdir -p "$(GOPATH)/src/$(TESTIFY)" ; git clone --depth 1 --branch v1.3.0 https://$(TESTIFY).git "$(GOPATH)/src/$(TESTIFY)" before 'go install' in the build script, and ran it in the container with Go 1.10. This change took 5 minutes to make and worked without any problems. Isn't the one of the reasons we use semver in the first place so that after you do some small change to the current application you're working on, you don't suddenly find yourself having to update 2/3 of your the environment just to compile it?
imtringued 6y ago
>Project identity doesn't change if a project changes, it is not a new package. It is a new package. From a practical perspective once you break backwards compatibility the new major version is its own independent thing. If you pretend its the same thing then you run into the situation where two libraries need v1 and v2 and therefore can't coexist. This is going to destroy your entire ecosystem. Replacing a few strings is nothing in comparison.
- JamesSwift 6y ago
  I'm not sure which ecosystem you are referring to (golangs or the package itself) but it seems like there are plenty of existing ecosystems chugging along even though they do, in fact, treat the two versions as the same thing. Not saying its right or wrong, but clearly it is not a death sentence.
  - noselasd 6y ago
    But it is cumbersome. Linux distros have the same issue, rpm or .deb packages cannot have the same name but still support having 2 versions of a package/library installed. If it's a shared library and they don't get the SONAME versioning right for incompatible backwards compabiliry, it's even more cumbersome, and incurs more work for everyone to consume that library.
- sagichmal 6y ago
  It is a new package, from one perspective. That perspective isn't appropriate to use outside of a narrow scope. In particular it's not appropriate to use at an ecosystem level, it's too surprising.
doonesbury 6y ago
>Hardly any projects use strict semver in the real world Part of the appeal of GO is simplicity: get some things down in a minimal yet complete way then consistently do it on the strength that it's a net-win: less complexity to learn that takes you further in most cases. This isn't the only game in town, but it is a good game. C++ is different: we've got 21 million ways of doing things, but you only pay for what you use. That's fine too if you know what you're doing. Back to GO: The fact that the real world is wishy-washy here sometimes in the mood, sometimes not, sometimes I like blondes some days I like brunettes other days isn't the go way ... is a sometimes depressing part of DEVOPS: not formal. It's not the GO way either. So I think GO's standpoint that changing something to explicitly ask for a breaking change isn't out of line. It can't be that tough of an ask to mean what you say and say what you mean when we label code with a semantic version. Both this post and the other V2 problem GO post fail to better get at the crux of the problem: In GO there are two names: - the path we import in code - the actual GIT URL (or vendor ref) down to tag/commitId in go.mod that the previous item ultimately resolves to The module name in code is ambiguous. It's desirable to some that code is unchanged ever wrt to imported module name. If true, then we must turn our attention to what's in go.mod. Here the ask is for hints that there's a breaking version available ... and it'll be up to the DEV to use it or ignore it changing go.mod as they deem makes sense. If false, then you've gotta change the code to reference the breaking version if that's the desire ... realizing it's still ambiguous because it doesn't resolve down to a commitId or tag ... so that leaves go.mod on the table for change too. Nobody debates that code must change to reflect breaking changes if the breaking change is included in go.mod. The questions are: did we ask? Where? Did we know there's a breaking change, and can't we get some help knowing there's a breaking change since GO hits GIT anyway? At it's crux then ... there are two names. Leaving the code unchanged and changing go.mod with some tool help is better IMHO.

kybernetikos 6y ago

> Hardly any projects use strict semver in the real world. Most other package managers don't enforce it. I think strict semver is reasonably possible for things that are libraries rather than products in their own right. But you're right, people want major versions to indicate something significant and new (and occasionally major versions have a contractual significance as well - requiring customers to pay an upgrade fee), rather than small breaking changes. It's even sometimes possible to create a new library that has huge changes in the intended way you use it, its conceptual model and masses of new functionality, but it maintains backwards compatibility - it's perfectly understandable that the package owner wants to indicate this with a major version bump. The other issue I've come across is that what constitutes a breaking change is much more subjective than many people realise. Any change is a breaking change if someone is reading in your library and tweaking bytes in specific locations. Of course, for most libraries they shouldn't be doing that, but that means that if someone is using your library wrong, then you don't worry about breaking them. But the question of if someone is using your library wrong is pretty subjective. At an extreme, that could be considered to be relying on any behaviour not explicitly documented as intended. Ultimately, it comes down to the judgement of the package maintainer, and that doesn't always match up with the judgement of the user. Having version numbers that mean something to machines is very useful when it works though. Perhaps we should just separate those from human-targetted versions rather than go full sentimental versioning: http://sentimentalversioning.org/ http://sentimentalversioning.org/ Maybe something like <human version>.<machine readable api version>.<unique incrementing build number> could work.

grey-area 6y ago
I think because of the many problems listed and the significant one you add (Just what is a breaking change? It turns out pretty much any change can be breaking for some user), the tooling needs to recognise that strict semver is unsound and a looser version is required. This isn't just a question of pesky humans with their marketing versions vs unerring machines - there are complexities in the problem space which shouldn't be ignored. Different producers/projects have different expectations for strictness in semver, and that's ok, they can use it differently in a negotiation with their consumers. Also different consumers have different requirements, and most tools provide a way for them to specify that on a per-import basis (upgrade only minor of these pls unattended, upgrade nothing on this one, only breaking changes on that one). In short, loose semver is a feature, not a bug.

jhoechtl 6y ago

I think you conflate marketing with technical change. You would like to announce a new major version with fanfare, bumping a version number. A programmer wants to be sure that his piece of software continues working if there has only been a minor version bump of a library he is using. Did you read the reasoning of Russ Cox in respect to vgo, versioning and the module system in general? It reads very plausible and understandable. Just because so many peopele interpret semver in their sense instead what it really means shouldn't prevent the Go core team to try to do it right.

grey-area 6y ago
I think you conflate marketing with technical change. There are multiple reasons given above why strict semantic versioning is not used in real-world software, that is a reductive summary of one of them. Yes, I've read those articles a while back when they were written. In a very real sense, a consumer can never be sure that software keeps running properly if a library changes in any way without extensive checks. Real-world loose semantic versioning is a promise, not a proof, and it works better that way. I don't personally think the current go proposal is terrible or sucks, but I do think it could be improved, and it'll be improved by listening to how people use versions.

sacado2 6y ago

> Just what is a breaking change anyway? It's just an API kind of thing. If a function / constant / var / type was available and is not anymore, it's a breaking change. It's an objective measure, there is nothing left to subjectivity. Now, of course a bug fix is going to "break" things if your code relied on the bogus behavior. An updated algorithm can also break things (I maintain a SAT solver, whenever I update the underlying algorithm, even if it's way faster on average, in some limited cases it can make one very specific problem way slower to solve, which can have bad consequences). But as it has no consequence on the API part of things, it's not a major change in semver's meaning.

grey-area 6y ago
There are far more extensive possibilities than what you export - almost every change can be breaking change, often without the producer knowing about it. This is why nobody uses strict semver in practice. https://www.bennadel.com/blog/3501-when-is-a-change-a-breaking-change-for-an-api.htm https://www.bennadel.com/blog/3501-when-is-a-change-a-breaki...
- brabel 6y ago
  The parent commenter is correct in the sense that API changes can be, and should be, automatically checked for server.. you're just saying that doing that still does not guarantee there's no breaking changes because, besides API changes, there's also semantic changes, and those are difficult, perhaps impossible, to automatically check. Or are they? Check this out: https://www.unisonweb.org/docs/tour#-the-big-technical-idea https://www.unisonweb.org/docs/tour#-the-big-technical-idea
  - grey-area 6y ago
    Thanks for the link, a merkle tree for code sounds interesting.
- sacado2 6y ago
  Are we talking about semver in general or semver the way it is (supposed to be) used in go mod?

Joker_vD 6y ago

Okay, here's a real world example. Three months ago, stretchr/testify was version 1.3 and supported Go 1.10. Yesterday, we tried to build some project, and suddenly tests don't run because, you see, now testify is version 1.6 and they support only Go 1.13 and up. That's a bloody breaking change, okay? And it happens all the time with Go packages: they're constantly in the process of dropping support for older Go versions without bumping major versions.

dullgiulio 6y ago
And is Go so hard to upgrade for your application? Go the language is really best in class when it comes to keeping backwards compatibility.
- Spivak 6y ago
  But this is still a breaking change though. If I build my app today I should be able to build it again tomorrow without any issue.
- Joker_vD 6y ago
  Well, in this particular case it was hard. The used version of one our internal libraries didn't support Go 1.13, so we bumped its version, and that required some more dependencies brought in and updated, and then it turned out that the container with Go 1.13 was misconfigured so it literally had no access to that internal library, at all (something with Gitlab deploy keys...), so in a hurry we tried to vendor this internal library wholesale, and then there were some problems with gomod-ed libs and those without go.mod, and some linters had to be explicitly told to skip the 'vendor' directory (wtf), and... And at this point, I just stopped digging down that rabbit hole and instead added TESTIFY := github.com/stretchr/testify and mkdir -p "$(GOPATH)/src/$(TESTIFY)" ; git clone --depth 1 --branch v1.3.0 https://$(TESTIFY).git "$(GOPATH)/src/$(TESTIFY)" before 'go install' in the build script, and ran it in the container with Go 1.10. This change took 5 minutes to make and worked without any problems. Isn't the one of the reasons we use semver in the first place so that after you do some small change to the current application you're working on, you don't suddenly find yourself having to update 2/3 of your the environment just to compile it?

imtringued 6y ago

>Project identity doesn't change if a project changes, it is not a new package. It is a new package. From a practical perspective once you break backwards compatibility the new major version is its own independent thing. If you pretend its the same thing then you run into the situation where two libraries need v1 and v2 and therefore can't coexist. This is going to destroy your entire ecosystem. Replacing a few strings is nothing in comparison.

JamesSwift 6y ago
I'm not sure which ecosystem you are referring to (golangs or the package itself) but it seems like there are plenty of existing ecosystems chugging along even though they do, in fact, treat the two versions as the same thing. Not saying its right or wrong, but clearly it is not a death sentence.
- noselasd 6y ago
  But it is cumbersome. Linux distros have the same issue, rpm or .deb packages cannot have the same name but still support having 2 versions of a package/library installed. If it's a shared library and they don't get the SONAME versioning right for incompatible backwards compabiliry, it's even more cumbersome, and incurs more work for everyone to consume that library.
sagichmal 6y ago
It is a new package, from one perspective. That perspective isn't appropriate to use outside of a narrow scope. In particular it's not appropriate to use at an ecosystem level, it's too surprising.

doonesbury 6y ago

>Hardly any projects use strict semver in the real world Part of the appeal of GO is simplicity: get some things down in a minimal yet complete way then consistently do it on the strength that it's a net-win: less complexity to learn that takes you further in most cases. This isn't the only game in town, but it is a good game. C++ is different: we've got 21 million ways of doing things, but you only pay for what you use. That's fine too if you know what you're doing. Back to GO: The fact that the real world is wishy-washy here sometimes in the mood, sometimes not, sometimes I like blondes some days I like brunettes other days isn't the go way ... is a sometimes depressing part of DEVOPS: not formal. It's not the GO way either. So I think GO's standpoint that changing something to explicitly ask for a breaking change isn't out of line. It can't be that tough of an ask to mean what you say and say what you mean when we label code with a semantic version. Both this post and the other V2 problem GO post fail to better get at the crux of the problem: In GO there are two names: - the path we import in code - the actual GIT URL (or vendor ref) down to tag/commitId in go.mod that the previous item ultimately resolves to The module name in code is ambiguous. It's desirable to some that code is unchanged ever wrt to imported module name. If true, then we must turn our attention to what's in go.mod. Here the ask is for hints that there's a breaking version available ... and it'll be up to the DEV to use it or ignore it changing go.mod as they deem makes sense. If false, then you've gotta change the code to reference the breaking version if that's the desire ... realizing it's still ambiguous because it doesn't resolve down to a commitId or tag ... so that leaves go.mod on the table for change too. Nobody debates that code must change to reflect breaking changes if the breaking change is included in go.mod. The questions are: did we ask? Where? Did we know there's a breaking change, and can't we get some help knowing there's a breaking change since GO hits GIT anyway? At it's crux then ... there are two names. Leaving the code unchanged and changing go.mod with some tool help is better IMHO.