6 ms·

I'm the author of the other recent post that made it to the front page about the trouble with Go's v2+ modules. I disagree with much of this post. > Go makes

by donatj 6y ago

I'm the author of the other recent post that made it to the front page about the trouble with Go's v2+ modules. I disagree with much of this post. > Go makes updating major versions so cumbersome that in the majority of cases, we have opted to just increment minor versions when we should increment major versions. I disagree with this, wholeheartedly. I think Go's module system making modules separate packages on major versions is actually great once you actually know how it works. The problem is simply that it's non-obvious and not well understood. I think they could have made it more obvious requiring in on v1 for starters. Beyond that you should absolutely be tagging major versions for every breaking change, and it should be something a developer has to stop and think about. Not something to be taken lightly, EVEN with regard to internal libraries. You can learn how to code things preemptively so they are flexible for additive changes and require fewer breaking changes. It's almost always worth the effort in the long run. To my posts point, the problem is just people don't know how it works, and it's design makes it something you learn late into a project, when you're tagging v2, rather than something you hit early. FWIW, my post for anyone didn't catch it: https://donatstudios.com/Go-v2-Modules https://donatstudios.com/Go-v2-Modules

bad_user 6y ago
To add, making the version part of the namespace is the right thing to do, also because it avoids conflicts with transitive dependencies. Changing the namespace means that the same project can depend on multiple different major versions of the same library. And that's fine, because when you break compatibility, you're actually not talking about the same library. And this makes upstream developers think twice about breaking compatibility. Accretion of features should be preferable to breakage. It's what Rich Hickey talks about in his Spec-ulation talk: https://www.youtube.com/watch?v=oyLBGkS5ICk https://www.youtube.com/watch?v=oyLBGkS5ICk
- grey-area 6y ago
  Accretion of features should be preferable to breakage. I think most people, particularly those using Go, would agree with this. It does not follow that we should make breaking changes easier or routine, nor that we should force people to use strict semver (which is not widely used for good reasons). I see why they've done that as it simplifies assumptions but prefer the way other package managers handle this where it is left to producers and consumers to negotiate how strict they want to be.
- stubish 6y ago
  > Changing the namespace means that the same project can depend on multiple different major versions of the same library. Do you actually want that? Do you actually know that the multiple versions of the dependency you happen to be using do not conflict? If the dependency is in different namespaces their locks and other globals are also in different namespaces.
  - bad_user 6y ago
    Yes, I do, because this isn't just about my source-code, but about the dependencies of dependencies.
    - grey-area 6y ago
      The two approaches to this are to force resolving dependencies (can be painful and require humans), or duplicate code (can cause bugs). There are disadvantages to both and in particular duplicating dependencies should be seen as a short-term fix for a transition, not something you do routinely. If you do it routinely you'd see bugs due to shared locks, state, or changed data structures - it breaks assumptions about pkg globals for example, an important part of the language used extensively in the stdlib.
      - yxhuvud 6y ago
        Or even conflicts rising from competing use of state space in some other app. For example, if your database have a limited set of connections it can set up then you absolutely don't want different dependencies to handle connection pooling by themselves.
      - imtringued 6y ago
        >There are disadvantages to both and in particular duplicating dependencies should be seen as a short-term fix for a transition, not something you do routinely. If you don't duplicate versions then you will have to wait for every single library in existence to update to the latest major version. This can take decades and still fail. Just take a look at Python 3.
    - 6y ago
- FreeFull 6y ago
  It's not like putting the version into the namespace is necessary for this, though. Rust's Cargo will let you use multiple versions of a library too, without extra namespacing.
  - blablabla123 6y ago
    ...and needs a package manager to solve NP-complete dep hell: https://research.swtch.com/version-sat https://research.swtch.com/version-sat Go's package management is really well designed I think and also actually semantically relying on semver
grey-area 6y ago
No, that is not the problem, people are not just holding it wrong. The problem is a mismatch between the Go team's perception of semver usage (or perhaps how they think it should be used) and its real-world usage. Some examples: Major versions are used to communicate more than breaking changes. Project identity doesn't change if a project changes, it is not a new package. Hardly any projects use strict semver in the real world, and other package managers allow a more flexible, looser semver. Just what is a breaking change anyway? Real-world packages make small security changes which break compatibility, sometimes minor changes do too, and that's ok if breakage is very limited in scope. In the current go modules, new major versions are not easily discoverable or announced, and new importers will need to consult docs or know about go get @latest to find the latest version. This could be fixed with tooling if the problem is recognised. There are possible solutions if they want to keep /v2 paths, but we shouldn't pretend there are no problems caused by this change. It makes it easier to produce breaking changes and harder to version increment just for new features (as most projects from linux to rails do). Personally I think it condones breaking changes and will lead to more breaking changes, not fewer - if you're forced to change import paths at higher versions why not just delete that api you don't like? Within Google say there are significant downsides to this and pressure to fix your own breakage, outside, not so much, including in packages people are required to use, like api client packages for popular services. Significant breaking changes introduce real pain for consumers and in an open ecosystem should be avoided at all costs.
- kybernetikos 6y ago
  > Hardly any projects use strict semver in the real world. Most other package managers don't enforce it. I think strict semver is reasonably possible for things that are libraries rather than products in their own right. But you're right, people want major versions to indicate something significant and new (and occasionally major versions have a contractual significance as well - requiring customers to pay an upgrade fee), rather than small breaking changes. It's even sometimes possible to create a new library that has huge changes in the intended way you use it, its conceptual model and masses of new functionality, but it maintains backwards compatibility - it's perfectly understandable that the package owner wants to indicate this with a major version bump. The other issue I've come across is that what constitutes a breaking change is much more subjective than many people realise. Any change is a breaking change if someone is reading in your library and tweaking bytes in specific locations. Of course, for most libraries they shouldn't be doing that, but that means that if someone is using your library wrong, then you don't worry about breaking them. But the question of if someone is using your library wrong is pretty subjective. At an extreme, that could be considered to be relying on any behaviour not explicitly documented as intended. Ultimately, it comes down to the judgement of the package maintainer, and that doesn't always match up with the judgement of the user. Having version numbers that mean something to machines is very useful when it works though. Perhaps we should just separate those from human-targetted versions rather than go full sentimental versioning: http://sentimentalversioning.org/ http://sentimentalversioning.org/ Maybe something like <human version>.<machine readable api version>.<unique incrementing build number> could work.
wokwokwok 6y ago
I'm not sure it's just an education issue; it should be an end to end solution. If people aren't using it, it's weird. That smells of trouble to me... either it should be good enough people want to use it, or painful enough not using it that people grit their teeth and do the right thing because its less effort in the long term (eg. you cannot use module at all unless you do it right). If there is no obvious downside to doing the 'wrong' thing, and it's less effort, and people are doing the 'wrong thing', in practice, in the wild... ...well, I'm not sure this has really worked out ideally. Certainly, "great" is not how I would describe it.

by donatj 6y ago

bad_user 6y ago

To add, making the version part of the namespace is the right thing to do, also because it avoids conflicts with transitive dependencies. Changing the namespace means that the same project can depend on multiple different major versions of the same library. And that's fine, because when you break compatibility, you're actually not talking about the same library. And this makes upstream developers think twice about breaking compatibility. Accretion of features should be preferable to breakage. It's what Rich Hickey talks about in his Spec-ulation talk: https://www.youtube.com/watch?v=oyLBGkS5ICk https://www.youtube.com/watch?v=oyLBGkS5ICk

grey-area 6y ago
Accretion of features should be preferable to breakage. I think most people, particularly those using Go, would agree with this. It does not follow that we should make breaking changes easier or routine, nor that we should force people to use strict semver (which is not widely used for good reasons). I see why they've done that as it simplifies assumptions but prefer the way other package managers handle this where it is left to producers and consumers to negotiate how strict they want to be.
stubish 6y ago
> Changing the namespace means that the same project can depend on multiple different major versions of the same library. Do you actually want that? Do you actually know that the multiple versions of the dependency you happen to be using do not conflict? If the dependency is in different namespaces their locks and other globals are also in different namespaces.
- bad_user 6y ago
  Yes, I do, because this isn't just about my source-code, but about the dependencies of dependencies.
  - grey-area 6y ago
    The two approaches to this are to force resolving dependencies (can be painful and require humans), or duplicate code (can cause bugs). There are disadvantages to both and in particular duplicating dependencies should be seen as a short-term fix for a transition, not something you do routinely. If you do it routinely you'd see bugs due to shared locks, state, or changed data structures - it breaks assumptions about pkg globals for example, an important part of the language used extensively in the stdlib.
    - yxhuvud 6y ago
      Or even conflicts rising from competing use of state space in some other app. For example, if your database have a limited set of connections it can set up then you absolutely don't want different dependencies to handle connection pooling by themselves.
    - imtringued 6y ago
      >There are disadvantages to both and in particular duplicating dependencies should be seen as a short-term fix for a transition, not something you do routinely. If you don't duplicate versions then you will have to wait for every single library in existence to update to the latest major version. This can take decades and still fail. Just take a look at Python 3.
  - 6y ago
FreeFull 6y ago
It's not like putting the version into the namespace is necessary for this, though. Rust's Cargo will let you use multiple versions of a library too, without extra namespacing.
- blablabla123 6y ago
  ...and needs a package manager to solve NP-complete dep hell: https://research.swtch.com/version-sat https://research.swtch.com/version-sat Go's package management is really well designed I think and also actually semantically relying on semver

grey-area 6y ago

No, that is not the problem, people are not just holding it wrong. The problem is a mismatch between the Go team's perception of semver usage (or perhaps how they think it should be used) and its real-world usage. Some examples: Major versions are used to communicate more than breaking changes. Project identity doesn't change if a project changes, it is not a new package. Hardly any projects use strict semver in the real world, and other package managers allow a more flexible, looser semver. Just what is a breaking change anyway? Real-world packages make small security changes which break compatibility, sometimes minor changes do too, and that's ok if breakage is very limited in scope. In the current go modules, new major versions are not easily discoverable or announced, and new importers will need to consult docs or know about go get @latest to find the latest version. This could be fixed with tooling if the problem is recognised. There are possible solutions if they want to keep /v2 paths, but we shouldn't pretend there are no problems caused by this change. It makes it easier to produce breaking changes and harder to version increment just for new features (as most projects from linux to rails do). Personally I think it condones breaking changes and will lead to more breaking changes, not fewer - if you're forced to change import paths at higher versions why not just delete that api you don't like? Within Google say there are significant downsides to this and pressure to fix your own breakage, outside, not so much, including in packages people are required to use, like api client packages for popular services. Significant breaking changes introduce real pain for consumers and in an open ecosystem should be avoided at all costs.

kybernetikos 6y ago
> Hardly any projects use strict semver in the real world. Most other package managers don't enforce it. I think strict semver is reasonably possible for things that are libraries rather than products in their own right. But you're right, people want major versions to indicate something significant and new (and occasionally major versions have a contractual significance as well - requiring customers to pay an upgrade fee), rather than small breaking changes. It's even sometimes possible to create a new library that has huge changes in the intended way you use it, its conceptual model and masses of new functionality, but it maintains backwards compatibility - it's perfectly understandable that the package owner wants to indicate this with a major version bump. The other issue I've come across is that what constitutes a breaking change is much more subjective than many people realise. Any change is a breaking change if someone is reading in your library and tweaking bytes in specific locations. Of course, for most libraries they shouldn't be doing that, but that means that if someone is using your library wrong, then you don't worry about breaking them. But the question of if someone is using your library wrong is pretty subjective. At an extreme, that could be considered to be relying on any behaviour not explicitly documented as intended. Ultimately, it comes down to the judgement of the package maintainer, and that doesn't always match up with the judgement of the user. Having version numbers that mean something to machines is very useful when it works though. Perhaps we should just separate those from human-targetted versions rather than go full sentimental versioning: http://sentimentalversioning.org/ http://sentimentalversioning.org/ Maybe something like <human version>.<machine readable api version>.<unique incrementing build number> could work.

wokwokwok 6y ago

I'm not sure it's just an education issue; it should be an end to end solution. If people aren't using it, it's weird. That smells of trouble to me... either it should be good enough people want to use it, or painful enough not using it that people grit their teeth and do the right thing because its less effort in the long term (eg. you cannot use module at all unless you do it right). If there is no obvious downside to doing the 'wrong' thing, and it's less effort, and people are doing the 'wrong thing', in practice, in the wild... ...well, I'm not sure this has really worked out ideally. Certainly, "great" is not how I would describe it.