7 ms·

I think most of the frustration I have seen comes from people expecting symmetry and uniformity in the vendor/package management solution. This is the same way

by harikb 6y ago

I think most of the frustration I have seen comes from people expecting symmetry and uniformity in the vendor/package management solution. This is the same way some looking at an architecture diagram expects clean straight lines or aligned boundaries. That said, Go team should have made it clear right in the title (for all blog posts) that they are attempting a unique non-intuitive solution to a hard problem of transition from 'no-versioned' world of existing packages to 'semver' and the explicit desire for force people to avoid versioning if they can (just like the old days). There is nothing wrong with weird solutions. We don't have to follow Rust or Ruby or npm. Their situation is different. Of course then the protobuf or grpc team (I forget which one) went and created a new "v2" without the "v2" suffix and that pissed a few people off. Of course that was still within the reasoning Russ explained (it was a reimplementation and domain part of the URL already changed, so there was no need to add v2), and Russ probably doesn't control the particular package's team. But that is life! People still expect perfection from opensource projects.

nemothekid 6y ago
What I dislike about Go's solution, and somewhat about semver in general is that no one makes major version changes anymore even if they break backwards compatibility. However if I use semver internally, like it's supposed to and make regular major changes (let's say once a month), my code repo becomes a mess. After a 2 years, I'll have folders for v1 up to v24. And the fact that I have to copy -R/grep|sed, means that people will be less likely to properly signal that a backwards incompatible change has occured. Disregarding uniformity, for the reasons above I don't think their design is a good one. It treats major version changes as a rare event.
- andrewprock 6y ago
  For a designed library, major version changes should be a rare event. For a collection of code that someone calls a library, all bets are off.
  - Aeolun 6y ago
    What open source project is a designed library? This will just pollute the API interface for the sake of maintaining backwards compatibility.
    - slavik81 6y ago
      Qt hasn't broken binary compatibility since the release of Qt 5.0 in 2012. The last one before that was the release of Qt 4.0 in 2005. It's a very impressive standard they set out for themselves (and met). https://wiki.qt.io/Qt-Version-Compatibility https://wiki.qt.io/Qt-Version-Compatibility
  - jerf 6y ago
    "For a designed library, major version changes should be a rare event." If you properly honor semver, for any non-static library, major version changes will either be a modestly-frequent event, or you're really straightjacketing yourself with your first v1, or you're a genius who got everything right the first time and never have to make any changes. Or, most likely, you're not honoring semver properly and kindasorta squint at your changes until they don't seem important anymore and then ship an incremental anyhow. Go may make it somewhat more annoying than the average language to ship a new major version in some ways, but libraries either not rev'ing their major version number when they technically should have and mostly expecting to mostly get away with it, or libraries bending themselves slowly into pretzels to stay on the same major version according to semver when they really ought to just make a new major version are things I see in many languages.
    - wahern 6y ago
      > If you properly honor semver, for any non-static library, major version changes will either be a modestly-frequent event, or you're really straightjacketing yourself with your first v1, or you're a genius who got everything right the first time and never have to make any changes. IMO, someone tackling a problem for the first time shouldn't be writing public libraries. At the very least, people shouldn't be using it. Quality, stable libraries tend to be written by people who already have at least 1 or 2 iterations under their belt. Maybe not complete iterations, and rarely public, but enough experience with the nuts & bolts of the problem, including the problem of balancing concerns between caller & callee, to have a good idea about what the proper interfaces should look like to ensure long-term stability. If someone doesn't have a strong sense that they could support the v1 interface in perpetuity, then maybe don't create a separate project. Not only does it save everybody else from the pain, it often saves the author the pain of constant churn, presuming the author even uses it in more than a handful of their own projects. For many of my own open source projects, I often have years of abandoned false starts lying around, or various just-so solutions manually copied and hacked into various other projects. Only when I feel like things have finally congealed do I then bother to write a dedicated library, whether private or public, and migrate everything to it. One way to know things are looking good is when you realize you're no longer fiddling with the API, or maybe just vacillating between two equally decent alternatives.
- infogulch 6y ago
  If you write something for internal use and can enforce version upgrades could you not just keep the last, say, 3 versions in the repo HEAD and delete folders v1-v20? Between go.mod and go.sum, historical dependencies will still resolve fine because they point at the old commit. If you know you will never release another patch for a particular major version I think you could just drop the directory from the repo.
- Skinney 6y ago
  If you're introducing breaking changes once per month, then your library really isn't ready. Tell people you have an unstable library. Breaking changes _will_ happen. Then, when you've used your library in production for a while and are reasonably sure it wont have to change, tell people it's stable. If you make breaking changes after that, then start with a v2 suffix. For some reason, semver has made people stop going through the alpha, beta and rc stages of a release, even though the point of such things is to discover breaking changes before you're commited to maintaining the version for the time to come. Major version changes should be a rare event. If it's not, you're really just shipping around a (pre)alpha library, and should be explicit about that.
  - barumi 6y ago
    > If you're introducing breaking changes once per month, then your library really isn't ready. The whole software industry moved away from waterfall to agile because it's unreasonable to assume that no requirements will emerge. I would disagree, for example, that Python3 is not ready just because they're introduced backward incompatible changes in some minor releases.
    - tick_tock_tick 6y ago
      I mean python is basically the prime example of how to poorly manage version upgrades with their failure of 2 => 3.
      - bigiain 6y ago
        They were just desperate to do a better job of screwing up than Perl with it's 5 => 6 updates... <grin>
        masklinn 6y ago
        As we can see from the Perl ecosystem having completely moved over to the extent that Perl 5 is now out of support while Python is still mired in the update.
- atombender 6y ago
  Go doesn't require a folder for each version. You can simply change go.mod to declare the module as github.com/nemothekid/foolib/v2 or whatever. That's one of the points of confusion — first that the go.mod module declaration doesn't need to exactly match the module location, and secondly, following this, that using a folder instead is a fallback option. Go is famous for favouring simplicity in its design, but this is a rare instance where it's not made things simple enough.
  - deleted 6y ago
    [deleted]
whatsmyusername 6y ago
No-version is incredibly dumb for the simple fact that vulnerabilities exist. If you make it hard for tools like Snyk to work with your language I'm not going to recommend it to anyone. I actually like Golang in specific circumstances (backend simple micro services, anything docker, and utility binaries) but they picked the versioning scheme as the, "This is the dumb hill we're going to die on," that every language has in some fashion (ex: no case statements in python, perl 6 coming soon, it's 2001 and we like it that way for php, java... just, all of it).
- Spivak 6y ago
  I mean isn’t no-version effectively just date versioning so tools like that just say “any version after $date” is patched?
  - johnisgood 6y ago
    Kind of off topic but that is how I name the cloned git repositories. foo-git says nothing. foo-xxxxxxx says nothing, really. foo-20200916 at least tells me when I cloned the repository, and if I really wanna know which was the last commit, I can just git log it (usually, well, always a git repository).
  - srtjstjsj 6y ago
    Only if the date is in the name.
newen 6y ago
Why do people still say that you can’t expect quality from open source projects when there are multi-million to multi-billion dollar organizations behind them?
- harikb 6y ago
  I don't know why people think that Google having multiple billions should mean they are willing to spend billions on every thing they do. That is not sustainable. Every project has a value to that company and any sane company will only spend whatever level matches that. They are paying for, say 100 engineers (just making up this number). Isn't that enough?
- paulryanrogers 6y ago
  Because those projects are a small fraction of all OSS.
- ex_amazon_sde 6y ago
  Because dollars are not correlated with quality.
- srtjstjsj 6y ago
  Perfection is different from quality.
jayd16 6y ago
>There is nothing wrong with weird solutions. I don't want to go as far as argumentum ad antiquitatem but if you break KISS and the principle of least astonishment it should probably be worth the hassle. >Their situation is different. Is it really, though?
neild 6y ago
I'm one of the people responsible for the choice of versioning for the Go protobuf module. (Sorry!) The original Go protobuf implementation is in the module "github.com/golang/protobuf". A couple years back, we decided that it was time to do a major overhaul of the protobuf implementation. Protobufs were one of the first major pieces of Go code outside the standard library (the second post ever on the Go blog is about it), and there were a number of design decisions that made sense at the time but which we wanted to go back and revisit. Doing all of this was not going to be possible while preserving API compatibility. Given that, we needed a new module path. That's the core of Go modules and API changes: You change the module path when you make an incompatible change. You can do this by bumping the major version of the module (since the module path includes the version), or by changing the module path entirely. (There's another option, which I think doesn't get enough attention: You can just make the change, and force your users to deal with it. If you're okay with breaking your users when they upgrade, nothing stops you from doing this. We take backwards compatibility very seriously, so that wasn't an option for us.) Given that we needed a new module path anyway, we took advantage of the opportunity to move to a path that isn't specific to a particular code hosting provider: "google.golang.org/protobuf". GitHub is great, we love GitHib, but we'd much rather have a provider-agnostic name for the module. So we have a new module path. The question then was: What version do we tag the new module as when we release it? Remember that there has never been any version of "google.golang.org/protobuf" at this point; it's a completely new module. One option was to tag it as v1.0.0. But that might be confusing, because this is "version 2" of the protobuf implementation, even if it's under a different module path. Also, if someone complains about a problem with "v1.2.0" and doesn't mention which module they're using, we won't have any way to tell which one they're talking about. Another option was to tag it as v2.0.0. But that might be confusing, because there's never been a v1 of this package. Also, this would likely cause unnecessary confusion with "proto2" and "proto3" (versions of the protocol buffer language). We waffled for quite a while, and eventually settled on v1.20.0, which avoids the missing v1 weirdness but also ensures that there is no overlap in versions between the two modules. That confused people. Alas. Perhaps every option would have led to confusion. In retrospect, perhaps we should have gone with v2.0.0 (and a module name of "google.golang.org/protobuf/v2"). Too late now; oh, well. If this was the worst decision we made in the new implementation (and so far, it looks like it may have been), I'll be quite happy.
srtjstjsj 6y ago
The versioning model says you have to rename your package for breaking changes. The v2 convention is for when you want to mostly keep the old name. Protobuf package got renamed because they didn't like the old name and wanted to change it more significantly.

by harikb 6y ago

nemothekid 6y ago

What I dislike about Go's solution, and somewhat about semver in general is that no one makes major version changes anymore even if they break backwards compatibility. However if I use semver internally, like it's supposed to and make regular major changes (let's say once a month), my code repo becomes a mess. After a 2 years, I'll have folders for v1 up to v24. And the fact that I have to copy -R/grep|sed, means that people will be less likely to properly signal that a backwards incompatible change has occured. Disregarding uniformity, for the reasons above I don't think their design is a good one. It treats major version changes as a rare event.

andrewprock 6y ago
For a designed library, major version changes should be a rare event. For a collection of code that someone calls a library, all bets are off.
- Aeolun 6y ago
  What open source project is a designed library? This will just pollute the API interface for the sake of maintaining backwards compatibility.
  - slavik81 6y ago
    Qt hasn't broken binary compatibility since the release of Qt 5.0 in 2012. The last one before that was the release of Qt 4.0 in 2005. It's a very impressive standard they set out for themselves (and met). https://wiki.qt.io/Qt-Version-Compatibility https://wiki.qt.io/Qt-Version-Compatibility
- jerf 6y ago
  "For a designed library, major version changes should be a rare event." If you properly honor semver, for any non-static library, major version changes will either be a modestly-frequent event, or you're really straightjacketing yourself with your first v1, or you're a genius who got everything right the first time and never have to make any changes. Or, most likely, you're not honoring semver properly and kindasorta squint at your changes until they don't seem important anymore and then ship an incremental anyhow. Go may make it somewhat more annoying than the average language to ship a new major version in some ways, but libraries either not rev'ing their major version number when they technically should have and mostly expecting to mostly get away with it, or libraries bending themselves slowly into pretzels to stay on the same major version according to semver when they really ought to just make a new major version are things I see in many languages.
  - wahern 6y ago
    > If you properly honor semver, for any non-static library, major version changes will either be a modestly-frequent event, or you're really straightjacketing yourself with your first v1, or you're a genius who got everything right the first time and never have to make any changes. IMO, someone tackling a problem for the first time shouldn't be writing public libraries. At the very least, people shouldn't be using it. Quality, stable libraries tend to be written by people who already have at least 1 or 2 iterations under their belt. Maybe not complete iterations, and rarely public, but enough experience with the nuts & bolts of the problem, including the problem of balancing concerns between caller & callee, to have a good idea about what the proper interfaces should look like to ensure long-term stability. If someone doesn't have a strong sense that they could support the v1 interface in perpetuity, then maybe don't create a separate project. Not only does it save everybody else from the pain, it often saves the author the pain of constant churn, presuming the author even uses it in more than a handful of their own projects. For many of my own open source projects, I often have years of abandoned false starts lying around, or various just-so solutions manually copied and hacked into various other projects. Only when I feel like things have finally congealed do I then bother to write a dedicated library, whether private or public, and migrate everything to it. One way to know things are looking good is when you realize you're no longer fiddling with the API, or maybe just vacillating between two equally decent alternatives.
infogulch 6y ago
If you write something for internal use and can enforce version upgrades could you not just keep the last, say, 3 versions in the repo HEAD and delete folders v1-v20? Between go.mod and go.sum, historical dependencies will still resolve fine because they point at the old commit. If you know you will never release another patch for a particular major version I think you could just drop the directory from the repo.
Skinney 6y ago
If you're introducing breaking changes once per month, then your library really isn't ready. Tell people you have an unstable library. Breaking changes _will_ happen. Then, when you've used your library in production for a while and are reasonably sure it wont have to change, tell people it's stable. If you make breaking changes after that, then start with a v2 suffix. For some reason, semver has made people stop going through the alpha, beta and rc stages of a release, even though the point of such things is to discover breaking changes before you're commited to maintaining the version for the time to come. Major version changes should be a rare event. If it's not, you're really just shipping around a (pre)alpha library, and should be explicit about that.
- barumi 6y ago
  > If you're introducing breaking changes once per month, then your library really isn't ready. The whole software industry moved away from waterfall to agile because it's unreasonable to assume that no requirements will emerge. I would disagree, for example, that Python3 is not ready just because they're introduced backward incompatible changes in some minor releases.
  - tick_tock_tick 6y ago
    I mean python is basically the prime example of how to poorly manage version upgrades with their failure of 2 => 3.
    - bigiain 6y ago
      They were just desperate to do a better job of screwing up than Perl with it's 5 => 6 updates... <grin>
      - masklinn 6y ago
        As we can see from the Perl ecosystem having completely moved over to the extent that Perl 5 is now out of support while Python is still mired in the update.
atombender 6y ago
Go doesn't require a folder for each version. You can simply change go.mod to declare the module as github.com/nemothekid/foolib/v2 or whatever. That's one of the points of confusion — first that the go.mod module declaration doesn't need to exactly match the module location, and secondly, following this, that using a folder instead is a fallback option. Go is famous for favouring simplicity in its design, but this is a rare instance where it's not made things simple enough.
- deleted 6y ago
  [deleted]

whatsmyusername 6y ago

No-version is incredibly dumb for the simple fact that vulnerabilities exist. If you make it hard for tools like Snyk to work with your language I'm not going to recommend it to anyone. I actually like Golang in specific circumstances (backend simple micro services, anything docker, and utility binaries) but they picked the versioning scheme as the, "This is the dumb hill we're going to die on," that every language has in some fashion (ex: no case statements in python, perl 6 coming soon, it's 2001 and we like it that way for php, java... just, all of it).

Spivak 6y ago
I mean isn’t no-version effectively just date versioning so tools like that just say “any version after $date” is patched?
- johnisgood 6y ago
  Kind of off topic but that is how I name the cloned git repositories. foo-git says nothing. foo-xxxxxxx says nothing, really. foo-20200916 at least tells me when I cloned the repository, and if I really wanna know which was the last commit, I can just git log it (usually, well, always a git repository).
- srtjstjsj 6y ago
  Only if the date is in the name.

newen 6y ago

Why do people still say that you can’t expect quality from open source projects when there are multi-million to multi-billion dollar organizations behind them?

harikb 6y ago
I don't know why people think that Google having multiple billions should mean they are willing to spend billions on every thing they do. That is not sustainable. Every project has a value to that company and any sane company will only spend whatever level matches that. They are paying for, say 100 engineers (just making up this number). Isn't that enough?
paulryanrogers 6y ago
Because those projects are a small fraction of all OSS.
ex_amazon_sde 6y ago
Because dollars are not correlated with quality.
srtjstjsj 6y ago
Perfection is different from quality.

jayd16 6y ago

>There is nothing wrong with weird solutions. I don't want to go as far as argumentum ad antiquitatem but if you break KISS and the principle of least astonishment it should probably be worth the hassle. >Their situation is different. Is it really, though?

neild 6y ago

I'm one of the people responsible for the choice of versioning for the Go protobuf module. (Sorry!) The original Go protobuf implementation is in the module "github.com/golang/protobuf". A couple years back, we decided that it was time to do a major overhaul of the protobuf implementation. Protobufs were one of the first major pieces of Go code outside the standard library (the second post ever on the Go blog is about it), and there were a number of design decisions that made sense at the time but which we wanted to go back and revisit. Doing all of this was not going to be possible while preserving API compatibility. Given that, we needed a new module path. That's the core of Go modules and API changes: You change the module path when you make an incompatible change. You can do this by bumping the major version of the module (since the module path includes the version), or by changing the module path entirely. (There's another option, which I think doesn't get enough attention: You can just make the change, and force your users to deal with it. If you're okay with breaking your users when they upgrade, nothing stops you from doing this. We take backwards compatibility very seriously, so that wasn't an option for us.) Given that we needed a new module path anyway, we took advantage of the opportunity to move to a path that isn't specific to a particular code hosting provider: "google.golang.org/protobuf". GitHub is great, we love GitHib, but we'd much rather have a provider-agnostic name for the module. So we have a new module path. The question then was: What version do we tag the new module as when we release it? Remember that there has never been any version of "google.golang.org/protobuf" at this point; it's a completely new module. One option was to tag it as v1.0.0. But that might be confusing, because this is "version 2" of the protobuf implementation, even if it's under a different module path. Also, if someone complains about a problem with "v1.2.0" and doesn't mention which module they're using, we won't have any way to tell which one they're talking about. Another option was to tag it as v2.0.0. But that might be confusing, because there's never been a v1 of this package. Also, this would likely cause unnecessary confusion with "proto2" and "proto3" (versions of the protocol buffer language). We waffled for quite a while, and eventually settled on v1.20.0, which avoids the missing v1 weirdness but also ensures that there is no overlap in versions between the two modules. That confused people. Alas. Perhaps every option would have led to confusion. In retrospect, perhaps we should have gone with v2.0.0 (and a module name of "google.golang.org/protobuf/v2"). Too late now; oh, well. If this was the worst decision we made in the new implementation (and so far, it looks like it may have been), I'll be quite happy.

srtjstjsj 6y ago

The versioning model says you have to rename your package for breaking changes. The v2 convention is for when you want to mostly keep the old name. Protobuf package got renamed because they didn't like the old name and wanted to change it more significantly.