33 ms·

Go modules have a v2+ problem

217 points by nitinreddy88 6y ago

grenoire 6y ago
Re: "They need at the very least a section of the documentation that lays it out clearly and in layman's terms." As a Go outsider who's trying to get his feet dipped in the lake, trying to use modules has felt... uneasy? I tried to look up the official resources on the matter and found the official blog posts to be lacking concrete and concise examples on the usage; documentation feels like it's written for people who work on the project and already digested the unit tests as examples. As for the case of v2, similarly messy and undocumented. I also appreciate author's call for action on warning messages in tooling. Wasn't the unified tooling supposed to be a selling point for Go? It just doesn't feel good to use.
- throwawayinfo 6y ago
  Poor documentation seems to be a Google culture thing. I suspect that's why godoc is literally just source code comments wrapped in HTML.
  - dan_can_code 6y ago
    I have found this to be almost universal when it comes to reading / writing documentation, not just Google. I have come to the understanding that to write good documentation, you should write it as soon as you have learned it, or at least, try and explain in a similar way in the way you learned (which is not easy at all). I feel the issue stems from abstraction. Once you become somewhat an expert in a topic or develop a deeper understanding of said topic, you automatically abstract away the information that lead you to that understanding in the first place. Otherwise you end up reading or writing documentation with a bunch of assumptions for knowledge and understanding, which are not only not stated but may not be available to the person trying to get to grips with the technology. A bit of a digression from the post, but I don't feel this is just a Google thing.
    - scns 6y ago
      There is a name for it: The experts blind spot
      - k00b 6y ago
        I’ve heard it called and prefer to call it the curse of knowledge
      - dan_can_code 6y ago
        Thanks for sharing this with me. I didn't know this was a widely known phenomenon. I'll take a look at the link to help avoid having these blind spots myself, thanks again!
      - dmitriid 6y ago
        Erlang's creator Joe Armstrong once said it like this (quoting from memory) "There's a moment when you go from not getting it to getting it, and you have about 24 hours when you remember both before and after, and that's when you have to write things down. After that you won't remember how it was not getting it"
    - virgilp 6y ago
      Rust has pretty decent documentation. My point is that it's possible/ we shouldn't settle for "that's the universal status quo, nothing can be done about it". Something _can_ be done. And we _should_ be demanding more - especially so from tech giants like Google.
      - dan_can_code 6y ago
        I absolutely agree. There is no reason to settle for anything less. Google certainly has the resources to fill the gap - recognising it would be the first step. Looking at the Go documentation, it seems they have feedback mechanisms for the features of Go, but no means of providing feedback on the actual documentation. Rust however have both Github contributions, as well as a channel on discord specifically for documentation discussions. I would assume this may contribute to the lack of satisfactory documentation for Go.
  - thefounder 6y ago
    I can't disagree more.
  - rob74 6y ago
    I've seen projects with much worse documentation than Go. And I think it's unfair to blame the quality of the documentation on godoc, which is actually a pretty powerful documentation tool, and the fact that it's included with the language is a great advantage. As with other aspects of Go, they may have taken the "keep it simple" principle a bit too far here too (e.g. godoc could detect and highlight if you mention the names of a method's parameters in a doc comment), but what's there is very solid. Of course, as with any documentation tool, you have to actually use it by writing good doc comments (end examples, and additional documentation files etc.) to make it useful, but that's not the tool's fault...
    - nobleach 6y ago
      Yeah, OCaml's docs are FAR worse as they're often just type signatures. I'm also not a fan of Oracle's Java docs as they do explain what each method does, but often in a rather opaque way. Where Elixir and Kotlin get things right, is they tend to also give quick examples of how one might use any given method.
  - clinta 6y ago
    Isn't that the case with most languages now? Pydoc and rustdoc do the same thing.
    - trumpeta 6y ago
      Last time I checked it was not possible to actually use godoc to render the doc, it just started an internal server. That made it super hard to publish docs internally.
  - jimktrains2 6y ago
    Isn't that all javadoc did all those years ago?
    - eternalban 6y ago
      Still does. Example: https://docs.oracle.com/en/java/javase/11/docs/api/index.html https://docs.oracle.com/en/java/javase/11/docs/api/index.htm... https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/nio/file/package-info.java https://github.com/openjdk/jdk/blob/master/src/java.base/sha... I think Gophers borrowed the idea of generating docs from header/source comment from Java. Java itself spec'd it in 1996. Where did Gosling and friends got the idea, I don't know. It may have precedents either in Sun's other languages or possibly an earlier precedent. Some relatively minor differences: In Go, documentation is externalized IFF top level code (function, type, var defs and declarations) immediately follows a comment line. In Java comments have two forms. The comment form using two asterisks is externalized. In Go comments (iirc) support some very basic text styling (of the generated doc). In Java externalized comments can use a basic set of markup ala HTML. This includes comment level hyperlinks to javadocs of referenced elements. Having used both languages rather extensively, Go's approach lends itself to CLI usage. Java provides richer markup and hyperlinks via java and is much better for someone who wants to explore the API via documentation.
      - gostsamo 6y ago
        Isn't emacs the first to do it? I have the vague memory of it self-advertising as self-documenting.
        eternalban 6y ago
        I don’t know about emacs but your comment prompted a search for Smalltalk-80 docs: http://stephane.ducasse.free.fr/FreeBooks/BlueBook/Bluebook.pdf http://stephane.ducasse.free.fr/FreeBooks/BlueBook/Bluebook.... On page 309 the `Metaclass Protocol` is defined. This class has a property “comment” [with value semantic of] ‘commentString’. So possibly they got it from Smalltalk. Don’t know.
  - scns 6y ago
    In Elixir the code inside comments is tested so it cant go stale.
    - Cthulhu_ 6y ago
      In Go the example code is (can be) part of the source as well, and shown as executable in the documentation viewer: https://blog.golang.org/examples https://blog.golang.org/examples
      - jerf 6y ago
        Arguably another feature that could use some shouting from the rooftops. I see it fairly rarely from public packages, and it's something a lot of public packages really ought to have. Little example snippets for specific functions are great, but round-trip, top-to-bottom examples of using your code base are very useful.
  - Thaxll 6y ago
    Most likely people don't know where the doc is, Go documentation for module is excellent: https://github.com/golang/go/wiki/Modules https://github.com/golang/go/wiki/Modules
    - mfer 6y ago
      A big problem with this documentation is that it's not with the rest of the docs and isn't found in search on golang.org. The docs provide enough to make modules work but are far from excellent in terms of docs. These docs also highlight things about how the Go team thinks. For example, if a project is versioned at v2 or later they recommend incrementing the major version of your code when adopting modules. Instead of modules being support tooling for the app it's designed and thought of as important as a major version change.
    - saturn_vk 6y ago
      But that's not the go modules documentation. That's just some wiki put together with duct tape and good intentions. The real documentation is here: https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more https://golang.org/cmd/go/#hdr-Modules__module_versions__and... and is identical to the output of `go help modules`
  - jtdev 6y ago
    Code never lies, documentation/comments/etc. sometimes do.
    - bxparks 6y ago
      Code can be wrong. Documentation/comment can explain what the code was supposed to do.
      - bostik 6y ago
        > was There's the important bit. Code can be wrong, but it gets less out of date than its comments.
  - jrockway 6y ago
    go doc is one of my favorite programming tools. You can get information about any function from the command-line, without having to open a browser or manually grep for what you're looking for. How does strings.Index work? "go doc strings.Index" func Index(s, substr string) int Index returns the index of the first instance of substr in s, or -1 if substr is not present in s. It even tells you what import statement to use to get the version of the code that you're reading the documentation about. It isn't perfect, but it's my favorite documentation tool and would be my number one reason against switching to another language. (For example, I would kill for this in Typescript.)
    - literallycancer 6y ago
      godoc is not go doc
- dmitriid 6y ago
  It's not only that the documentation is quite bad. It's also the rest of go. Here's what it spews when you specidy a dependency incorrectly Dependency: github.com/robfig/cron v3.0.1 go's handling of this: require github.com/robfig/cron: version "v3.0.1" invalid: module contains a go.mod file, so major version must be compatible: should be v0 or v1, not v3 Right. What? I solved it by looking through this issue that comes at the top of Google Search for it: https://github.com/golang/go/issues/35732 https://github.com/golang/go/issues/35732
- xzel 6y ago
  I come to judge language's quality of documentation almost solely on their documentation around local and module imports. If they don't provide clear documentation on that subject, which is paramount imo to getting past simple scripts, then the rest of their higher level documentation is probably lacking. I've been upgrading some personal Go projects I have to a more modern folder layout (pkg, internal, etc. vs. src folder) and I really found their documentation lacking. Rust for example, in their imports, is so so, but I've found way better external examples for imports in Rust than Go.
  - deleted 6y ago
    [deleted]
alexruf 6y ago
The Go Team has actually communicated this matter extensively through your blog (https://blog.golang.org/v2-go-modules https://blog.golang.org/v2-go-modules).
- brabel 6y ago
  This is mentioned in the article. The problem is that the official solution, as mentioned in the article, seems really bad... you just have to copy the whole project into the `v2` folder, then do the same for a future `v3` and so on... it kind of goes against normal practices of using version control tools for this kind of thing, where each version would be in a different branch, not folder. Not to mention having duplicate files like this can make finding files in your editor a bit confusing as well. Though I like how Go solved dependency management and dared to go on a different path than almost every other package manager, this particular solution seems to go a bit too far?!
  - jimktrains2 6y ago
    > kind of goes against normal practices of using version control tools for this kind of thing, where each version would be in a different branch, not folder. I mean, it goes against how got works. SVN does the branch-in-a-folder thing, so it's not anathema to "version control". That said, as for branch-in-a-folder, "thanks, I hate it".
  - rossmohax 6y ago
    Copying into versioned folder is probably how large monorepos work. If so, then it is another example of Google influence on Go even though core team denies it.
  - kardianos 6y ago
    No, you just tag a v2 release like "git tag v2.0.0" and ensure your module path has a /v2. That's not hard.
    - donatj 6y ago
      As the post states, both are valid, but this method is not backwards compatible.
compsciphd 6y ago
I don't understand whats so hard at making it easy, if one just use source repo tags (which in go module land you should be using). 1) your go.mod should include the version 2) the module itself always assumes its the version of it's go.mod file 3) to release a version you just tag it based on the version in the go.mod file (with later semantic versions being considered the best and what go will use by default) so module authors have to do. so module authors dont have to do anything special besides include the version in the go.mod file (and keep it up to date) + tag release appropriately in a way that keeps to go's compatibility promise. 4) when you import a module, you specify what version you want (with a default assumption of v1) so your go.mod would either include the module or modules/v2... 5) your imports in actual .go files would not have to include the /v2 as they would know based on the go.mod file. you as the end user of the module will follow v0/v1 tag for as long as it exists, but will have to modify your go.mod to use v2 if you want to switch. does my conception not make sense?
- mcdee 6y ago
  The point of having separate imports for separate versions is to allow for use of (for example) both v1 and v2 of a module within the same codebase, if required. Your idea would force a single version, which means that would no longer be possible.
  - compsciphd 6y ago
    ok, reasonable, though it would seem that one could do both. i.e. for those that dont want to include multiple versions, be easy, for those that need them, then you have extra complexity.
- simiones 6y ago
  > if one just use source repo tags I've never been able to find docs on what exact format those should have. One of the many problems with Go modules.
  - llimllib 6y ago
    > Modules must be semantically versioned according to semver, usually in the form v(major).(minor).(patch), such as v0.1.0, v1.2.3, or v1.5.0-rc.1. The leading v is required. If using Git, tag released commits with their versions https://github.com/golang/go/wiki/Modules#modules https://github.com/golang/go/wiki/Modules#modules (I find this reference exceedingly helpful, and wish it were somewhere more official than the `golang/go` wiki, but c'est la vie)
fnoof 6y ago
A consequence of the complexity is that some devs stay below v2 forever because it’s easier. Backwards incompatible changes pile up in v0.x’s. It’s then really difficult for consumers to install two versions of the same package, which can happen with diamond dependencies.
- mfcl 6y ago
  Do you have an example of such package? I've used Go modules for years and never encountered that.
  - durbatuluk 6y ago
    someone cited kubernetes above
deleted 6y ago
[deleted]
simiones 6y ago
> Some projects like GORM sidestep the issue entirely by tagging their 2.0 releases as far flung 1.x releases. That something of a solution, but smells terrible. Amusingly, the Google protobuf team did a similar thing for the Go protobuf bindings. The new, backwards-incompatible `protobuf` module started with version 1.20, while the old module is at 1.4. If you've followed this, you know I've omitted one detail - they actually changed the name of the module as well, from `github.com/golang/protobuf` to `google.golang.org/protobuf`. Because, of course, Golang modules names are actually URLs, not identifiers like in every other package manager out there.
- xyzzy_plugh 6y ago
  > Because, of course, Golang modules names are actually URLs, not identifiers like in every other package manager out there. They don't have to be URLs, though. What's the difference between `google.golang.org/protobuf` and `org.golang.google.protobuf`?
  - simiones 6y ago
    As far as I know, the difference is that the first one will work with `go mod`, while the second one won't work, unless you have a replace directive. As far as I know, `go mod` will simply connect to https://$MODULE_NAME https://$MODULE_NAME and retrieve metadata and contents from there.
  - a1369209993 6y ago
    > What's the difference between `google.golang.org/protobuf` and `org.golang.google.protobuf`? There isn't one, because that's also a (mangled) URL, namely "http://protobuf.google.golang.org/" http://protobuf.google.golang.org/", and should be rejected for the same reasons. A identifier would be "protobuf"; note the lack of any reference to the DNS domain.
    - grey-area 6y ago
      Do that and soon you're going to have to introduce namespaces, and you're back to something that looks like urls. People use these longer import identifiers because they avoid polluting the global namespace, which be reserved for very important packages (in the case of go the std lib).
rezeroed 6y ago
What a mess. Starting with the GOPATH. This is far worse than the lack of generics which has had much airtime. For all the historical credits Pike and Thompson have, I don't know what they're doing here, it's making them look like buffoons; maybe they're not involved with this. I've mostly abandoned Go because of this ongoing red flag mess. Is there an end goal? The "we're watching the community" sounds like a weak excuse.
- _wldu 6y ago
  Ken has been retired for years.
fourseventy 6y ago
I knew about this rule and I still got bitten by it last week when I was trying to upgrade a package. It's one of the many things I hate about go modules. As I understand it the reason for this design decision was to allow a project to use multiple versions of the same dependency at once. To me this seems like a pretty rare edge case. They burdened the 99% of users with weird V2 syntax so that 1% of users could handle this edge case (which could have been handled by other means).
- xyzzy_plugh 6y ago
  It's not really an edge case, it's sort of to the core of how modules work, and allows a whole slew of issues to be elided. See https://research.swtch.com/vgo-import https://research.swtch.com/vgo-import for more background and examples. > They burdened the 99% of users with weird V2 syntax so that 1% of users could handle this edge case (which could have been handled by other means). Really the only burden lands on package authors who are making breaking changes. And really, the path provided is so much easier than actually making breaking changes for your users. Everyone wins.
  - grey-area 6y ago
    > Really the only burden lands on package authors who are making breaking changes. And really, the path provided is so much easier than actually making breaking changes for your users. Everyone wins. The burden falls on every module developer who wants to use a version number above 1.x. That is not congruent with developers who make breaking changes as you seem to assume. There's an assumption here about v2 meaning breaking changes that not even Go 2 itself may adhere to (as it may well be more of a 'new features' version number than a 'breaking changes' version number increment).
    - geodel 6y ago
      > The burden falls on every module developer who wants to use a version number above 1.x. And this is fine as it gives module developers an opportunity to make breaking changes which they wanted for quite some time. It makes much clearer for module user as opposed to Java where one does not know which module will blow up code was it 2.11 or may be 2.13.2 etc. > There's an assumption here about v2 meaning breaking changes that not even Go 2 itself may adhere to If anything Go is trying to be even more conservative that they are avoiding breaking changes in 2.0 as opposed to being rash making breaking changes in 1.x. I see no problem with that.
blablabla123 6y ago
I don't see any big problem. There is a blog post on the Golang website explaining everything. (My only complaint is there is no cheatsheet on golang.org for this) Also it's actually quite nice that it's possible to keep all maintained - and possibly incompatible - versions on the same branch without the overhead of a 3rdparty acting as package curator/distributor.
kstenerud 6y ago
This is why I never use anything but "1" as the first number in my go modules. The second number is the major (breaking changes) version field, and the third number is the minor field (non breaking changes). These are just the kinds of things you put up with when writing go code and need to work around bad designs of the language and ecosystem.
cybrexalpha 6y ago
Go has a bigger tooling problem. Build systems are hard, and Go has decided it doesn't want to be hard so it's ignored anything other than the "happy path". I wish other Go dependency tools still existed.
kortex 6y ago
What I don't understand is why they chose to put the major version as part of the path part of the uri (/), as opposed to some other separator, e.g. fragment (#). Especially since it's optional. You can always convert "example.com/foo#v2" into the "/v2" if needed, but if you see "foo/v8" is that a repo "foo" version major v8, a subfolder of "foo" named "v8" or a repo "v8" in the project "foo"?
- CHY872 6y ago
  It’s traditional at least with web browsers to not send the fragment to the server. Query param would work, but implies your routes don’t change much between major versions.
  - kortex 6y ago
    > It’s traditional at least with web browsers to not send the fragment to the server. Ah, TIL, thanks! I meant more "why must the import syntax use specifically this scheme to denote versions". I guess backwards compat?
    - yencabulator 6y ago
      Import paths are used to figure out where to fetch the source code from. To simplify a bit, if you see `example.com/foo/bar`, it works because there's an appropriate meta tag at https://example.com/foo/bar?go-get=1 https://example.com/foo/bar?go-get=1 . The import paths are pretty directly used as URLs.
marcus_holmes 6y ago
I'm a huge fan of Go, and not a fan of dependencies at all, so I tend to vendor everything. Go mod just looks like a total mess from the outside. To be honest I (like most gophers I've talked to) never really understood why dep wasn't adopted as the official solution for this. I'm resisting converting my projects to modules until there's a clear and definite advantage to it. So far, none.
- Thaxll 6y ago
  Dep was crap and ultra slow.
- dewey 6y ago
  > I'm resisting converting my projects to modules until there's a clear and definite advantage to it. So far, none. Go modules are not going to go away so what benefits will waiting as long as possible give you? I'm not a big fan but it's fine in day to day life and the most common issue for me is that some indirect dependency is having issues which is usually solved with some "replace" lines in the go mod file. Not ideal but also not too annoying.
  - marcus_holmes 6y ago
    Well the main benefit is not wasting time dealing with it ;) I know I'll have to do it one day. Hopefully that day will be after they've brought more sanity to it.
- CamouflagedKiwi 6y ago
  I tried dep for a project of mine and could never even get it to the point of doing anything useful. Conversely go mod took about three minutes to migrate to and was super straightforward. I am very happy that they did not adopt dep, it was in no way ready (and I don't think it was going to significantly improve given more time).
  - marcus_holmes 6y ago
    True, there were problems with it, it wasn't perfect by any means. I guess the main complaint at the time was that it appeared to be "not invented here" syndrome. The community was converging on dep, and then the Go team decided to do something different, for reasons that weren't completely clear. It kinda rammed home the reality that Go is not a community project, it's a Google product controlled by a small team. That definitely has benefits and good points, but it doesn't match everyone's expectations.
shp0ngle 6y ago
> Even as a seasoned developer, I found this write-up somewhat impenetrable As is the usual case with these write-ups
ausjke 6y ago
This is where "technical writer" profession shines, I know companies who have technical writing teams do excellent job there, but if we leave it for busy coders, that's what we got in the end.
sempron64 6y ago
2-3 years ago I was working on a large project in Go with many dependencies. We moved in the span of 12 months from go get, to glide, to dep, and when vgo was announced I realized I was on a pointless treadmill, despite community assurances that the current package manager is "blessed" and will be maintained. I then decided that Go is actually quite unstable and is not yet ready for enterprise projects, and recommend against it despite the fact that I really like and agree with the design philosophy of the language.
- kardianos 6y ago
  It would have been best to have skipped glide and dep. Sorry. The Go the language is stable, and Go modules is stable and is the future. Everything, including new features, is based on Go modules. That isn't changing.
- wrs 6y ago
  Unlike most ecosystems, just because a problem is obviously important doesn’t mean the Go team will solve it in a hurry. They don’t like to change their minds once they commit (exactly the property you’re looking for!). So Go is a very stable, sometimes a bit irritatingly stable :), core surrounded by continuous experiments. Once rsc or equivalent has blessed something, you can rely on it being that way for a long time. Before that, hedge your bets.
grey-area 6y ago
I'm still really not clear on why the Go team thought this was a good idea. It makes no sense to me to try to impose this rule on the ecosystem - the costs (confusion, multiple urls, multiple ways of doing it) vastly outweigh the benefits (multiple versions in use at once), and if a particular project wishes to use this rule, they could do so, without imposing it on the thousands of packages which might want to use higher numbers to indicate large feature releases or are already using higher numbers. In theory a larger number in semantic versioning indicates breaking changes, in practice small breaking changes are made all the time, it's a matter of degree and the context of a given project, and people also use version numbers for marketing or indicating lots of new features - a good case in point being Go 2.0 which may well be backwards compatible to Go 1.0. That makes this assumption in their blog false: > By definition, a new major version of a package is not backwards compatible with the previous version.[0] That's not the only, or even the main, reason projects use higher version numbers. I imagine lots of people will keep their version forever in the 1.x range to work around this rule. I too believe Go should drop this requirement before it is too late. 0. https://blog.golang.org/v2-go-modules https://blog.golang.org/v2-go-modules
- drvd 6y ago
  There is a difference between a software (executable) and a library. Semver is overrated but at least common and well understood with clear semantics. And no, Go 2 probably will be Go v1.23.
cap10morgan 6y ago
I see a lot of comments in here about the v2+ rule’s reason for existing being to allow importing multiple major versions of a module. That’s not it at all. As the blog post (which was easily Googled by me and explains things quite well IMO) states: “If an old package and a new package have the same import path, the new package must be backwards compatible with the old package.” They’re trying not to break compatibility until a downstream developer explicitly opts in to that breakage. The simplest way to do that is to give the module a new name. And the simplest way to do that is append the major version to the name. Here is the post: https://blog.golang.org/v2-go-modules https://blog.golang.org/v2-go-modules For more background on this principle, I recommend Rich Hickey’s Clojure/conj keynote “Spec-ulation” from 2016: http://blog.ezyang.com/2016/12/thoughts-about-spec-ulation-rich-hickey/ http://blog.ezyang.com/2016/12/thoughts-about-spec-ulation-r...
- giancarlostoro 6y ago
  I swear I remember reading this somewhere in the Go docs as well but only found it in the FAQ: https://golang.org/doc/faq#get_version https://golang.org/doc/faq#get_version I think this is a reasonable enough approach. You see it in Debian for dependencies so I'm okay with it for Go. It outlines clearly what your dependencies are.
- ubertaco 6y ago
  That's weird, how then do other languages manage to solve importing packages/modules cleanly without requiring that the name of the package include the version number, while still allowing developers to specify exactly which version of the module they want to use? Surely someone else has solved this problem before, like Rust's Cargo, or Java's Maven, or JS's NPM, or Python's PIP, Ruby's Gems, or... Unlike in every single one of those other languages, this Golang decision means that you can't pin to a specific minor version, or a specific patch version (in the sense of MAJOR.MINOR.PATCH). It's very in-character for the Golang team though: they value simplicity-for-themselves no matter what the complexity-for-you cost is.
  - saturn_vk 6y ago
    I'm not aware of maven allowing me to use multiple major-version libraries in the same package.
kardianos 6y ago
In the go ecosystem, packages are imported per file. Go is designed for large systems to evolve over time. You need to be able to refer to v1 and v2 at the same time. Thus the "/v2" requirement. Yes, it is different. Stop complaining, do it, and it just works. Here is the tutorial. 1. Modify go.mod package path to include the "/v2" element. 2. Commit. 3. Tag as "v2.0.0" 4. Push tag. I'm not seeing what is too confusing about releasing. To consume, on packages that you want it, import the new package. Done. Being able to import multiple versions without conflict has been extremely helpful when I've needed new features in one version, but didn't want to update (at the same time time) all the other places. I really don't get why people don't like it, other then it is different then what they expect elsewhere.
- jacquesm 6y ago
  It's actually pretty much the only consistent and guaranteed backwards compatible way to deal with such changes.
  - barrkel 6y ago
    There's no guarantee of backwards compatibility without limiting of forward functionality. The trade-off can't be escaped. If you can make do without any persistent state, or global or externally visible resources (network connections, configuration files, etc.), then you're OK. But that's a limitation.
    - roskilli 6y ago
      Modules that rely on global state for anything other than memory pooling or what have you should be avoided. It’s a lot more testable and clean to return a high level data structure that contains any of that state you would have held before globally in the module and have that be the “context” or just the parent data structure to any others that are spawned. Global state makes thing impossible like parallel unit tests that all use another module, or changing things like “MaxConcurrency” in a way that is synchronized across goroutines that might already be calling into the third party module. My 2c.
draw_down 6y ago
> I brought the issue up at my local Go meetup, and no one had ever heard about the rule. They were very skeptical of me. This resonates with my experience writing Golang (plus Bazel) this year. Anything that isn’t working, you must have just messed up and did something wrong. It can’t be that something is actually broken here. Even in this comment thread, “stop complaining and just do it.”
- throw_m239339 6y ago
  This kind of dismissive behaviour have been in the Go community since the beginning, unfortunately. It's the product of Rob Pike's behaviour and his rejection of "complexity". But you can't just dismissing complexity by deeming it irrelevant, it's there. Ian Lance Taylor is probably the only person in the Go team that has a more pragmatic approach to things. The rest is a bunch of deaf people led by the blinds and a few vocal gatekeepers outside Google that will publicly humiliate you if you stray away from the current orthodoxy.
  - draw_down 6y ago
    > But you can't just dismissing complexity by deeming it irrelevant, it's there. This is my sense as well- by refusing to allow complexity in one area, you often just end up forcing it into another area. Moving the toothpaste around the tube. Overall Go was a bad fit for me in ways that come down to differences of opinion, and that's nobody's fault. But I did also come across the dismissiveness you mention. Overall I feel there is a callousness to the community, not just interpersonal interactions but the ways that certain things aren't provided for and tough shit if you don't like it.
konart 6y ago
Never realised this was a problem. How so? Tag your version v2.0.1, import it as .../v2 in project - done. You don't even have to read docs or blog posts to understand it, just look at the source code of a new project, notice the ../vN part, see the corresponding tag in repo and do the math.
saturn_vk 6y ago
The whole premise that the "communication of this rule has been weak" is rather weak. It has been hammered in throughout the development process of modules, and is part of the official documentation. If the author has missed the numerous blog posts throughout the development, and hasn't read the official documentation, well, I'm not sure we can fault someone else for that. Luckily, it appears that the community has picked up the memo somewhere, as there are quite a few v2+ modules around.
enos_feedler 6y ago
"I have seen many very large projects including Google owned projects get it wrong." As an ex-Googler I can tell you that this above statement can be more broadly applied across _many_ developer facing Google products (most notably for me, Android frameworks and libraries). Google's apps don't use the guidance and recommendations we gave to external Android devs. I wonder if there is something cultural about Google not even being able to coordinate changes and improvements within the org itself.
deklerk 6y ago
Hiyo, I'm one of the co-authors of https://blog.golang.org/v2-go-modules https://blog.golang.org/v2-go-modules. One of the takeaways from this article was, "there needs to be more documentation", and I think I can speak to that: First, thanks for the feedback. We also want there to be a loooot more documentation, of all kinds. To that end, several folks on the Go team and many community members have been working on Go module documentation. We have published several blog posts, rewritten "How to write Go code" https://golang.org/doc/code.html https://golang.org/doc/code.html, have been writing loads of reference material (follow in https://github.com/golang/go/issues/33637 https://github.com/golang/go/issues/33637), have several tutorials on the way, are thinking and talking about a "cheatsheet", are building more tooling, and more. If you have ideas for how to improve modules, module documentation, or just want to chat about modules, please feel free to post ideas at github.com/golang/go/issues or come chat at gophers.slack.com#modules.
- dilap 6y ago
  I noticed that a lot of the documentation seems to combine "how this works" with "why this works the way it does"; the why is great when you're interested in diving deeper, but it's frustrating when all you're interested in is the how. For example, the linked blog post spends a lot of time talking about diamond dependencies and other package managers. This is just noise that gets in the way when you're trying to figure out how does this work? If you did want to combine both in a single reference doc, I would move the why out into separate, skippable sections. When I first was learning Go, I was really impressed by how easy it was to understand the language just by reading the spec. I've found the opposite to be true for Go modules. (Which also, as near as I can tell, doesn't have a spec, but just various scattered blog posts, for various different iterations of the idea.)
  - deklerk 6y ago
    Thanks, this is good feedback. In general, I reckon having a variety of documentation is great, so also would like more of "how" documentation. I believe the tutorials that are in the work take more of the "short and sweet" approach, which should help with this.
dilap 6y ago
I wonder if they should've just required v1 or v0 when using Go modules†, so that v2 and above didn't seem like a red-headed step child / surprising / non-obvious by comparison. The basic idea of having separate import paths for non-compatible versions of a project seems, in fact, really good; an elegant solution to a tricky problem. It certainly seems much nicer than secretly using multiple versions of a package behind your back (especially when you consider that if this happens and a package uses mutable internal state, you could actually get broken behavior). I guess for me the takeaway is in aggregate, people will be maximally lazy. You need to make the correct thing be the easiest thing. (†Also release a patch to older versions of Go to just ignore v1/v0, and error for above, so you can have source compatible w/ both module and non-module Go.)
donatj 6y ago
Author of the post here. Really neat to see something I posted last night on HN the very next morning. With a night to sleep on it, I think requiring `/v0` and `/v1` would have been the better option, that way you'd hit the issue right away rather than years into your project. Teach things early. I've updated the post with a note of this idea. I think that's likely not something they could add at this point however. I don't even mean to be critical of the v2-ing itself, I just think it's entirely non-obvious.
earthboundkid 6y ago
I think people are too quick to declare v1. All of my modules are (deliberately!) v0 because v1 means "I commit to never breaking this API." Most Go modules should be v0.year.minor, eg v0.20.5 for the fifth release this year. You should only be using v1 if you have a lot of users, a set of core maintainers, etc. etc.
- fnoof 6y ago
  I feel semver combines two separate ideas into the meaning of v1: By labeling as v1 you're: - communicating the package is stable and will be maintained - communication that breaking changes will now increment the major version (saying nothing about stability, backported bug fixes, or any maintenance) I wish they were not combined. Incrementing major versions frequently on early stage projects would make them work better with package managers, but without the expectation of maintenance from the community. Not incrementing major versions makes it hard for package managers to make sensible decisions.
- perlgeek 6y ago
  How does that make anything better for the user? They know they cannot rely on a stable API, which means they have to carefully read the diff for every single upgrade, so saving no work over explicitly picking a major version.
  - earthboundkid 6y ago
    The advantage is truth in advertising.
alexandrerond 6y ago
Modules had been in discussion for like years. Modules HowTo has been around from day one and it tells about this very clearly. Modules adoption rolled over a couple of releases which gave all the pointers they could.. Any minimally maintained project uses modules already. It is a must for any Go programmer to be skilled at managing modules because otherwise things just break. And yet it seems some people have lived under rocks. People don't have time to RTFM, but apparently lots of time to complain and write about things they should have known with minimal due diligence around the tech they use. shrug
apta 6y ago
It's absurd that golang didn't take established solutions such as Java's packaging system, and had to re-invent the wheel in a completely suboptimal way.
patatsuh 6y ago
The dep fans try to strike back again?? I hoped we are done with this...
wallyqs 6y ago
It has gotten much better but in our project we got bitten by this. I gave a talk about it at GoSF last year if looking for some Go v2 modules schadenfreude :) https://www.youtube.com/watch?v=8xAaZDSDWOc https://www.youtube.com/watch?v=8xAaZDSDWOc

217 points by nitinreddy88 6y ago

Re: "They need at the very least a section of the documentation that lays it out clearly and in layman's terms." As a Go outsider who's trying to get his feet dipped in the lake, trying to use modules has felt... uneasy? I tried to look up the official resources on the matter and found the official blog posts to be lacking concrete and concise examples on the usage; documentation feels like it's written for people who work on the project and already digested the unit tests as examples. As for the case of v2, similarly messy and undocumented. I also appreciate author's call for action on warning messages in tooling. Wasn't the unified tooling supposed to be a selling point for Go? It just doesn't feel good to use.

throwawayinfo 6y ago
Poor documentation seems to be a Google culture thing. I suspect that's why godoc is literally just source code comments wrapped in HTML.
- dan_can_code 6y ago
  I have found this to be almost universal when it comes to reading / writing documentation, not just Google. I have come to the understanding that to write good documentation, you should write it as soon as you have learned it, or at least, try and explain in a similar way in the way you learned (which is not easy at all). I feel the issue stems from abstraction. Once you become somewhat an expert in a topic or develop a deeper understanding of said topic, you automatically abstract away the information that lead you to that understanding in the first place. Otherwise you end up reading or writing documentation with a bunch of assumptions for knowledge and understanding, which are not only not stated but may not be available to the person trying to get to grips with the technology. A bit of a digression from the post, but I don't feel this is just a Google thing.
  - scns 6y ago
    There is a name for it: The experts blind spot
    - k00b 6y ago
      I’ve heard it called and prefer to call it the curse of knowledge
    - dan_can_code 6y ago
      Thanks for sharing this with me. I didn't know this was a widely known phenomenon. I'll take a look at the link to help avoid having these blind spots myself, thanks again!
    - dmitriid 6y ago
      Erlang's creator Joe Armstrong once said it like this (quoting from memory) "There's a moment when you go from not getting it to getting it, and you have about 24 hours when you remember both before and after, and that's when you have to write things down. After that you won't remember how it was not getting it"
  - virgilp 6y ago
    Rust has pretty decent documentation. My point is that it's possible/ we shouldn't settle for "that's the universal status quo, nothing can be done about it". Something _can_ be done. And we _should_ be demanding more - especially so from tech giants like Google.
    - dan_can_code 6y ago
      I absolutely agree. There is no reason to settle for anything less. Google certainly has the resources to fill the gap - recognising it would be the first step. Looking at the Go documentation, it seems they have feedback mechanisms for the features of Go, but no means of providing feedback on the actual documentation. Rust however have both Github contributions, as well as a channel on discord specifically for documentation discussions. I would assume this may contribute to the lack of satisfactory documentation for Go.
- thefounder 6y ago
  I can't disagree more.
- rob74 6y ago
  I've seen projects with much worse documentation than Go. And I think it's unfair to blame the quality of the documentation on godoc, which is actually a pretty powerful documentation tool, and the fact that it's included with the language is a great advantage. As with other aspects of Go, they may have taken the "keep it simple" principle a bit too far here too (e.g. godoc could detect and highlight if you mention the names of a method's parameters in a doc comment), but what's there is very solid. Of course, as with any documentation tool, you have to actually use it by writing good doc comments (end examples, and additional documentation files etc.) to make it useful, but that's not the tool's fault...
  - nobleach 6y ago
    Yeah, OCaml's docs are FAR worse as they're often just type signatures. I'm also not a fan of Oracle's Java docs as they do explain what each method does, but often in a rather opaque way. Where Elixir and Kotlin get things right, is they tend to also give quick examples of how one might use any given method.
- clinta 6y ago
  Isn't that the case with most languages now? Pydoc and rustdoc do the same thing.
  - trumpeta 6y ago
    Last time I checked it was not possible to actually use godoc to render the doc, it just started an internal server. That made it super hard to publish docs internally.
- jimktrains2 6y ago
  Isn't that all javadoc did all those years ago?
  - eternalban 6y ago
    Still does. Example: https://docs.oracle.com/en/java/javase/11/docs/api/index.html https://docs.oracle.com/en/java/javase/11/docs/api/index.htm... https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/nio/file/package-info.java https://github.com/openjdk/jdk/blob/master/src/java.base/sha... I think Gophers borrowed the idea of generating docs from header/source comment from Java. Java itself spec'd it in 1996. Where did Gosling and friends got the idea, I don't know. It may have precedents either in Sun's other languages or possibly an earlier precedent. Some relatively minor differences: In Go, documentation is externalized IFF top level code (function, type, var defs and declarations) immediately follows a comment line. In Java comments have two forms. The comment form using two asterisks is externalized. In Go comments (iirc) support some very basic text styling (of the generated doc). In Java externalized comments can use a basic set of markup ala HTML. This includes comment level hyperlinks to javadocs of referenced elements. Having used both languages rather extensively, Go's approach lends itself to CLI usage. Java provides richer markup and hyperlinks via java and is much better for someone who wants to explore the API via documentation.
    - gostsamo 6y ago
      Isn't emacs the first to do it? I have the vague memory of it self-advertising as self-documenting.
      - eternalban 6y ago
        I don’t know about emacs but your comment prompted a search for Smalltalk-80 docs: http://stephane.ducasse.free.fr/FreeBooks/BlueBook/Bluebook.pdf http://stephane.ducasse.free.fr/FreeBooks/BlueBook/Bluebook.... On page 309 the `Metaclass Protocol` is defined. This class has a property “comment” [with value semantic of] ‘commentString’. So possibly they got it from Smalltalk. Don’t know.
- scns 6y ago
  In Elixir the code inside comments is tested so it cant go stale.
  - Cthulhu_ 6y ago
    In Go the example code is (can be) part of the source as well, and shown as executable in the documentation viewer: https://blog.golang.org/examples https://blog.golang.org/examples
    - jerf 6y ago
      Arguably another feature that could use some shouting from the rooftops. I see it fairly rarely from public packages, and it's something a lot of public packages really ought to have. Little example snippets for specific functions are great, but round-trip, top-to-bottom examples of using your code base are very useful.
- Thaxll 6y ago
  Most likely people don't know where the doc is, Go documentation for module is excellent: https://github.com/golang/go/wiki/Modules https://github.com/golang/go/wiki/Modules
  - mfer 6y ago
    A big problem with this documentation is that it's not with the rest of the docs and isn't found in search on golang.org. The docs provide enough to make modules work but are far from excellent in terms of docs. These docs also highlight things about how the Go team thinks. For example, if a project is versioned at v2 or later they recommend incrementing the major version of your code when adopting modules. Instead of modules being support tooling for the app it's designed and thought of as important as a major version change.
  - saturn_vk 6y ago
    But that's not the go modules documentation. That's just some wiki put together with duct tape and good intentions. The real documentation is here: https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more https://golang.org/cmd/go/#hdr-Modules__module_versions__and... and is identical to the output of `go help modules`
- jtdev 6y ago
  Code never lies, documentation/comments/etc. sometimes do.
  - bxparks 6y ago
    Code can be wrong. Documentation/comment can explain what the code was supposed to do.
    - bostik 6y ago
      > was There's the important bit. Code can be wrong, but it gets less out of date than its comments.
- jrockway 6y ago
  go doc is one of my favorite programming tools. You can get information about any function from the command-line, without having to open a browser or manually grep for what you're looking for. How does strings.Index work? "go doc strings.Index" func Index(s, substr string) int Index returns the index of the first instance of substr in s, or -1 if substr is not present in s. It even tells you what import statement to use to get the version of the code that you're reading the documentation about. It isn't perfect, but it's my favorite documentation tool and would be my number one reason against switching to another language. (For example, I would kill for this in Typescript.)
  - literallycancer 6y ago
    godoc is not go doc
dmitriid 6y ago
It's not only that the documentation is quite bad. It's also the rest of go. Here's what it spews when you specidy a dependency incorrectly Dependency: github.com/robfig/cron v3.0.1 go's handling of this: require github.com/robfig/cron: version "v3.0.1" invalid: module contains a go.mod file, so major version must be compatible: should be v0 or v1, not v3 Right. What? I solved it by looking through this issue that comes at the top of Google Search for it: https://github.com/golang/go/issues/35732 https://github.com/golang/go/issues/35732
xzel 6y ago
I come to judge language's quality of documentation almost solely on their documentation around local and module imports. If they don't provide clear documentation on that subject, which is paramount imo to getting past simple scripts, then the rest of their higher level documentation is probably lacking. I've been upgrading some personal Go projects I have to a more modern folder layout (pkg, internal, etc. vs. src folder) and I really found their documentation lacking. Rust for example, in their imports, is so so, but I've found way better external examples for imports in Rust than Go.
- deleted 6y ago
  [deleted]

alexruf 6y ago

The Go Team has actually communicated this matter extensively through your blog (https://blog.golang.org/v2-go-modules https://blog.golang.org/v2-go-modules).

brabel 6y ago
This is mentioned in the article. The problem is that the official solution, as mentioned in the article, seems really bad... you just have to copy the whole project into the `v2` folder, then do the same for a future `v3` and so on... it kind of goes against normal practices of using version control tools for this kind of thing, where each version would be in a different branch, not folder. Not to mention having duplicate files like this can make finding files in your editor a bit confusing as well. Though I like how Go solved dependency management and dared to go on a different path than almost every other package manager, this particular solution seems to go a bit too far?!
- jimktrains2 6y ago
  > kind of goes against normal practices of using version control tools for this kind of thing, where each version would be in a different branch, not folder. I mean, it goes against how got works. SVN does the branch-in-a-folder thing, so it's not anathema to "version control". That said, as for branch-in-a-folder, "thanks, I hate it".
- rossmohax 6y ago
  Copying into versioned folder is probably how large monorepos work. If so, then it is another example of Google influence on Go even though core team denies it.
- kardianos 6y ago
  No, you just tag a v2 release like "git tag v2.0.0" and ensure your module path has a /v2. That's not hard.
  - donatj 6y ago
    As the post states, both are valid, but this method is not backwards compatible.

compsciphd 6y ago

I don't understand whats so hard at making it easy, if one just use source repo tags (which in go module land you should be using). 1) your go.mod should include the version 2) the module itself always assumes its the version of it's go.mod file 3) to release a version you just tag it based on the version in the go.mod file (with later semantic versions being considered the best and what go will use by default) so module authors have to do. so module authors dont have to do anything special besides include the version in the go.mod file (and keep it up to date) + tag release appropriately in a way that keeps to go's compatibility promise. 4) when you import a module, you specify what version you want (with a default assumption of v1) so your go.mod would either include the module or modules/v2... 5) your imports in actual .go files would not have to include the /v2 as they would know based on the go.mod file. you as the end user of the module will follow v0/v1 tag for as long as it exists, but will have to modify your go.mod to use v2 if you want to switch. does my conception not make sense?

mcdee 6y ago
The point of having separate imports for separate versions is to allow for use of (for example) both v1 and v2 of a module within the same codebase, if required. Your idea would force a single version, which means that would no longer be possible.
- compsciphd 6y ago
  ok, reasonable, though it would seem that one could do both. i.e. for those that dont want to include multiple versions, be easy, for those that need them, then you have extra complexity.
simiones 6y ago
> if one just use source repo tags I've never been able to find docs on what exact format those should have. One of the many problems with Go modules.
- llimllib 6y ago
  > Modules must be semantically versioned according to semver, usually in the form v(major).(minor).(patch), such as v0.1.0, v1.2.3, or v1.5.0-rc.1. The leading v is required. If using Git, tag released commits with their versions https://github.com/golang/go/wiki/Modules#modules https://github.com/golang/go/wiki/Modules#modules (I find this reference exceedingly helpful, and wish it were somewhere more official than the `golang/go` wiki, but c'est la vie)

fnoof 6y ago

A consequence of the complexity is that some devs stay below v2 forever because it’s easier. Backwards incompatible changes pile up in v0.x’s. It’s then really difficult for consumers to install two versions of the same package, which can happen with diamond dependencies.

mfcl 6y ago
Do you have an example of such package? I've used Go modules for years and never encountered that.
- durbatuluk 6y ago
  someone cited kubernetes above

deleted 6y ago

[deleted]

simiones 6y ago

> Some projects like GORM sidestep the issue entirely by tagging their 2.0 releases as far flung 1.x releases. That something of a solution, but smells terrible. Amusingly, the Google protobuf team did a similar thing for the Go protobuf bindings. The new, backwards-incompatible `protobuf` module started with version 1.20, while the old module is at 1.4. If you've followed this, you know I've omitted one detail - they actually changed the name of the module as well, from `github.com/golang/protobuf` to `google.golang.org/protobuf`. Because, of course, Golang modules names are actually URLs, not identifiers like in every other package manager out there.

xyzzy_plugh 6y ago
> Because, of course, Golang modules names are actually URLs, not identifiers like in every other package manager out there. They don't have to be URLs, though. What's the difference between `google.golang.org/protobuf` and `org.golang.google.protobuf`?
- simiones 6y ago
  As far as I know, the difference is that the first one will work with `go mod`, while the second one won't work, unless you have a replace directive. As far as I know, `go mod` will simply connect to https://$MODULE_NAME https://$MODULE_NAME and retrieve metadata and contents from there.
- a1369209993 6y ago
  > What's the difference between `google.golang.org/protobuf` and `org.golang.google.protobuf`? There isn't one, because that's also a (mangled) URL, namely "http://protobuf.google.golang.org/" http://protobuf.google.golang.org/", and should be rejected for the same reasons. A identifier would be "protobuf"; note the lack of any reference to the DNS domain.
  - grey-area 6y ago
    Do that and soon you're going to have to introduce namespaces, and you're back to something that looks like urls. People use these longer import identifiers because they avoid polluting the global namespace, which be reserved for very important packages (in the case of go the std lib).

rezeroed 6y ago

What a mess. Starting with the GOPATH. This is far worse than the lack of generics which has had much airtime. For all the historical credits Pike and Thompson have, I don't know what they're doing here, it's making them look like buffoons; maybe they're not involved with this. I've mostly abandoned Go because of this ongoing red flag mess. Is there an end goal? The "we're watching the community" sounds like a weak excuse.

_wldu 6y ago
Ken has been retired for years.

fourseventy 6y ago

I knew about this rule and I still got bitten by it last week when I was trying to upgrade a package. It's one of the many things I hate about go modules. As I understand it the reason for this design decision was to allow a project to use multiple versions of the same dependency at once. To me this seems like a pretty rare edge case. They burdened the 99% of users with weird V2 syntax so that 1% of users could handle this edge case (which could have been handled by other means).

xyzzy_plugh 6y ago
It's not really an edge case, it's sort of to the core of how modules work, and allows a whole slew of issues to be elided. See https://research.swtch.com/vgo-import https://research.swtch.com/vgo-import for more background and examples. > They burdened the 99% of users with weird V2 syntax so that 1% of users could handle this edge case (which could have been handled by other means). Really the only burden lands on package authors who are making breaking changes. And really, the path provided is so much easier than actually making breaking changes for your users. Everyone wins.
- grey-area 6y ago
  > Really the only burden lands on package authors who are making breaking changes. And really, the path provided is so much easier than actually making breaking changes for your users. Everyone wins. The burden falls on every module developer who wants to use a version number above 1.x. That is not congruent with developers who make breaking changes as you seem to assume. There's an assumption here about v2 meaning breaking changes that not even Go 2 itself may adhere to (as it may well be more of a 'new features' version number than a 'breaking changes' version number increment).
  - geodel 6y ago
    > The burden falls on every module developer who wants to use a version number above 1.x. And this is fine as it gives module developers an opportunity to make breaking changes which they wanted for quite some time. It makes much clearer for module user as opposed to Java where one does not know which module will blow up code was it 2.11 or may be 2.13.2 etc. > There's an assumption here about v2 meaning breaking changes that not even Go 2 itself may adhere to If anything Go is trying to be even more conservative that they are avoiding breaking changes in 2.0 as opposed to being rash making breaking changes in 1.x. I see no problem with that.

blablabla123 6y ago

I don't see any big problem. There is a blog post on the Golang website explaining everything. (My only complaint is there is no cheatsheet on golang.org for this) Also it's actually quite nice that it's possible to keep all maintained - and possibly incompatible - versions on the same branch without the overhead of a 3rdparty acting as package curator/distributor.

kstenerud 6y ago

This is why I never use anything but "1" as the first number in my go modules. The second number is the major (breaking changes) version field, and the third number is the minor field (non breaking changes). These are just the kinds of things you put up with when writing go code and need to work around bad designs of the language and ecosystem.

cybrexalpha 6y ago

Go has a bigger tooling problem. Build systems are hard, and Go has decided it doesn't want to be hard so it's ignored anything other than the "happy path". I wish other Go dependency tools still existed.

kortex 6y ago

What I don't understand is why they chose to put the major version as part of the path part of the uri (/), as opposed to some other separator, e.g. fragment (#). Especially since it's optional. You can always convert "example.com/foo#v2" into the "/v2" if needed, but if you see "foo/v8" is that a repo "foo" version major v8, a subfolder of "foo" named "v8" or a repo "v8" in the project "foo"?

CHY872 6y ago
It’s traditional at least with web browsers to not send the fragment to the server. Query param would work, but implies your routes don’t change much between major versions.
- kortex 6y ago
  > It’s traditional at least with web browsers to not send the fragment to the server. Ah, TIL, thanks! I meant more "why must the import syntax use specifically this scheme to denote versions". I guess backwards compat?
  - yencabulator 6y ago
    Import paths are used to figure out where to fetch the source code from. To simplify a bit, if you see `example.com/foo/bar`, it works because there's an appropriate meta tag at https://example.com/foo/bar?go-get=1 https://example.com/foo/bar?go-get=1 . The import paths are pretty directly used as URLs.

marcus_holmes 6y ago

I'm a huge fan of Go, and not a fan of dependencies at all, so I tend to vendor everything. Go mod just looks like a total mess from the outside. To be honest I (like most gophers I've talked to) never really understood why dep wasn't adopted as the official solution for this. I'm resisting converting my projects to modules until there's a clear and definite advantage to it. So far, none.

Thaxll 6y ago
Dep was crap and ultra slow.
dewey 6y ago
> I'm resisting converting my projects to modules until there's a clear and definite advantage to it. So far, none. Go modules are not going to go away so what benefits will waiting as long as possible give you? I'm not a big fan but it's fine in day to day life and the most common issue for me is that some indirect dependency is having issues which is usually solved with some "replace" lines in the go mod file. Not ideal but also not too annoying.
- marcus_holmes 6y ago
  Well the main benefit is not wasting time dealing with it ;) I know I'll have to do it one day. Hopefully that day will be after they've brought more sanity to it.
CamouflagedKiwi 6y ago
I tried dep for a project of mine and could never even get it to the point of doing anything useful. Conversely go mod took about three minutes to migrate to and was super straightforward. I am very happy that they did not adopt dep, it was in no way ready (and I don't think it was going to significantly improve given more time).
- marcus_holmes 6y ago
  True, there were problems with it, it wasn't perfect by any means. I guess the main complaint at the time was that it appeared to be "not invented here" syndrome. The community was converging on dep, and then the Go team decided to do something different, for reasons that weren't completely clear. It kinda rammed home the reality that Go is not a community project, it's a Google product controlled by a small team. That definitely has benefits and good points, but it doesn't match everyone's expectations.

shp0ngle 6y ago

> Even as a seasoned developer, I found this write-up somewhat impenetrable As is the usual case with these write-ups

ausjke 6y ago

This is where "technical writer" profession shines, I know companies who have technical writing teams do excellent job there, but if we leave it for busy coders, that's what we got in the end.

sempron64 6y ago

2-3 years ago I was working on a large project in Go with many dependencies. We moved in the span of 12 months from go get, to glide, to dep, and when vgo was announced I realized I was on a pointless treadmill, despite community assurances that the current package manager is "blessed" and will be maintained. I then decided that Go is actually quite unstable and is not yet ready for enterprise projects, and recommend against it despite the fact that I really like and agree with the design philosophy of the language.

kardianos 6y ago
It would have been best to have skipped glide and dep. Sorry. The Go the language is stable, and Go modules is stable and is the future. Everything, including new features, is based on Go modules. That isn't changing.
wrs 6y ago
Unlike most ecosystems, just because a problem is obviously important doesn’t mean the Go team will solve it in a hurry. They don’t like to change their minds once they commit (exactly the property you’re looking for!). So Go is a very stable, sometimes a bit irritatingly stable :), core surrounded by continuous experiments. Once rsc or equivalent has blessed something, you can rely on it being that way for a long time. Before that, hedge your bets.

grey-area 6y ago

I'm still really not clear on why the Go team thought this was a good idea. It makes no sense to me to try to impose this rule on the ecosystem - the costs (confusion, multiple urls, multiple ways of doing it) vastly outweigh the benefits (multiple versions in use at once), and if a particular project wishes to use this rule, they could do so, without imposing it on the thousands of packages which might want to use higher numbers to indicate large feature releases or are already using higher numbers. In theory a larger number in semantic versioning indicates breaking changes, in practice small breaking changes are made all the time, it's a matter of degree and the context of a given project, and people also use version numbers for marketing or indicating lots of new features - a good case in point being Go 2.0 which may well be backwards compatible to Go 1.0. That makes this assumption in their blog false: > By definition, a new major version of a package is not backwards compatible with the previous version.[0] That's not the only, or even the main, reason projects use higher version numbers. I imagine lots of people will keep their version forever in the 1.x range to work around this rule. I too believe Go should drop this requirement before it is too late. 0. https://blog.golang.org/v2-go-modules https://blog.golang.org/v2-go-modules

drvd 6y ago
There is a difference between a software (executable) and a library. Semver is overrated but at least common and well understood with clear semantics. And no, Go 2 probably will be Go v1.23.

cap10morgan 6y ago

I see a lot of comments in here about the v2+ rule’s reason for existing being to allow importing multiple major versions of a module. That’s not it at all. As the blog post (which was easily Googled by me and explains things quite well IMO) states: “If an old package and a new package have the same import path, the new package must be backwards compatible with the old package.” They’re trying not to break compatibility until a downstream developer explicitly opts in to that breakage. The simplest way to do that is to give the module a new name. And the simplest way to do that is append the major version to the name. Here is the post: https://blog.golang.org/v2-go-modules https://blog.golang.org/v2-go-modules For more background on this principle, I recommend Rich Hickey’s Clojure/conj keynote “Spec-ulation” from 2016: http://blog.ezyang.com/2016/12/thoughts-about-spec-ulation-rich-hickey/ http://blog.ezyang.com/2016/12/thoughts-about-spec-ulation-r...

giancarlostoro 6y ago
I swear I remember reading this somewhere in the Go docs as well but only found it in the FAQ: https://golang.org/doc/faq#get_version https://golang.org/doc/faq#get_version I think this is a reasonable enough approach. You see it in Debian for dependencies so I'm okay with it for Go. It outlines clearly what your dependencies are.
ubertaco 6y ago
That's weird, how then do other languages manage to solve importing packages/modules cleanly without requiring that the name of the package include the version number, while still allowing developers to specify exactly which version of the module they want to use? Surely someone else has solved this problem before, like Rust's Cargo, or Java's Maven, or JS's NPM, or Python's PIP, Ruby's Gems, or... Unlike in every single one of those other languages, this Golang decision means that you can't pin to a specific minor version, or a specific patch version (in the sense of MAJOR.MINOR.PATCH). It's very in-character for the Golang team though: they value simplicity-for-themselves no matter what the complexity-for-you cost is.
- saturn_vk 6y ago
  I'm not aware of maven allowing me to use multiple major-version libraries in the same package.

kardianos 6y ago

In the go ecosystem, packages are imported per file. Go is designed for large systems to evolve over time. You need to be able to refer to v1 and v2 at the same time. Thus the "/v2" requirement. Yes, it is different. Stop complaining, do it, and it just works. Here is the tutorial. 1. Modify go.mod package path to include the "/v2" element. 2. Commit. 3. Tag as "v2.0.0" 4. Push tag. I'm not seeing what is too confusing about releasing. To consume, on packages that you want it, import the new package. Done. Being able to import multiple versions without conflict has been extremely helpful when I've needed new features in one version, but didn't want to update (at the same time time) all the other places. I really don't get why people don't like it, other then it is different then what they expect elsewhere.

jacquesm 6y ago
It's actually pretty much the only consistent and guaranteed backwards compatible way to deal with such changes.
- barrkel 6y ago
  There's no guarantee of backwards compatibility without limiting of forward functionality. The trade-off can't be escaped. If you can make do without any persistent state, or global or externally visible resources (network connections, configuration files, etc.), then you're OK. But that's a limitation.
  - roskilli 6y ago
    Modules that rely on global state for anything other than memory pooling or what have you should be avoided. It’s a lot more testable and clean to return a high level data structure that contains any of that state you would have held before globally in the module and have that be the “context” or just the parent data structure to any others that are spawned. Global state makes thing impossible like parallel unit tests that all use another module, or changing things like “MaxConcurrency” in a way that is synchronized across goroutines that might already be calling into the third party module. My 2c.

draw_down 6y ago

> I brought the issue up at my local Go meetup, and no one had ever heard about the rule. They were very skeptical of me. This resonates with my experience writing Golang (plus Bazel) this year. Anything that isn’t working, you must have just messed up and did something wrong. It can’t be that something is actually broken here. Even in this comment thread, “stop complaining and just do it.”

throw_m239339 6y ago
This kind of dismissive behaviour have been in the Go community since the beginning, unfortunately. It's the product of Rob Pike's behaviour and his rejection of "complexity". But you can't just dismissing complexity by deeming it irrelevant, it's there. Ian Lance Taylor is probably the only person in the Go team that has a more pragmatic approach to things. The rest is a bunch of deaf people led by the blinds and a few vocal gatekeepers outside Google that will publicly humiliate you if you stray away from the current orthodoxy.
- draw_down 6y ago
  > But you can't just dismissing complexity by deeming it irrelevant, it's there. This is my sense as well- by refusing to allow complexity in one area, you often just end up forcing it into another area. Moving the toothpaste around the tube. Overall Go was a bad fit for me in ways that come down to differences of opinion, and that's nobody's fault. But I did also come across the dismissiveness you mention. Overall I feel there is a callousness to the community, not just interpersonal interactions but the ways that certain things aren't provided for and tough shit if you don't like it.

konart 6y ago

Never realised this was a problem. How so? Tag your version v2.0.1, import it as .../v2 in project - done. You don't even have to read docs or blog posts to understand it, just look at the source code of a new project, notice the ../vN part, see the corresponding tag in repo and do the math.

saturn_vk 6y ago

The whole premise that the "communication of this rule has been weak" is rather weak. It has been hammered in throughout the development process of modules, and is part of the official documentation. If the author has missed the numerous blog posts throughout the development, and hasn't read the official documentation, well, I'm not sure we can fault someone else for that. Luckily, it appears that the community has picked up the memo somewhere, as there are quite a few v2+ modules around.

enos_feedler 6y ago

"I have seen many very large projects including Google owned projects get it wrong." As an ex-Googler I can tell you that this above statement can be more broadly applied across _many_ developer facing Google products (most notably for me, Android frameworks and libraries). Google's apps don't use the guidance and recommendations we gave to external Android devs. I wonder if there is something cultural about Google not even being able to coordinate changes and improvements within the org itself.

deklerk 6y ago

Hiyo, I'm one of the co-authors of https://blog.golang.org/v2-go-modules https://blog.golang.org/v2-go-modules. One of the takeaways from this article was, "there needs to be more documentation", and I think I can speak to that: First, thanks for the feedback. We also want there to be a loooot more documentation, of all kinds. To that end, several folks on the Go team and many community members have been working on Go module documentation. We have published several blog posts, rewritten "How to write Go code" https://golang.org/doc/code.html https://golang.org/doc/code.html, have been writing loads of reference material (follow in https://github.com/golang/go/issues/33637 https://github.com/golang/go/issues/33637), have several tutorials on the way, are thinking and talking about a "cheatsheet", are building more tooling, and more. If you have ideas for how to improve modules, module documentation, or just want to chat about modules, please feel free to post ideas at github.com/golang/go/issues or come chat at gophers.slack.com#modules.

dilap 6y ago
I noticed that a lot of the documentation seems to combine "how this works" with "why this works the way it does"; the why is great when you're interested in diving deeper, but it's frustrating when all you're interested in is the how. For example, the linked blog post spends a lot of time talking about diamond dependencies and other package managers. This is just noise that gets in the way when you're trying to figure out how does this work? If you did want to combine both in a single reference doc, I would move the why out into separate, skippable sections. When I first was learning Go, I was really impressed by how easy it was to understand the language just by reading the spec. I've found the opposite to be true for Go modules. (Which also, as near as I can tell, doesn't have a spec, but just various scattered blog posts, for various different iterations of the idea.)
- deklerk 6y ago
  Thanks, this is good feedback. In general, I reckon having a variety of documentation is great, so also would like more of "how" documentation. I believe the tutorials that are in the work take more of the "short and sweet" approach, which should help with this.

dilap 6y ago

I wonder if they should've just required v1 or v0 when using Go modules†, so that v2 and above didn't seem like a red-headed step child / surprising / non-obvious by comparison. The basic idea of having separate import paths for non-compatible versions of a project seems, in fact, really good; an elegant solution to a tricky problem. It certainly seems much nicer than secretly using multiple versions of a package behind your back (especially when you consider that if this happens and a package uses mutable internal state, you could actually get broken behavior). I guess for me the takeaway is in aggregate, people will be maximally lazy. You need to make the correct thing be the easiest thing. (†Also release a patch to older versions of Go to just ignore v1/v0, and error for above, so you can have source compatible w/ both module and non-module Go.)

donatj 6y ago

Author of the post here. Really neat to see something I posted last night on HN the very next morning. With a night to sleep on it, I think requiring `/v0` and `/v1` would have been the better option, that way you'd hit the issue right away rather than years into your project. Teach things early. I've updated the post with a note of this idea. I think that's likely not something they could add at this point however. I don't even mean to be critical of the v2-ing itself, I just think it's entirely non-obvious.

earthboundkid 6y ago

I think people are too quick to declare v1. All of my modules are (deliberately!) v0 because v1 means "I commit to never breaking this API." Most Go modules should be v0.year.minor, eg v0.20.5 for the fifth release this year. You should only be using v1 if you have a lot of users, a set of core maintainers, etc. etc.

fnoof 6y ago
I feel semver combines two separate ideas into the meaning of v1: By labeling as v1 you're: - communicating the package is stable and will be maintained - communication that breaking changes will now increment the major version (saying nothing about stability, backported bug fixes, or any maintenance) I wish they were not combined. Incrementing major versions frequently on early stage projects would make them work better with package managers, but without the expectation of maintenance from the community. Not incrementing major versions makes it hard for package managers to make sensible decisions.
perlgeek 6y ago
How does that make anything better for the user? They know they cannot rely on a stable API, which means they have to carefully read the diff for every single upgrade, so saving no work over explicitly picking a major version.
- earthboundkid 6y ago
  The advantage is truth in advertising.

alexandrerond 6y ago

Modules had been in discussion for like years. Modules HowTo has been around from day one and it tells about this very clearly. Modules adoption rolled over a couple of releases which gave all the pointers they could.. Any minimally maintained project uses modules already. It is a must for any Go programmer to be skilled at managing modules because otherwise things just break. And yet it seems some people have lived under rocks. People don't have time to RTFM, but apparently lots of time to complain and write about things they should have known with minimal due diligence around the tech they use. shrug

apta 6y ago

It's absurd that golang didn't take established solutions such as Java's packaging system, and had to re-invent the wheel in a completely suboptimal way.

patatsuh 6y ago

The dep fans try to strike back again?? I hoped we are done with this...

wallyqs 6y ago

It has gotten much better but in our project we got bitten by this. I gave a talk about it at GoSF last year if looking for some Go v2 modules schadenfreude :) https://www.youtube.com/watch?v=8xAaZDSDWOc https://www.youtube.com/watch?v=8xAaZDSDWOc