10 ms·

Using Go Modules

302 points by spacey 8y ago

sethammons 8y ago
The meat: > go mod init creates a new module, initializing the go.mod file that describes it. > go build, go test, and other package-building commands add new dependencies to go.mod as needed. > go list -m all prints the current module’s dependencies. > go get changes the required version of a dependency (or adds a new dependency). > go mod tidy removes unused dependencies. And you can use some sugar to declare exact versions you want.
ilovecaching 8y ago
I've been using Go modules, and I really like them. It's obvious they really paid a lot of attention to backwards compatibility. Being able to vendor using mod is really awesome, as well as see all your transitive dependancies automatically. It's very goish to focus on minimizing your dependancies and actually think about what you're pulling in. I'm not yet sure how I feel about the v2+ stuff. Having a separate module for breaking changes is something I think I'll have to try more to form an opinion on, but I dislike the idea of embedding the version in the name instead of using the mod file. I do think it's kind of odd that it decides to put packages in GOPATH, but disallows you from coexisting go mod projects in GOPATH. I organize all my code for all my languages using the go repo as dir. I've had to maintain a separate tree for go mod which is not ideal. After using Go dep and mod, I felt that dep was the super straightforward and obvious way to do it. It's how I would have done it. Go mod is much more Goish, it's opinionated and based on a philosophy that fits with the language. It Gives me hope that if they do add generics they will make them uniquely Go as well.
- jrockway 8y ago
  It does not put modules in GOPATH, at least not where you'd normally find them. Pre-modules github.com/foo/bar would be in $GOPATH/src/github.com/foo/bar. Now they are in $GOPATH/pkg/mod/... (... being based on the URL with some handling for versions and characters that don't work well in filesystems, I don't know the exact details). The main problems I've had with go modules are fast-and-loose upstreams that happily rewrite history. go mod detects this and stops the world, and you have to manually edit go.sum. I also feel like "go get github.com/foo/bar" doesn't always get me the latest version. You see that there was a commit 1 hour ago, but then "go get" adds a version like 20181130-93874837 to your dependencies. I assume they know what they're doing? But I'm probably wrong.
  - ilovecaching 8y ago
    That's what I meant, pkg is definitely in GOPATH. It's annoying because I can't put my code in src anymore, which is where I've been organizing it for years, but I still have to keep GOPATH around for bin and pkg.
    - grogenaut 8y ago
      You don't really need bin anymore, run go install or go build with a output target. I'm doing platform independent stuff anyway so I have a json file in my projects that describes what execs I want to release and where they are in go and what the filename will be when done. I then create shim scripts for all platforms (in my multi-platform distro), or could easily just make 3 different "distributions". Yes I know this isn't using the tools idiomatically or whatever but I've found it more useful since I work with people who are on multiple platforms.
    - fhs 8y ago
      What the blog post neglected to mention is that you can force module usage, even inside $GOPATH, by setting the environment variable GO111MODULE=on. I have a go command wrapper script called vgo that sets GO111MODULE=on, which I use when I want to use modules inside $GOPATH. More info here: https://github.com/golang/go/wiki/Modules https://github.com/golang/go/wiki/Modules
    - jrockway 8y ago
      Fair enough. I clarified because the days of editing code in some dependency of yours are over; it's not easily exposed for that purpose (but you can do it ;).
      - shabbyrobe 8y ago
        Which is, of course, totally unacceptable. This is a critical use case for doing deep-dive debugging or for code exploration. Fortunately for us there is the wonderfully named "gohack", which does easily expose your deps for editing (though not as easily as before): https://github.com/rogpeppe/gohack https://github.com/rogpeppe/gohack
dimgl 8y ago
I've been using Go modules for a bit and I really like it. I can easily set up local dependencies using the `replace` keyword as well. For instance, all of the apps in `cmd` which need access to common packages are their own modules, and then I just add: replace pkg => ../../pkg And I can refer to everything inside of `pkg` (`pkg` is its own module as well).
dylkil 8y ago
is it worth using?
joshklein 8y ago
Making sure I understand this: I’m some independent developer working in many languages. I like every project I work on to be inside ~/work/ within a subdir I name based on the project. I used to be annoyed that I had to put every go project into a dir 7-8 layers below that e.g ~/work/go/src/github.com/joshklein/project/cmd/hello_world.go, but now I can have ~/work/go_hello/src/main.go. Right? I understand this isn’t the main point, but frankly it’s the thing I care the most about.
- sudhirj 8y ago
  You can put any project anywhere you want.
  - sudhirj 8y ago
    Why is this inaccurate? Go modules allows you to put any Go project anywhere on your machine and have it work fine - what am I missing?
- ilovecaching 8y ago
  I actually organize all my projects using the go way, there's nothing that stops you from coexisting Haskell in the Go tree. The advantage is that if you are working on something like the kernel, you can maintain separate branches easily. That said, mod allows you to put your project anywhere but in the GOPATH, so the answer to your question is yes. The path is virtualized in the mod file.
  - deleted 8y ago
    [deleted]
  - Waterluvian 8y ago
    It awfully breaks when you run into multiple systems that want things done their way. Like Go and ROS workspaces.
  - humblebee 8y ago
    When I first started using go I also was a bit annoyed at the fact that I had to have everything in my gopath, particularly when I forked a repo as none of the paths would lead to my fork. I've since started to also organize all of my code the gopath way and I just set $GOPATH to $HOME. When forking I just add a new remote and work out of the upstream repo, which seems saner now as it reduces object duplication. I don't know if I will ever move away from organizing my node this way if I can help it. > That said, mod allows you to put your project anywhere but in the GOPATH I don't know if I've run into this yet, though the only project[0] I've used gomod with has a makefile so it might do something to handle it. [0] https://github.com/ipfs/go-ipfs https://github.com/ipfs/go-ipfs
fourseventy 8y ago
The main thing that annoys me about Go modules is that it broke so much tooling. I still can't get GoCode to work properly, and there are 4 or 5 different forks of the repo all trying to add module support, its a mess.
- techaul 8y ago
  Can't you just keep using GOPATH until they're fixed?
  - ilovecaching 8y ago
    You can, go mod is completely opt in.
- throwaway8879 8y ago
  Gocode works in vim but for some reason won't do autocomplete in spacemac's go-mode. I haven't found the time to look for a fix yet. There's a fork of Gocode that fixes it, but makes vim's autocomplete very very slow. As someone who switches between vim and emacs I wish this mess gets sorted out soon!
  - pcnix 8y ago
    What's the fork that fixes the problem? I haven't used Go since modules released, but I'm going to be needing it soon, and this is something I need.
    - throwaway8879 8y ago
      It's this one. https://github.com/stamblerre/gocode https://github.com/stamblerre/gocode
  - farslan 8y ago
    HEAD of vim-go has now gopls support, which has excellent support for Go modules for code completion and jumpt to definition. Give it a try.
aw4y 8y ago
question: what happens if you wanna organize your software in modules? so for example, a main module (package main) and a secondary module (core). Do you init both as separate modules? and then you use the second as dependency on the first one? do they have to be actually published somewhere to be accessible in this way?
- ilovecaching 8y ago
  By definition of you calling main and core modules, yes, you would init both. They would both need to be available via git source control, but otherwise go mod is going to go get the virtualized path that's written into the mod file the same as if you had done it yourself with go get, so nothing has really changed.
- skybrian 8y ago
  You could use an unpublished version of "core" using the "replace" directive, but this might not be a good idea. If you have two modules that are so closely tied together that they always need to be released simultaneously, it's probably better to make it a single module with multiple packages in it.
  - aw4y 8y ago
    thanks!
antwerpen 8y ago
Google API's latest semantic version is behind the doc site (godoc.org/google.golang.org/api). Is this standard practice in Go? I was coincidentally converting my Go project to use Go modules yesterday. I depended on a Google API which was originally retrieved via 'Go get', corresponded to docs and worked fine as this pulled from HEAD. `go mod` did not work out of the box, as it required the latest semantic version (v.0.2.0) of my this Google API import. This version, however, is behind documentation and broke my code. I understand I can require a specific commit in the go.mod file, but the strings for specific commit seem cryptic. Where can I look up the version hash that matches the doc site?
- antwerpen 8y ago
  Ah. The trick is to substitute in the go.mod file's require block.: s/$SEMANTIC_VERSION_BRANCH/$BRANCH In this case, $BRANCH == 'master'. 'go build' will resolve 'master' to a specific commit hash version.
  - chrisbroadfoot 8y ago
    you can also `go get google.golang.org/api@master`
skybrian 8y ago
There are at least two bugs in this tutorial: - There's a duplicated section starting with: "Note that our module now depends on both rsc.io/quote and rsc.io/quote/v3:" - In the example of upgrading a major version, Hello was renamed to HelloV3, but the caller isn't renamed. It should be "quoteV3.HelloV3".
mikepurvis 8y ago
I'm not an active golang user, but something which gives me pause here is the insistence on a strict 3-number semver. For a commercial entity shipping software which may include upstream components, it's important to have options for maintaining (and versioning) in-house forks of upstream components. This becomes a problem when you fork v1.2.3 from upstream and want to internally release your fixes, but now your internal 1.2.4 and upstream's 1.2.4 both have the same number but diverge in content. I like the Debian solution to this, which permits freeform textual suffixes, so that in the hypothetical scenario above, you can release v1.2.3bigco1, v1.2.3bigco2, with the final number indicating the version of the patchset being applied onto the upstream release; then it's also clear what to do when you rebase your fork because you've maintained the integrity of the upstream version.
- ilovecaching 8y ago
  This has never been a problem in Go, the repo name encodes the owner of a fork, and two separate repo paths with the same version are still completely different modules. The nice thing about not doing what debian does is that there is consistency. Freeform usually means diverging opinions, which is not the Go way.
  - rdsubhas 8y ago
    I believe parent's question was: Yes, you will have different repo names. But there is sub version pinning. i.e. The first patch of v1.2.3 is called v1.2.3.1 (or v1.2.3-1 or something). Then the repo itself will evolve, and keep publishing v1.2.3-2, v1.2.3-4 and so on. Patched versions are "sub" versions of the main version, and those sub versions also keep evolving for the same upstream version.
- deleted 8y ago
  [deleted]
- skybrian 8y ago
  For a single module, there is a "replace" directive that you can use to point to a local fork: https://github.com/golang/go/wiki/Modules#when-should-i-use-the-replace-directive https://github.com/golang/go/wiki/Modules#when-should-i-use-... But if you have an internal version that's used in lots of modules, it's probably better to use an import path under your own organization, to avoid confusion.
mirceal 8y ago
Every mainstream programming language that came out in the last 20 years has already solved this. Java, Javascript, Ruby, Python, Rust, you name it. I am baffled to why Go has not figured this out and are presenting recent developments as some kind of breakthroughs - they are not. To me, and I'm not trying to be disrespectful here, because of the shortcomings it has, Go is in the experimental/keep an eye on bucket. It does some things really, really well but it's far from the panacea most developers that use it think it is.
- ilovecaching 8y ago
  I never had any issues with the GOPATH and vendor approach, but I have experience with using a huge monorepo, so I never had expectations that it would be any different. But go mod is great, and it is very different than cargo, pip, npm, etc. I don't think they are claiming it's a spiritual revolution, but it solves the problems people who needed traditional package versioning where having in an elegant and Goish way. Go is also far from experimental. It has areas where it could be better, but it's a panacea compared to writing Java in an IDE with Maven for a living.
  - mirceal 8y ago
    given the option I will take java any day or night. we developers like to experiment things and put them on our resumes, but once the kool-aid is gone very few things stand the test of time. imho, go wouldn’t be a thing if it didn’t have [initial] backing from google.
    - ilovecaching 8y ago
      Go grew as a result of the open source ecosystem that grew around Docker. Google has backed plenty of other languages that have gone nowhere, so I think it's a fallacy to claim that's why Go has risen in popularity. If anything it was containerization and maybe the fact that Ken and Pike were involved more than the Google brand. Go is also just really great. You don't need an IDE or a big crusty language to write good software. Go is small, compiles quickly, runs efficiently, is easy to teach, and has great tooling that lets you get up in running in just a few minutes. The ecosystem is centered around open source, the Go project itself including the name, logos, and specification are open source, and the Go conference in Denver is fantastic. Go is not kool-aid.
gfs 8y ago
In the "Upgrading dependencies" section when they use "go get", will the command be local to that module? I am used to "go get" working with the idea of a $GOPATH. I almost wish they introduced another sub-command to update a dependency.
Animats 8y ago
"the go command automatically looks up the module containing that package" Looks up where? GOPATH? The tree below the current file? The current directory? What's the "current module", anyway? Where is that stored? "go: downloading rsc.io/sampler v1.3.0" Downloading from where? Github? From what repository? The page for "go get" now talks about "module-aware" mode, but doesn't make it clear how that works. This is the usual headache with dependency systems. There's some specific directory layout required, and you have to know what it is. This article needs to be more explicit about that. Otherwise you end up depending on lore and monkey copying.
- saturn_vk 8y ago
  > There's some specific directory layout required, and you have to know what it is No you don't. If it supposedly works, why would you care where these modules end up in when they are downloaded? There's currently no indication that you need to be aware at all of whatever directory layout the module system needs, as it takes care of it automatically. Finally, the article does actually mention where the modules are downloaded: > ... the downloaded modules are cached locally (in $GOPATH/pkg/mod)
- kemitche 8y ago
  > from where? Given that the import paths are URL-ish, I personally find it fairly obvious where things are coming from - far more so than any other language/dependency manager I've worked with. That's something unchanged with go modules vs GOPATH.
- LukeShu 8y ago
  > What's the "current module", anyway? Where is that stored? The article uses "current module" like you might use "current git repository"; it's the module containing the directory that you are currently `cd`ed to. Just as git identifies the repo by looking for `.git` in successive parent directories, go identifies the module by looking for `go.mod` in successive parent directories. > > "go: downloading rsc.io/sampler v1.3.0" > Downloading from where? Github? From what repository? From the same place that non-module-aware `go get` downloads it. The only difference on the "remote side" of that from old `go get` is that it grabs the `v1.3.0` git tag, instead of git HEAD; if you don't specify a version it grabs the most recent `vSEMVER` git tag instead of git HEAD. > > "the go command automatically looks up the module containing that package" > Looks up where? GOPATH? The tree below the current file? The current directory? Looks it up on the network, à la `go get`. Exceptions: - The `replace` directive in your `go.mod` can manually override where it looks up a specific package. - Telling Go `-mod=vendor` will have it look up all depended-upon modules in `vendor/`, rather than via the normal mechanisms; you can create the `vendor/` directory with `go mod vendor`. It doesn't use the network every time; a module cache lives in `${GOPATH:-${HOME}/go}/pkg/mod/`; but you don't need to know that any more than you need to know that a build cache lives in `${GOCACHE:-${HOME}/.cache/go-build}`. > There's some specific directory layout required, and you have to know what it is. Not really, the only requirement is that you have a `go.mod` file that identifies a package name for the directory that it's in. So that if I have github.com/lukeshu/foo, it just needs a `go.mod` saying module github.com/lukeshu/foo instead of having the requirement that it be checked out to `$GOPATH/src/github.com/lukeshu/foo`. Beyond having the `go.mod` file, there aren't really any structure requirements. > This article needs to be more explicit about that. Does it, though? It's a blog-post, not the "normal" documentation. The full docs are much more explicit and nitty-gritty than a high-level blog post.
fiatjaf 8y ago
All I wanted was a way to safely and easily cleanup my $GOPATH. I don't have a lot disk space and $GOPATH takes up most of it. This thing apparently doesn't solve my problem. `go mod tidy` simply removes a dependency from a module, but that dependency is still cached in a big arcane directory at $GOPATH/src/mod. Why? I think we all should be using something like Nix for dependency management that solves all problems, but that is so hard to setup!
- jerf 8y ago
  Not having a lot of disk space is a minority case for developers now. I wouldn't hold your breath for the Go authors to address it. If you have a disk space problem, Nix would seem to be the opposite of the solution to that. One of the ways Nix does its magic is to chew through disk space a lot more freely than most distros do.
  - fiatjaf 8y ago
    But it has automatic and safe cleanup, right? So I can use only what I need at any point in time instead of having an opaque directory full of unused stuff I don't know if I can delete or not.
  - fiatjaf 8y ago
    Also, there are a lot of people who think disk space is an issue. See https://pnpm.js.org/ https://pnpm.js.org/ for example. (Ironically, pnpm doesn't have an auto cleanup feature.)
    - jerf 8y ago
      npm was especially broken with disk space, above and beyond what Go has ever had, so their problem was much worse. And my point is not that disk space is never an issue. That's why I said it was a minority issue, not "not an issue". My point is more than Go is not a language about addressing every fiddly minority's issues. It's definitely about the 20% that does 80% of the work. So waiting for a language lead by a philosophy like that to address a disk space issue is probably not a good plan. To be a bit more constructive, I'd observe I've had great experiences with cross-compiling. On the chance you're disk-space limited because you're developing right on a target resource-constrained device, you may be able to move your development to a more powerful system, even of a different architecture, and cross-compile fairly easily. I often have a workflow where I just "go build blahblahblah && rsync blahblahblah target:blahblahblah && ssh target blahblahblah" and I just press up & enter on a shell when I want to push & test the code. As long as you've got half-decent bandwidth to the target device, it's fine. There may be a couple of other buttons you may want to push to speed that up, because IIRC when cross-compiling it'll end up building the entire app, and you'll want to pre-compile things for the new arch.
tschellenbach 8y ago
not ready yet for production, most of the cli tools such as errcheck dont work. https://github.com/golang/go/issues/24661 https://github.com/golang/go/issues/24661 other than that it works like a charm, here's a tutorial i wrote: https://getstream.io/blog/go-1-11-rocket-tutorial/ https://getstream.io/blog/go-1-11-rocket-tutorial/
- LukeShu 8y ago
  Do note that you can use errcheck with modules if you call it through golangci-lint.
presscast 8y ago
I haven't been using Go modules, to the point where I only have a vague notion of what they do. I haven't felt the need for them. At what point do they become necessary? Which pain-points should I be on the lookout for?
- yiyus 8y ago
  Probably at the point new versions of your dependencies break your code. If you have no external dependencies, you should be fine. It also gives you the freedom to have your projects wherever you want.
cortexio 8y ago
Go: wE mADe pAcKagEmANageMeNT I really wonder who came up with such a bad package management system.
minieggs 8y ago
I've returned to university recently. We're learning Make in one of my classes at the moment ("learning" Make seems so weird to me). When going over dependency graphs I couldn't stop thinking of how awesome `go mod tidy` is.
nicpottier 8y ago
Be warned that if you like all the tooling VSCode provides for Go then that is still not available if you are using go modules. They are working on it and there are some rough betas out there for some limited functionality but simple things like renaming variables etc, are still not available. Realize this is a tooling issue and not necessarily a language one, but do feel like they should be mentioning this. It makes working with modules pretty painful. (we are doing it but man do I miss simple refactors / usages etc..)
- le_didil 8y ago
  I agree, I tried go modules a couple of months ago but the VSCode tools were much slower and cpu usage increased when using go mod extensions. I reverted back to not using go modules at the time, will try again soon to see if it improved. I still like go modules.
mmgutz 8y ago
How do you reference local go modules that are under development? The equivalent in node is `npm link`.
- _mway 8y ago
  with a `replace` entry. e.g. replace ( github.com/repo/module/path vX.X.X => /path/to/github.com/repo/module/path )
- chrisbroadfoot 8y ago
  https://github.com/rogpeppe/gohack https://github.com/rogpeppe/gohack is great for this.

302 points by spacey 8y ago

The meat: > go mod init creates a new module, initializing the go.mod file that describes it. > go build, go test, and other package-building commands add new dependencies to go.mod as needed. > go list -m all prints the current module’s dependencies. > go get changes the required version of a dependency (or adds a new dependency). > go mod tidy removes unused dependencies. And you can use some sugar to declare exact versions you want.

ilovecaching 8y ago

I've been using Go modules, and I really like them. It's obvious they really paid a lot of attention to backwards compatibility. Being able to vendor using mod is really awesome, as well as see all your transitive dependancies automatically. It's very goish to focus on minimizing your dependancies and actually think about what you're pulling in. I'm not yet sure how I feel about the v2+ stuff. Having a separate module for breaking changes is something I think I'll have to try more to form an opinion on, but I dislike the idea of embedding the version in the name instead of using the mod file. I do think it's kind of odd that it decides to put packages in GOPATH, but disallows you from coexisting go mod projects in GOPATH. I organize all my code for all my languages using the go repo as dir. I've had to maintain a separate tree for go mod which is not ideal. After using Go dep and mod, I felt that dep was the super straightforward and obvious way to do it. It's how I would have done it. Go mod is much more Goish, it's opinionated and based on a philosophy that fits with the language. It Gives me hope that if they do add generics they will make them uniquely Go as well.

jrockway 8y ago
It does not put modules in GOPATH, at least not where you'd normally find them. Pre-modules github.com/foo/bar would be in $GOPATH/src/github.com/foo/bar. Now they are in $GOPATH/pkg/mod/... (... being based on the URL with some handling for versions and characters that don't work well in filesystems, I don't know the exact details). The main problems I've had with go modules are fast-and-loose upstreams that happily rewrite history. go mod detects this and stops the world, and you have to manually edit go.sum. I also feel like "go get github.com/foo/bar" doesn't always get me the latest version. You see that there was a commit 1 hour ago, but then "go get" adds a version like 20181130-93874837 to your dependencies. I assume they know what they're doing? But I'm probably wrong.
- ilovecaching 8y ago
  That's what I meant, pkg is definitely in GOPATH. It's annoying because I can't put my code in src anymore, which is where I've been organizing it for years, but I still have to keep GOPATH around for bin and pkg.
  - grogenaut 8y ago
    You don't really need bin anymore, run go install or go build with a output target. I'm doing platform independent stuff anyway so I have a json file in my projects that describes what execs I want to release and where they are in go and what the filename will be when done. I then create shim scripts for all platforms (in my multi-platform distro), or could easily just make 3 different "distributions". Yes I know this isn't using the tools idiomatically or whatever but I've found it more useful since I work with people who are on multiple platforms.
  - fhs 8y ago
    What the blog post neglected to mention is that you can force module usage, even inside $GOPATH, by setting the environment variable GO111MODULE=on. I have a go command wrapper script called vgo that sets GO111MODULE=on, which I use when I want to use modules inside $GOPATH. More info here: https://github.com/golang/go/wiki/Modules https://github.com/golang/go/wiki/Modules
  - jrockway 8y ago
    Fair enough. I clarified because the days of editing code in some dependency of yours are over; it's not easily exposed for that purpose (but you can do it ;).
    - shabbyrobe 8y ago
      Which is, of course, totally unacceptable. This is a critical use case for doing deep-dive debugging or for code exploration. Fortunately for us there is the wonderfully named "gohack", which does easily expose your deps for editing (though not as easily as before): https://github.com/rogpeppe/gohack https://github.com/rogpeppe/gohack

dimgl 8y ago

I've been using Go modules for a bit and I really like it. I can easily set up local dependencies using the `replace` keyword as well. For instance, all of the apps in `cmd` which need access to common packages are their own modules, and then I just add: replace pkg => ../../pkg And I can refer to everything inside of `pkg` (`pkg` is its own module as well).

dylkil 8y ago

is it worth using?

joshklein 8y ago

Making sure I understand this: I’m some independent developer working in many languages. I like every project I work on to be inside ~/work/ within a subdir I name based on the project. I used to be annoyed that I had to put every go project into a dir 7-8 layers below that e.g ~/work/go/src/github.com/joshklein/project/cmd/hello_world.go, but now I can have ~/work/go_hello/src/main.go. Right? I understand this isn’t the main point, but frankly it’s the thing I care the most about.

sudhirj 8y ago
You can put any project anywhere you want.
- sudhirj 8y ago
  Why is this inaccurate? Go modules allows you to put any Go project anywhere on your machine and have it work fine - what am I missing?
ilovecaching 8y ago
I actually organize all my projects using the go way, there's nothing that stops you from coexisting Haskell in the Go tree. The advantage is that if you are working on something like the kernel, you can maintain separate branches easily. That said, mod allows you to put your project anywhere but in the GOPATH, so the answer to your question is yes. The path is virtualized in the mod file.
- deleted 8y ago
  [deleted]
- Waterluvian 8y ago
  It awfully breaks when you run into multiple systems that want things done their way. Like Go and ROS workspaces.
- humblebee 8y ago
  When I first started using go I also was a bit annoyed at the fact that I had to have everything in my gopath, particularly when I forked a repo as none of the paths would lead to my fork. I've since started to also organize all of my code the gopath way and I just set $GOPATH to $HOME. When forking I just add a new remote and work out of the upstream repo, which seems saner now as it reduces object duplication. I don't know if I will ever move away from organizing my node this way if I can help it. > That said, mod allows you to put your project anywhere but in the GOPATH I don't know if I've run into this yet, though the only project[0] I've used gomod with has a makefile so it might do something to handle it. [0] https://github.com/ipfs/go-ipfs https://github.com/ipfs/go-ipfs

fourseventy 8y ago

The main thing that annoys me about Go modules is that it broke so much tooling. I still can't get GoCode to work properly, and there are 4 or 5 different forks of the repo all trying to add module support, its a mess.

techaul 8y ago
Can't you just keep using GOPATH until they're fixed?
- ilovecaching 8y ago
  You can, go mod is completely opt in.
throwaway8879 8y ago
Gocode works in vim but for some reason won't do autocomplete in spacemac's go-mode. I haven't found the time to look for a fix yet. There's a fork of Gocode that fixes it, but makes vim's autocomplete very very slow. As someone who switches between vim and emacs I wish this mess gets sorted out soon!
- pcnix 8y ago
  What's the fork that fixes the problem? I haven't used Go since modules released, but I'm going to be needing it soon, and this is something I need.
  - throwaway8879 8y ago
    It's this one. https://github.com/stamblerre/gocode https://github.com/stamblerre/gocode
- farslan 8y ago
  HEAD of vim-go has now gopls support, which has excellent support for Go modules for code completion and jumpt to definition. Give it a try.

aw4y 8y ago

question: what happens if you wanna organize your software in modules? so for example, a main module (package main) and a secondary module (core). Do you init both as separate modules? and then you use the second as dependency on the first one? do they have to be actually published somewhere to be accessible in this way?

ilovecaching 8y ago
By definition of you calling main and core modules, yes, you would init both. They would both need to be available via git source control, but otherwise go mod is going to go get the virtualized path that's written into the mod file the same as if you had done it yourself with go get, so nothing has really changed.
skybrian 8y ago
You could use an unpublished version of "core" using the "replace" directive, but this might not be a good idea. If you have two modules that are so closely tied together that they always need to be released simultaneously, it's probably better to make it a single module with multiple packages in it.
- aw4y 8y ago
  thanks!

antwerpen 8y ago

Google API's latest semantic version is behind the doc site (godoc.org/google.golang.org/api). Is this standard practice in Go? I was coincidentally converting my Go project to use Go modules yesterday. I depended on a Google API which was originally retrieved via 'Go get', corresponded to docs and worked fine as this pulled from HEAD. `go mod` did not work out of the box, as it required the latest semantic version (v.0.2.0) of my this Google API import. This version, however, is behind documentation and broke my code. I understand I can require a specific commit in the go.mod file, but the strings for specific commit seem cryptic. Where can I look up the version hash that matches the doc site?

antwerpen 8y ago
Ah. The trick is to substitute in the go.mod file's require block.: s/$SEMANTIC_VERSION_BRANCH/$BRANCH In this case, $BRANCH == 'master'. 'go build' will resolve 'master' to a specific commit hash version.
- chrisbroadfoot 8y ago
  you can also `go get google.golang.org/api@master`

skybrian 8y ago

There are at least two bugs in this tutorial: - There's a duplicated section starting with: "Note that our module now depends on both rsc.io/quote and rsc.io/quote/v3:" - In the example of upgrading a major version, Hello was renamed to HelloV3, but the caller isn't renamed. It should be "quoteV3.HelloV3".

mikepurvis 8y ago

I'm not an active golang user, but something which gives me pause here is the insistence on a strict 3-number semver. For a commercial entity shipping software which may include upstream components, it's important to have options for maintaining (and versioning) in-house forks of upstream components. This becomes a problem when you fork v1.2.3 from upstream and want to internally release your fixes, but now your internal 1.2.4 and upstream's 1.2.4 both have the same number but diverge in content. I like the Debian solution to this, which permits freeform textual suffixes, so that in the hypothetical scenario above, you can release v1.2.3bigco1, v1.2.3bigco2, with the final number indicating the version of the patchset being applied onto the upstream release; then it's also clear what to do when you rebase your fork because you've maintained the integrity of the upstream version.

ilovecaching 8y ago
This has never been a problem in Go, the repo name encodes the owner of a fork, and two separate repo paths with the same version are still completely different modules. The nice thing about not doing what debian does is that there is consistency. Freeform usually means diverging opinions, which is not the Go way.
- rdsubhas 8y ago
  I believe parent's question was: Yes, you will have different repo names. But there is sub version pinning. i.e. The first patch of v1.2.3 is called v1.2.3.1 (or v1.2.3-1 or something). Then the repo itself will evolve, and keep publishing v1.2.3-2, v1.2.3-4 and so on. Patched versions are "sub" versions of the main version, and those sub versions also keep evolving for the same upstream version.
deleted 8y ago
[deleted]
skybrian 8y ago
For a single module, there is a "replace" directive that you can use to point to a local fork: https://github.com/golang/go/wiki/Modules#when-should-i-use-the-replace-directive https://github.com/golang/go/wiki/Modules#when-should-i-use-... But if you have an internal version that's used in lots of modules, it's probably better to use an import path under your own organization, to avoid confusion.

mirceal 8y ago

Every mainstream programming language that came out in the last 20 years has already solved this. Java, Javascript, Ruby, Python, Rust, you name it. I am baffled to why Go has not figured this out and are presenting recent developments as some kind of breakthroughs - they are not. To me, and I'm not trying to be disrespectful here, because of the shortcomings it has, Go is in the experimental/keep an eye on bucket. It does some things really, really well but it's far from the panacea most developers that use it think it is.

ilovecaching 8y ago
I never had any issues with the GOPATH and vendor approach, but I have experience with using a huge monorepo, so I never had expectations that it would be any different. But go mod is great, and it is very different than cargo, pip, npm, etc. I don't think they are claiming it's a spiritual revolution, but it solves the problems people who needed traditional package versioning where having in an elegant and Goish way. Go is also far from experimental. It has areas where it could be better, but it's a panacea compared to writing Java in an IDE with Maven for a living.
- mirceal 8y ago
  given the option I will take java any day or night. we developers like to experiment things and put them on our resumes, but once the kool-aid is gone very few things stand the test of time. imho, go wouldn’t be a thing if it didn’t have [initial] backing from google.
  - ilovecaching 8y ago
    Go grew as a result of the open source ecosystem that grew around Docker. Google has backed plenty of other languages that have gone nowhere, so I think it's a fallacy to claim that's why Go has risen in popularity. If anything it was containerization and maybe the fact that Ken and Pike were involved more than the Google brand. Go is also just really great. You don't need an IDE or a big crusty language to write good software. Go is small, compiles quickly, runs efficiently, is easy to teach, and has great tooling that lets you get up in running in just a few minutes. The ecosystem is centered around open source, the Go project itself including the name, logos, and specification are open source, and the Go conference in Denver is fantastic. Go is not kool-aid.

gfs 8y ago

In the "Upgrading dependencies" section when they use "go get", will the command be local to that module? I am used to "go get" working with the idea of a $GOPATH. I almost wish they introduced another sub-command to update a dependency.

Animats 8y ago

"the go command automatically looks up the module containing that package" Looks up where? GOPATH? The tree below the current file? The current directory? What's the "current module", anyway? Where is that stored? "go: downloading rsc.io/sampler v1.3.0" Downloading from where? Github? From what repository? The page for "go get" now talks about "module-aware" mode, but doesn't make it clear how that works. This is the usual headache with dependency systems. There's some specific directory layout required, and you have to know what it is. This article needs to be more explicit about that. Otherwise you end up depending on lore and monkey copying.

saturn_vk 8y ago
> There's some specific directory layout required, and you have to know what it is No you don't. If it supposedly works, why would you care where these modules end up in when they are downloaded? There's currently no indication that you need to be aware at all of whatever directory layout the module system needs, as it takes care of it automatically. Finally, the article does actually mention where the modules are downloaded: > ... the downloaded modules are cached locally (in $GOPATH/pkg/mod)
kemitche 8y ago
> from where? Given that the import paths are URL-ish, I personally find it fairly obvious where things are coming from - far more so than any other language/dependency manager I've worked with. That's something unchanged with go modules vs GOPATH.
LukeShu 8y ago
> What's the "current module", anyway? Where is that stored? The article uses "current module" like you might use "current git repository"; it's the module containing the directory that you are currently `cd`ed to. Just as git identifies the repo by looking for `.git` in successive parent directories, go identifies the module by looking for `go.mod` in successive parent directories. > > "go: downloading rsc.io/sampler v1.3.0" > Downloading from where? Github? From what repository? From the same place that non-module-aware `go get` downloads it. The only difference on the "remote side" of that from old `go get` is that it grabs the `v1.3.0` git tag, instead of git HEAD; if you don't specify a version it grabs the most recent `vSEMVER` git tag instead of git HEAD. > > "the go command automatically looks up the module containing that package" > Looks up where? GOPATH? The tree below the current file? The current directory? Looks it up on the network, à la `go get`. Exceptions: - The `replace` directive in your `go.mod` can manually override where it looks up a specific package. - Telling Go `-mod=vendor` will have it look up all depended-upon modules in `vendor/`, rather than via the normal mechanisms; you can create the `vendor/` directory with `go mod vendor`. It doesn't use the network every time; a module cache lives in `${GOPATH:-${HOME}/go}/pkg/mod/`; but you don't need to know that any more than you need to know that a build cache lives in `${GOCACHE:-${HOME}/.cache/go-build}`. > There's some specific directory layout required, and you have to know what it is. Not really, the only requirement is that you have a `go.mod` file that identifies a package name for the directory that it's in. So that if I have github.com/lukeshu/foo, it just needs a `go.mod` saying module github.com/lukeshu/foo instead of having the requirement that it be checked out to `$GOPATH/src/github.com/lukeshu/foo`. Beyond having the `go.mod` file, there aren't really any structure requirements. > This article needs to be more explicit about that. Does it, though? It's a blog-post, not the "normal" documentation. The full docs are much more explicit and nitty-gritty than a high-level blog post.

fiatjaf 8y ago

All I wanted was a way to safely and easily cleanup my $GOPATH. I don't have a lot disk space and $GOPATH takes up most of it. This thing apparently doesn't solve my problem. `go mod tidy` simply removes a dependency from a module, but that dependency is still cached in a big arcane directory at $GOPATH/src/mod. Why? I think we all should be using something like Nix for dependency management that solves all problems, but that is so hard to setup!

jerf 8y ago
Not having a lot of disk space is a minority case for developers now. I wouldn't hold your breath for the Go authors to address it. If you have a disk space problem, Nix would seem to be the opposite of the solution to that. One of the ways Nix does its magic is to chew through disk space a lot more freely than most distros do.
- fiatjaf 8y ago
  But it has automatic and safe cleanup, right? So I can use only what I need at any point in time instead of having an opaque directory full of unused stuff I don't know if I can delete or not.
- fiatjaf 8y ago
  Also, there are a lot of people who think disk space is an issue. See https://pnpm.js.org/ https://pnpm.js.org/ for example. (Ironically, pnpm doesn't have an auto cleanup feature.)
  - jerf 8y ago
    npm was especially broken with disk space, above and beyond what Go has ever had, so their problem was much worse. And my point is not that disk space is never an issue. That's why I said it was a minority issue, not "not an issue". My point is more than Go is not a language about addressing every fiddly minority's issues. It's definitely about the 20% that does 80% of the work. So waiting for a language lead by a philosophy like that to address a disk space issue is probably not a good plan. To be a bit more constructive, I'd observe I've had great experiences with cross-compiling. On the chance you're disk-space limited because you're developing right on a target resource-constrained device, you may be able to move your development to a more powerful system, even of a different architecture, and cross-compile fairly easily. I often have a workflow where I just "go build blahblahblah && rsync blahblahblah target:blahblahblah && ssh target blahblahblah" and I just press up & enter on a shell when I want to push & test the code. As long as you've got half-decent bandwidth to the target device, it's fine. There may be a couple of other buttons you may want to push to speed that up, because IIRC when cross-compiling it'll end up building the entire app, and you'll want to pre-compile things for the new arch.

tschellenbach 8y ago

not ready yet for production, most of the cli tools such as errcheck dont work. https://github.com/golang/go/issues/24661 https://github.com/golang/go/issues/24661 other than that it works like a charm, here's a tutorial i wrote: https://getstream.io/blog/go-1-11-rocket-tutorial/ https://getstream.io/blog/go-1-11-rocket-tutorial/

LukeShu 8y ago
Do note that you can use errcheck with modules if you call it through golangci-lint.

presscast 8y ago

I haven't been using Go modules, to the point where I only have a vague notion of what they do. I haven't felt the need for them. At what point do they become necessary? Which pain-points should I be on the lookout for?

yiyus 8y ago
Probably at the point new versions of your dependencies break your code. If you have no external dependencies, you should be fine. It also gives you the freedom to have your projects wherever you want.

cortexio 8y ago

Go: wE mADe pAcKagEmANageMeNT I really wonder who came up with such a bad package management system.

minieggs 8y ago

I've returned to university recently. We're learning Make in one of my classes at the moment ("learning" Make seems so weird to me). When going over dependency graphs I couldn't stop thinking of how awesome `go mod tidy` is.

nicpottier 8y ago

Be warned that if you like all the tooling VSCode provides for Go then that is still not available if you are using go modules. They are working on it and there are some rough betas out there for some limited functionality but simple things like renaming variables etc, are still not available. Realize this is a tooling issue and not necessarily a language one, but do feel like they should be mentioning this. It makes working with modules pretty painful. (we are doing it but man do I miss simple refactors / usages etc..)

le_didil 8y ago
I agree, I tried go modules a couple of months ago but the VSCode tools were much slower and cpu usage increased when using go mod extensions. I reverted back to not using go modules at the time, will try again soon to see if it improved. I still like go modules.

mmgutz 8y ago

How do you reference local go modules that are under development? The equivalent in node is `npm link`.

_mway 8y ago
with a `replace` entry. e.g. replace ( github.com/repo/module/path vX.X.X => /path/to/github.com/repo/module/path )
chrisbroadfoot 8y ago
https://github.com/rogpeppe/gohack https://github.com/rogpeppe/gohack is great for this.