5 ms·

Go mod’s lesser known features

philosopher1234 5y ago
I found this article in the discussion here (https://news.ycombinator.com/item?id=30869261 https://news.ycombinator.com/item?id=30869261). I thought it was worth posting because that discussion seemed to contain a lot of confusion & misinformation about go modules, so there seems to be some need for clarification!
- sigzero 5y ago
  If you the one or know them there is a spelling error: but also for security resaons. << reasons not resaons
foolfoolz 5y ago
go modules is one of the hardest things about learning go as a new learner. most of the documentation talks of gopath. the go mod error messages are extremely cryptic. why are go mod vendor and go mod tidy different commands? the lack of “one true way” is very strange. some projects module. some projects check in dependent source code. some projects have a version number in the package name. do i download with go get? or go install? or go get -u? i can’t think of another build system that has so much variation just in declaring and importing dependencies
- tedunangst 5y ago
  go mod tidy and go mod vendor are different commands because they do very different things.
- mkdirp 5y ago
  I can shed some light on some of your questions for you. > most of the documentation talks of gopath Historically go used a GOPATH env var. Go code had to live under there under `$GOPATH/src/<module-path>`. When doing an import, go would look at the import path, and find it in GOPATH. This wasn't flexible enough but still exists under the hood as it's where go saves (caches?) packages globally. If you have a go.mod/go.sum file in your project, in general, you won't have to worry about GOPATH. Worth noting that go, while widely used, was created to solve Google's problems (I guess at least initially?), where (from my understanding) they have a huge monorepo. At that point, having everything live under such a GOPATH somewhat makes sense. > why are go mod vendor and go mod tidy different commands? Go mod vendor puts all the dependencies in a ./vendor directory. Often used to commit your packages directly into git. There might be other advantages, not sure, but personally I don't use it. Go is able to pick up the packages in your go.mod directly from $GOPATH. Go mod tidy ensures your go.mod matches your used package. E.g. if you remove usage of a package in your code, go mod tidy is able to pick that up. In addition, it also ensures your go.sum matches go.mod. > the lack of “one true way” is very strange. some projects module. some projects check in dependent source code This is a historic issue. Prior to 1.12 (I think), go modules didn't exist, and there were different community projects that attempted to solve go package management. > do i download with go get? or go install? or go get -u? Again, historic issues, iirc go get -u forces an update of a package, meaning this updates your go.mod file. Go get without -u does not force update the package. It update your go.mod file if you haven't included the module previously. Go install is used for go main packages. It fetches the code, builds it (so it must be a main package), and puts the final binary in $GOPATH/bin. Assuming you have that in your patch, you can use it straight away.
  - p_l 5y ago
    GOPATH is arguably more an artifact of Plan9 than google3 monorepo, with Plan9 kinda ending up in monorepo-style access with public replica servers linked at /n/
  - na85 5y ago
    Ugh, reading this reminded me why I felt good moving away from Go after the honeymoon period was over. Actually writing the code was fun at first, but everything else around the actual act of composing code was thoroughly awful UX.
    - philosopher1234 5y ago
      Can you elaborate? It seems to me like the minimum set of features I want from package management
      - CannisterFlux 5y ago
        I have a go thing I wrote that I hadn't touched for years but had the misfortune to need to modify it recently, and the way to migrate from $GOPATH/src/ to go.mod is not really clear (to me at least). To build this thing, the only go code I have, I used git submodules to put all the dependencies in ./src, so I had ./src/github.com and a load of 3rd party modules in there, then my code was in ./src/Mything. To build it, I ran "env GOPATH=$PWD go build Mything/MyPackage" and it did the right thing, with the final binary appearing in the current directory. Now this gives an error "go.mod file not found in current directory", but if I add that file, go build says "unexpected go.mod file found in current directory". Luckily, for now, I can use GO111MODULE=off to keep pretending everything is fine, but my eyes start to glaze over when I read the migration guide at https://go.dev/blog/migrating-to-go-modules https://go.dev/blog/migrating-to-go-modules - it's one of those moments where I start to wish I'd used a different language :(
        philosopher1234 5y ago
        I’m pretty sure the migration is just 1. Go mod init in your code’s root dir 2. Done..? Go get ./…?
tedunangst 5y ago
One feature not listed is that the proxy caches 404 (which is probably reasonable), but the consequence is if you try to build with the new version before pushing the dependency, you won't be able to build without GONOPROXY until the cache expires. (Update dep code, test with go.mod replace =>, tag dep, (forget push), remove replace =>, boops.)
- verst 5y ago
  I literally ran into this 3 hours ago and couldn't figure out why. That explains it! Thanks for the insight!
- ayanamist 5y ago
  workspace of Go1.18 may solve your issues https://sebastian-holstein.de/post/2021-11-08-go-1.18-features/ https://sebastian-holstein.de/post/2021-11-08-go-1.18-featur...
pxeger1 5y ago
> A module is a collection of packages that are released, versioned, and distributed together. Each package within a module is a collection of source files in the same directory that are compiled together. I wish Go had got these names the right way round. Every other programming language and packaging (NB. not "modeling"!) ecosystem uses package to mean distribution, and module to mean compilation unit.
- lstamour 5y ago
  It could be worse, it could be Java. I might be wrong here, but as I understand it: Java’s use of the terms module and package which these days vary depending on context. Once upon a time, you put modules of code in a namespaced package, but then these packages were distributed and run as modules, and suddenly both package and module ceased to have clear meanings - context is required. Then again, maybe we should make up new terms instead of re-using existing ones. Like Ruby “Gems” or Homebrew’s “Casks”. It’s not intrinsically descriptive, but it is unique and unambiguous. I suppose again it could be worse, we could be talking about “libraries” and “frameworks” while describing “modules” and “packages”.
  - pjmlp 5y ago
    Those packages were distributed and run as JAR, WAR or EAR files. I never remember having seen them referenced as modules.
    - lstamour 5y ago
      OSGi modules? It’s true there are other names, but if you Google “Java 9 packages modules” you’ll get some very weird results. For example: https://softwareengineering.stackexchange.com/questions/311280/why-are-packages-and-modules-separate-concepts-in-java-9 https://softwareengineering.stackexchange.com/questions/3112...
      - pjmlp 5y ago
        OSGi modules are related to Eclipse and how they modularized their components. Yes modules in Java don't mean the same thing as in Modula-2, Ada has packages, child packages and nested packages, Object Pascal/UCSD Pascal has units, while VMS Pascal has modules, and so on. There is plenty of literature to educate onself about the subject instead of Internet random searches.
cespare 5y ago
> Requires a domain name to be the first part of the module identifier This is only true if you want the module to be publicly 'go get'-able. Private modules can be named whatever you want. (Some tools use whether or not the first import path segment contains a '.' as a heuristic for "is this package stdlib", and those won't work correctly on a module that doesn't use a dot. There's a proposal, not yet accepted, to document this as a naming requirement for modules: https://github.com/golang/go/issues/32819 https://github.com/golang/go/issues/32819 This is of course a looser requirement than "must be a domain name".)

I found this article in the discussion here (https://news.ycombinator.com/item?id=30869261 https://news.ycombinator.com/item?id=30869261). I thought it was worth posting because that discussion seemed to contain a lot of confusion & misinformation about go modules, so there seems to be some need for clarification!

sigzero 5y ago
If you the one or know them there is a spelling error: but also for security resaons. << reasons not resaons

foolfoolz 5y ago

go modules is one of the hardest things about learning go as a new learner. most of the documentation talks of gopath. the go mod error messages are extremely cryptic. why are go mod vendor and go mod tidy different commands? the lack of “one true way” is very strange. some projects module. some projects check in dependent source code. some projects have a version number in the package name. do i download with go get? or go install? or go get -u? i can’t think of another build system that has so much variation just in declaring and importing dependencies

tedunangst 5y ago
go mod tidy and go mod vendor are different commands because they do very different things.
mkdirp 5y ago
I can shed some light on some of your questions for you. > most of the documentation talks of gopath Historically go used a GOPATH env var. Go code had to live under there under `$GOPATH/src/<module-path>`. When doing an import, go would look at the import path, and find it in GOPATH. This wasn't flexible enough but still exists under the hood as it's where go saves (caches?) packages globally. If you have a go.mod/go.sum file in your project, in general, you won't have to worry about GOPATH. Worth noting that go, while widely used, was created to solve Google's problems (I guess at least initially?), where (from my understanding) they have a huge monorepo. At that point, having everything live under such a GOPATH somewhat makes sense. > why are go mod vendor and go mod tidy different commands? Go mod vendor puts all the dependencies in a ./vendor directory. Often used to commit your packages directly into git. There might be other advantages, not sure, but personally I don't use it. Go is able to pick up the packages in your go.mod directly from $GOPATH. Go mod tidy ensures your go.mod matches your used package. E.g. if you remove usage of a package in your code, go mod tidy is able to pick that up. In addition, it also ensures your go.sum matches go.mod. > the lack of “one true way” is very strange. some projects module. some projects check in dependent source code This is a historic issue. Prior to 1.12 (I think), go modules didn't exist, and there were different community projects that attempted to solve go package management. > do i download with go get? or go install? or go get -u? Again, historic issues, iirc go get -u forces an update of a package, meaning this updates your go.mod file. Go get without -u does not force update the package. It update your go.mod file if you haven't included the module previously. Go install is used for go main packages. It fetches the code, builds it (so it must be a main package), and puts the final binary in $GOPATH/bin. Assuming you have that in your patch, you can use it straight away.
- p_l 5y ago
  GOPATH is arguably more an artifact of Plan9 than google3 monorepo, with Plan9 kinda ending up in monorepo-style access with public replica servers linked at /n/
- na85 5y ago
  Ugh, reading this reminded me why I felt good moving away from Go after the honeymoon period was over. Actually writing the code was fun at first, but everything else around the actual act of composing code was thoroughly awful UX.
  - philosopher1234 5y ago
    Can you elaborate? It seems to me like the minimum set of features I want from package management
    - CannisterFlux 5y ago
      I have a go thing I wrote that I hadn't touched for years but had the misfortune to need to modify it recently, and the way to migrate from $GOPATH/src/ to go.mod is not really clear (to me at least). To build this thing, the only go code I have, I used git submodules to put all the dependencies in ./src, so I had ./src/github.com and a load of 3rd party modules in there, then my code was in ./src/Mything. To build it, I ran "env GOPATH=$PWD go build Mything/MyPackage" and it did the right thing, with the final binary appearing in the current directory. Now this gives an error "go.mod file not found in current directory", but if I add that file, go build says "unexpected go.mod file found in current directory". Luckily, for now, I can use GO111MODULE=off to keep pretending everything is fine, but my eyes start to glaze over when I read the migration guide at https://go.dev/blog/migrating-to-go-modules https://go.dev/blog/migrating-to-go-modules - it's one of those moments where I start to wish I'd used a different language :(
      - philosopher1234 5y ago
        I’m pretty sure the migration is just 1. Go mod init in your code’s root dir 2. Done..? Go get ./…?

tedunangst 5y ago

One feature not listed is that the proxy caches 404 (which is probably reasonable), but the consequence is if you try to build with the new version before pushing the dependency, you won't be able to build without GONOPROXY until the cache expires. (Update dep code, test with go.mod replace =>, tag dep, (forget push), remove replace =>, boops.)

verst 5y ago
I literally ran into this 3 hours ago and couldn't figure out why. That explains it! Thanks for the insight!
ayanamist 5y ago
workspace of Go1.18 may solve your issues https://sebastian-holstein.de/post/2021-11-08-go-1.18-features/ https://sebastian-holstein.de/post/2021-11-08-go-1.18-featur...

pxeger1 5y ago

> A module is a collection of packages that are released, versioned, and distributed together. Each package within a module is a collection of source files in the same directory that are compiled together. I wish Go had got these names the right way round. Every other programming language and packaging (NB. not "modeling"!) ecosystem uses package to mean distribution, and module to mean compilation unit.

lstamour 5y ago
It could be worse, it could be Java. I might be wrong here, but as I understand it: Java’s use of the terms module and package which these days vary depending on context. Once upon a time, you put modules of code in a namespaced package, but then these packages were distributed and run as modules, and suddenly both package and module ceased to have clear meanings - context is required. Then again, maybe we should make up new terms instead of re-using existing ones. Like Ruby “Gems” or Homebrew’s “Casks”. It’s not intrinsically descriptive, but it is unique and unambiguous. I suppose again it could be worse, we could be talking about “libraries” and “frameworks” while describing “modules” and “packages”.
- pjmlp 5y ago
  Those packages were distributed and run as JAR, WAR or EAR files. I never remember having seen them referenced as modules.
  - lstamour 5y ago
    OSGi modules? It’s true there are other names, but if you Google “Java 9 packages modules” you’ll get some very weird results. For example: https://softwareengineering.stackexchange.com/questions/311280/why-are-packages-and-modules-separate-concepts-in-java-9 https://softwareengineering.stackexchange.com/questions/3112...
    - pjmlp 5y ago
      OSGi modules are related to Eclipse and how they modularized their components. Yes modules in Java don't mean the same thing as in Modula-2, Ada has packages, child packages and nested packages, Object Pascal/UCSD Pascal has units, while VMS Pascal has modules, and so on. There is plenty of literature to educate onself about the subject instead of Internet random searches.

cespare 5y ago

> Requires a domain name to be the first part of the module identifier This is only true if you want the module to be publicly 'go get'-able. Private modules can be named whatever you want. (Some tools use whether or not the first import path segment contains a '.' as a heuristic for "is this package stdlib", and those won't work correctly on a module that doesn't use a dot. There's a proposal, not yet accepted, to document this as a naming requirement for modules: https://github.com/golang/go/issues/32819 https://github.com/golang/go/issues/32819 This is of course a looser requirement than "must be a domain name".)