5 ms·

I've found explaining multi-module repos w/ go.mod files in subdirs and how that affects using a pseudo-version vs a real version to be the most challenging par

I've found explaining multi-module repos w/ go.mod files in subdirs and how that affects using a pseudo-version vs a real version to be the most challenging part. That could be a good addition.

tsimionescu 5y ago
I'm always surprised just how difficult it is to have Go code in a monorepo given Google's famous use of one. I guess they just don't use modules internally?
- philwelch 5y ago
  The original GOPATH approach to dependency management is probably a better fit for monorepos.
  - tsimionescu 5y ago
    That's only true if everyone uses the latest version of the code. If you want to have an internal package ecosystem, with stable versions of packages, feature branches etc, then GOPATH is an even worse kludge.
- q3k 5y ago
  It works great if you don't use modules internally. Either go with plain GOPATH, or use something like Bazel/rules_go/Gazelle. Modules are not for monorepos and internal components, modules are for third party dependencies that need to be pulled in at compatible versions, and for general multirepo work.
  - tsimionescu 5y ago
    "third-party" dependencies can easily exist between teams in the same company sharing the same mono-repo. It's perhaps not as common with git, but other version control systems are much more often used as monorepos, often for building multiple unrelated projects, with some common tooling libraries used as modules. This is trivial to do with any other module system I've used (Maven, Nuget, Konan, pip, cargo), but it is extraordinarily brittle with Go.
    - q3k 5y ago
      > "third-party" dependencies can easily exist between teams in the same company sharing the same mono-repo. [...] This is trivial to do with any other module system I've used (Maven, Nuget, Konan, pip, cargo), but it is extraordinarily brittle with Go. I don't understand? This should literally just be an import statement. If your Git repository is anchored at importpath "source.example.com/git", and your code lives under "app/service/backend", and wants to import "lib/db/mysql", it just does so via `import "source.example.com/git/lib/db/mysql"`. No need to use Go modules at all. Or if you do need to use Go modules, just have one module for the entire repository. The only reason I can see this not work is if you have multiple Go modules in a single repository, or even worse have module import paths not corresponding to paths within the monorepo. But in that case the fix is easy: just don't do that?
      - tsimionescu 5y ago
        What if I want to use version 1.4.1 in one project, but 1.0.3 in another project? Modules serve a purpose inside a (large) organization just as between organizations, and tying modules to your source control choices is a generally bad idea.
        q3k 5y ago
        > What if I want to use version 1.4.1 in one project, but 1.0.3 in another project? One of the main reasons to use a monorepo is to work at a single HEAD, letting you do large scale refactors within single/linked commits, without ever having to fork any code due to breakage (as you change both sides of a potentially breaking contract in the same change). I would argue that if you're attempting to do multiple code versions in a monorepo, you're doing monorepos wrong. Or at least, you're not doing them Google-style, and that's really what GOPATH and Go has been engineered to work well with. In that case, yes, you will likely hit an impedance mismatch with Go OOTB. But I'm also quite sure it's possible to write custom tooling around the Go compiler to work with this setup (ie. use `go tool compile` and set things up manually for compilation, without using `go build` which is quite opinionated). However, if you have a monorepo with multiple code versions per subpath, what you're really doing is multirepo disguised as a technically-a-monorepo-I-guess-I-mean-it's-a-single-repo-that-makes-it-a-monorepo, with none of the benefits of either approach.
- coryrc 5y ago
  We use bazel. It's weird how different it is from the Go cli tooling.
- Steltek 5y ago
  I figured GOPATH was designed around the monorepos and they've been trying to get rid of it using the module system. "/v2", git tags but not branches, and the blast radius of trying to use a forked dependency: Golang's awful module system is top of my list for reasons to recommend away from golang. The design is full of weird kludges and your choices are zero documentation or overwhelming documentation that obscures what you really need.

I've found explaining multi-module repos w/ go.mod files in subdirs and how that affects using a pseudo-version vs a real version to be the most challenging part. That could be a good addition.

tsimionescu 5y ago

I'm always surprised just how difficult it is to have Go code in a monorepo given Google's famous use of one. I guess they just don't use modules internally?

philwelch 5y ago
The original GOPATH approach to dependency management is probably a better fit for monorepos.
- tsimionescu 5y ago
  That's only true if everyone uses the latest version of the code. If you want to have an internal package ecosystem, with stable versions of packages, feature branches etc, then GOPATH is an even worse kludge.
q3k 5y ago
It works great if you don't use modules internally. Either go with plain GOPATH, or use something like Bazel/rules_go/Gazelle. Modules are not for monorepos and internal components, modules are for third party dependencies that need to be pulled in at compatible versions, and for general multirepo work.
- tsimionescu 5y ago
  "third-party" dependencies can easily exist between teams in the same company sharing the same mono-repo. It's perhaps not as common with git, but other version control systems are much more often used as monorepos, often for building multiple unrelated projects, with some common tooling libraries used as modules. This is trivial to do with any other module system I've used (Maven, Nuget, Konan, pip, cargo), but it is extraordinarily brittle with Go.
  - q3k 5y ago
    > "third-party" dependencies can easily exist between teams in the same company sharing the same mono-repo. [...] This is trivial to do with any other module system I've used (Maven, Nuget, Konan, pip, cargo), but it is extraordinarily brittle with Go. I don't understand? This should literally just be an import statement. If your Git repository is anchored at importpath "source.example.com/git", and your code lives under "app/service/backend", and wants to import "lib/db/mysql", it just does so via `import "source.example.com/git/lib/db/mysql"`. No need to use Go modules at all. Or if you do need to use Go modules, just have one module for the entire repository. The only reason I can see this not work is if you have multiple Go modules in a single repository, or even worse have module import paths not corresponding to paths within the monorepo. But in that case the fix is easy: just don't do that?
    - tsimionescu 5y ago
      What if I want to use version 1.4.1 in one project, but 1.0.3 in another project? Modules serve a purpose inside a (large) organization just as between organizations, and tying modules to your source control choices is a generally bad idea.
      - q3k 5y ago
        > What if I want to use version 1.4.1 in one project, but 1.0.3 in another project? One of the main reasons to use a monorepo is to work at a single HEAD, letting you do large scale refactors within single/linked commits, without ever having to fork any code due to breakage (as you change both sides of a potentially breaking contract in the same change). I would argue that if you're attempting to do multiple code versions in a monorepo, you're doing monorepos wrong. Or at least, you're not doing them Google-style, and that's really what GOPATH and Go has been engineered to work well with. In that case, yes, you will likely hit an impedance mismatch with Go OOTB. But I'm also quite sure it's possible to write custom tooling around the Go compiler to work with this setup (ie. use `go tool compile` and set things up manually for compilation, without using `go build` which is quite opinionated). However, if you have a monorepo with multiple code versions per subpath, what you're really doing is multirepo disguised as a technically-a-monorepo-I-guess-I-mean-it's-a-single-repo-that-makes-it-a-monorepo, with none of the benefits of either approach.
coryrc 5y ago
We use bazel. It's weird how different it is from the Go cli tooling.
Steltek 5y ago
I figured GOPATH was designed around the monorepos and they've been trying to get rid of it using the module system. "/v2", git tags but not branches, and the blast radius of trying to use a forked dependency: Golang's awful module system is top of my list for reasons to recommend away from golang. The design is full of weird kludges and your choices are zero documentation or overwhelming documentation that obscures what you really need.