5 ms·

I absolutely love writing programs in Go and at my current startup we're basing all our core backend infrastructure on it. One thing that I find extremely trou

I absolutely love writing programs in Go and at my current startup we're basing all our core backend infrastructure on it. One thing that I find extremely troubling about Go's dependency handling though (and which at least partly contributes to people's frustration I think) is the mixing of the concepts of a namespace and a resource identifier in Go's package names. It's absolutely reasonable that package names should be unique, making the package name double as a storage location is not always a good idea though (IMHO): For example, we recently switched from using gitlab.com to our own private Gitlab instance, which runs under a custom (non-publicly resolvable) domain. Since all our packages were named like "gitlab.com/.../..." (to make them "go get"-table) we would have to rename all of them to make them "go get"-table from the new location. In the end we solved it by defining a git URL rewriting rule that rewrites the old to the new URL when fetching the repository. This ensured that we could work primarily on our new Gitlab server while maintaining the possiblity to build packages on the gitlab.com server. While that works I think it would have been a better idea to keep package names and storage locations separate, or at least give users a way to override/modify the resolution of packages into locations using a more standardized approach: Such a mechanism could then allow for different name resolution strategies (like a package file, a list of standard repositories) while falling back to the standard way of resolving packages into URLs for "go get". Granted, this would add another layer of complexity, but as we already need a separate file for proper vendoring (to e.g. specify versions) it wouldn't hurt too much to also put (optional) information about storage locations into that file. Just my 2 cents, apart from this minor inconvenience I'm absolutely happy with Go and its ecosystem!

aargh_aargh 8y ago
Ah, it's a variation on the URL vs. URI vs. URN debate again.
ithkuil 8y ago
if you want to decouple the namespace from the repository location you can! https://golang.org/cmd/go/#hdr-Remote_import_paths https://golang.org/cmd/go/#hdr-Remote_import_paths (read until the end) or TL;DR: https://blog.bramp.net/post/2017/10/02/vanity-go-import-paths/ https://blog.bramp.net/post/2017/10/02/vanity-go-import-path...
- TheDong 8y ago
  You still can't really. If I have a canonical import path of "mydomain.io/foo", and I host html there pointing to github or whatever, I now have a different single point of failure: the domain. If I lose control of the domain or the Most package management systems have one additional layer of resolution: a centralized pluggable repository location which can be configured to some default, and then lookups are made against that instead of against a url specified in the source code. For example, with cargo, I can open my ".cargo/config" file and specify "registry.index = whatever-url.com", and then for package "foo" it'll look it up on that registry. If I forget to renew the domain, I don't have to change a bunch of import comments in my source code to update the registry location; I have to update one cargo config file.
  - m_sahaf 8y ago
    I believe the solution you're looking for is Athens[0], right? [0] https://github.com/gomods/athens https://github.com/gomods/athens
  - Merovius 8y ago
    > If I have a canonical import path of "mydomain.io/foo", and I host html there pointing to github or whatever, I now have a different single point of failure: the domain. If I lose control of the domain or the > Most package management systems have one additional layer of resolution: a centralized pluggable repository location which can be configured to some default, and then lookups are made against that instead of against a url specified in the source code. TBH, these two seems to contradict each other. You don't want to use your own domain, because that's a SPOF that you could lose control over. Whereas the centralized repository is a SPOF that you never had any control over to begin with. Seems strictly worse to me. And the nice thing about the chosen approach is that it can coexist with what you seem to be wanting: gopkg.in exists and is exactly that. A centralized, hosted (and free!) registry for go packages with its own registration and authorization process. What I find so beautiful about the go-get solution is a) that you can build any other mechanism I know of on top of it - i.e. the user can decide on their own tradeoffs of convenience/scalability/security/future-proofedness… and b) that domain names already provide an elaborate, globally agreed upon namespace that can separate ownership from administration and hosting, with its own takedown-semantics… It's decentralized, existing, robust infrastructure. It gives full control to the user. And it solves a whole bunch of technical challenges already, for free. It means the Go team doesn't have to manage user accounts of package maintainers and it doesn't have to have policies of what code to accept or not accept - as it can't enforce them. It's really quite brilliant, IMO.
weitzj 8y ago
I agree it sounds a bit “painful”. But I think on the plus side it was very easy for you to reason about the tooling to get your simple solution going. Thinking more of: It is great that you can kind of apply “Dependency inversion of the tooling” without jumping to too many hoops to solve your problem.
Merovius 8y ago
> While that works I think it would have been a better idea to keep package names and storage locations separate Then why didn't you? Go had that ability since the inception of go-get. You could have used your own domain to name your packages from the beginning and have avoided that.

aargh_aargh 8y ago

Ah, it's a variation on the URL vs. URI vs. URN debate again.

ithkuil 8y ago

if you want to decouple the namespace from the repository location you can! https://golang.org/cmd/go/#hdr-Remote_import_paths https://golang.org/cmd/go/#hdr-Remote_import_paths (read until the end) or TL;DR: https://blog.bramp.net/post/2017/10/02/vanity-go-import-paths/ https://blog.bramp.net/post/2017/10/02/vanity-go-import-path...

TheDong 8y ago
You still can't really. If I have a canonical import path of "mydomain.io/foo", and I host html there pointing to github or whatever, I now have a different single point of failure: the domain. If I lose control of the domain or the Most package management systems have one additional layer of resolution: a centralized pluggable repository location which can be configured to some default, and then lookups are made against that instead of against a url specified in the source code. For example, with cargo, I can open my ".cargo/config" file and specify "registry.index = whatever-url.com", and then for package "foo" it'll look it up on that registry. If I forget to renew the domain, I don't have to change a bunch of import comments in my source code to update the registry location; I have to update one cargo config file.
- m_sahaf 8y ago
  I believe the solution you're looking for is Athens[0], right? [0] https://github.com/gomods/athens https://github.com/gomods/athens
- Merovius 8y ago
  > If I have a canonical import path of "mydomain.io/foo", and I host html there pointing to github or whatever, I now have a different single point of failure: the domain. If I lose control of the domain or the > Most package management systems have one additional layer of resolution: a centralized pluggable repository location which can be configured to some default, and then lookups are made against that instead of against a url specified in the source code. TBH, these two seems to contradict each other. You don't want to use your own domain, because that's a SPOF that you could lose control over. Whereas the centralized repository is a SPOF that you never had any control over to begin with. Seems strictly worse to me. And the nice thing about the chosen approach is that it can coexist with what you seem to be wanting: gopkg.in exists and is exactly that. A centralized, hosted (and free!) registry for go packages with its own registration and authorization process. What I find so beautiful about the go-get solution is a) that you can build any other mechanism I know of on top of it - i.e. the user can decide on their own tradeoffs of convenience/scalability/security/future-proofedness… and b) that domain names already provide an elaborate, globally agreed upon namespace that can separate ownership from administration and hosting, with its own takedown-semantics… It's decentralized, existing, robust infrastructure. It gives full control to the user. And it solves a whole bunch of technical challenges already, for free. It means the Go team doesn't have to manage user accounts of package maintainers and it doesn't have to have policies of what code to accept or not accept - as it can't enforce them. It's really quite brilliant, IMO.

weitzj 8y ago

I agree it sounds a bit “painful”. But I think on the plus side it was very easy for you to reason about the tooling to get your simple solution going. Thinking more of: It is great that you can kind of apply “Dependency inversion of the tooling” without jumping to too many hoops to solve your problem.

Merovius 8y ago

> While that works I think it would have been a better idea to keep package names and storage locations separate Then why didn't you? Go had that ability since the inception of go-get. You could have used your own domain to name your packages from the beginning and have avoided that.