4 ms·
Go package management is a mess of hacks. It doesn't even have a repository, instead relying on source control systems to do the actual work, with special hacks
by tsimionescu 4y ago
Go package management is a mess of hacks. It doesn't even have a repository, instead relying on source control systems to do the actual work, with special hacks for each source control system to define the artifacts that can be downloaded (e.g. using Git tags with some magic format, or Perforce commit Metadata). It requires you to physically move all code to a new folder in your version control if you want to increase the major version number. It requires users of your code to update all of their import statements throughout their code whenever you move your hosting. It relies on DNS for package identity. It takes arcane magic to support multiple Go modules in the same repo.
I can go on, but it's a terrible hodge-podge of systems. It works nicely for simple cases (consuming libraries off Github), but it's awful when you go into details. And it's not even used by its creators - since Google has a monorepo and they actually use their internal universal build tool to just compile everything from source.
- gwd 4y ago> It relies on DNS for package identity. The flip side of this is that it never has to worry about naming collisions or namespacing: Your public package name must be a URL you control. Additionally, there is no requirement for a centralized package facility to be run. The Golang project is currently running pkg.go.dev, but that's only been in the last few years; and if they decided to get rid of it, it wouldn't significantly impact the development environment. Finally, the current system makes "typo-squatting attacks" harder to do. Consider the popular golang package github.com/mattn/go-sqlite3. The only way to "typosquat" the package is to typosquat somewhere up the dependency tree; e.g., by creating github.com/matn/go-sqlite3 or something. You can't typosquat github.com/mattn/sqlite3, or github.com/mattn/go-sqlite, because you don't own those namespaces; whereas with non-DNS-based package systems, the package would be called `go-sqlite3`, and `sqlite3` or `go-sqlite` would be much easier to typosquat. All those things I find really valuable; and honestly it's something I wish the Rust ecosystem had picked up. > It requires users of your code to update all of their import statements throughout their code whenever you move your hosting. This is a necessary cost of the item above. It can be somewhat annoying, but I believe this can be done with a one-line change to the go.mod. I'd much rather occasionally deal with this. > It requires you to physically move all code to a new folder in your version control if you want to increase the major version number. And the benefit of this is that legacy code will continue to compile into the future. I do tend to find this annoying, but it was explicit trade-off that was decided back when they were developing their packaging system. Packaging is a hard problem, with lots of trade-offs; I think Go has done a pretty good job. One way in which Go and Rust have it easier than Python or Node is that the former only have to deal with developers; the latter have to deal with both developers and users, whose requirements are often at odds with one another.
- tsimionescu 4y agoPeople lose DNS names by accident all the time. It's also easy to typosquat many DNS domains, and even Github projects occasionally. Non-DNS-based packages don't have to be named "go-sqlite". You can easily require some namespacing, and even use DNS as a base for that, but having an abstraction over it that recognizes the specific needs of package management is better. For example, Maven packages are called things like org.apache.commons, and registering a new package requires control of an equivalent DNS domain. However, if you later lose control of that domain, the new owners don't simply get to replace the packet in Maven just because they snipes your domain. Go's choice to require full paths for import in each file is also not a direct implication of the previous item - they could have allowed the go.mod file to specify the path and a name, and then allow source files to import based on that name. Instead, this comes from Go tooling that existed before modules support, when the tooling would scour all files in your project to find dependencies. Moving code to a v2 dir does not specifically help (or hinder) with backwards compatibility. Old code can always simply keep using the old versions of the package anyway. It is also a very unpopular decision, with very few packages actually adopting v2 versions precisely because of this requirement, even when making major breaking changes. Even the team maintaing the Go protobuf bindings decided not to use v2 when they rehauled their code (opting instead to create a new v1 at a new location with minor versions starting at 1.20...). Sure, packaging is hard, but the Go team has chosen to go against the flow of essentially all other package managers, and instead of learning from their mistakes, they seem to have decided to make original mistakes all their own.
- jakewins 4y agoJava enforces domain name ownership as well; I can’t publish to maven central under io.netty. This removes a whole slew of attack surfaces that JS and Python have, and leverages an existing “prove you are who you say you are” infra
- tsimionescu 4y agoNote that Maven Central only imposes this once when creating a new artifact - which is, I believe, much better than Go, which imposes this for every go mod download. The advantage being, if you later lose access to the DNS domain that you used to publish an artifact to Maven Central, the new owner doesn't automatically get to compromise your artifact for all (new) users.
- jen20 4y agoSome of your criticism is reasonable, and I’m no fan of Go’s module system as a standalone artifact, but much of your criticism is unfounded. > It requires you to physically move all code to a new folder in your version control if you want to increase the major version number. This is untrue. > It requires users of your code to update all of their import statements throughout their code whenever you move your hosting. This is only true if not using a vanity URL, but is sadly often the case. > It takes arcane magic to support multiple Go modules in the same repo. I don’t know what you’re calling arcane magic here, but we maintain repos at work with 6-7 go modules in without it being an issue whatsoever, and no “arcane magic” required, so I’m going to go ahead and say this is untrue too.
- Cieric 4y agoNot the one you we're replying to, but I have an idea of the arcane magic he might be referring to. Back when I started learning Go I wanted to make a few example projects to test how well the language works, but I didn't want to push anything anywhere. When I tried to break my project into modules it ended up being that I needed to give them fake urls or something and then tell it to redirect the fake url to a folder instead. This[1] and this[2] gives a good example of the stuff I was running into at the time. I also realize that I should have probably been using sub-packages and not sub-modules, but coming from other languages I didn't realize they were suppose to be different. The thing that screwed with me most though was that there were half solutions for sub-modules to work so I kept using that terminology when searching google. [1] https://stackoverflow.com/questions/10687627/relative-import-from-parent-directory https://stackoverflow.com/questions/10687627/relative-import... [2] https://stackoverflow.com/questions/57806081/how-to-access-nested-modules-submodules-in-go https://stackoverflow.com/questions/57806081/how-to-access-n...
- metaltyphoon 4y agoNow a days you can use go workspaces to do this so you don't have to modify your go.mod files anymore.
- 4y ago
- mseepgood 4y ago> It requires you to physically move all code to a new folder in your version control if you want to increase the major version number That's simply not true. That's only one way you can do it. Another way is to create a branch.
- tsimionescu 4y agoIt's the recommended way, but you're right it's not the only one. Still, updating your major version number is much harder than in any other version control system I've seen (since however you do it, it requires you to update every single file in your repo to point to the new module). It also complicates the relationship between git tags, go.mod file location, and location within the Git repo significantly.
- skovati 4y agoInteresting read on the official golang.org proxy DoSing git.sr.ht https://sourcehut.org/blog/2023-01-09-gomodulemirror/ https://sourcehut.org/blog/2023-01-09-gomodulemirror/