13 ms·
Keeping your Go modules compatible
- candiddevmike 6y agoReading these blog posts, you get see Go's simplicity start to unravel with modules. Uncomparable structs? Type checking workarounds? You start to wonder if these bandaids would be necessary in a generics world. For the record, I'm indifferent on the chosen module solution being right or wrong. My ideal module tool is something like npm, but it shows me the impact: diff the module changes, look at my code, and tell me what's impacted by the upgrade. Giving developers more information when upgrading libraries will go a long way towards making "update your libraries" a regular housekeeping task.
- mseepgood 6y ago> Reading these blog posts, you get see Go's simplicity start to unravel with modules. Uncomparable structs? Type checking workarounds? You start to wonder if these bandaids would be necessary in a generics world. Nothing of this has to do with or would be better with generics.
- Ericson2314 6y agoGah in every language I want that interface comparison tool, and I keep on being disappointed.
- jbamsterdam 6y agoUntil gorelease is ready, you can try apidiff (https://go.googlesource.com/exp/+/refs/heads/master/cmd/apidiff/ https://go.googlesource.com/exp/+/refs/heads/master/cmd/apid...).
- Ar-Curunir 6y agorust-semverver does what you want for Rust, I think?
- Nemo157 6y agoIt would if it worked, but every time I've tried using it I've hit some combination of crashes and weird false positives.
- jbamsterdam 6y agoDespite the title, the problems that this article discusses could have happened even before modules were introduced. Before modules, your only choices were to stay compatible or break users. Modules give us the freedom to make breaking changes in a safe, documented way.
- jrockway 6y agoModules have nothing to do with comparable structs. It's just that making a comparable struct uncomparable could break users of your API, so it's something to be on the lookout for if you value backcompat. (The module system ensures that users will get the version of the code they want; you only have to worry about backwards compatibility if you expect users to upgrade to your new version.) There are millions of ways to break users with an update that machines can't detect. For example: Old version: func Lt(a, b YourType) bool { return a < b } New version: func Lt(a, b YourType) bool { return a >= b } This is not specific to Go.
- merb 6y agobtw. this blog is about binary/source compatibility, which is a fucking hard problem in nearly EVERY language. (maybe not in javascript tough)
- arp242 6y ago"Argumentum ad generics"
- opqpo 6y agoGo modules and the entire package management and versioning system is just embarrassingly horrible. It looks like it was designed by people who have no background in software engineering at all. The language as a whole despite being now universally used as the backbone of cloud native, it honestly looks like it's intentionally designed to be horrible and insulting for anyone with sensible mind. Every obvious right decision looks like it's intentionally being replaced by a bad one if it's the right path is easier.
- hinkley 6y agoSomething I knew starting out, forgot, and learned again, is that there's a huge liability to spending your entire career working for one company. There will be things they do so well they aren't even on your radar, and you will have a huge blindspot there. And there are things they tolerate that pretty much nobody else will. So you will either suffer in ignorance or learn some nasty, nasty habits. Without moving you see no diversity, and without seeing diversity you shouldn't be put in charge of building anything for 'everyone'. You are only qualified for niche solutions, which unfortunately outsiders might not clue in on until they've invested heavily in your ideas.
- Ericson2314 6y agoThat's true, but Google is big and they use many languages, and the amount of ignorance needed to make Go cannot be explained by that alone.
- Jtsummers 6y agoIn college we referred to this same idea (among my peers who were going on through graduate studies, particularly PhDs) as "academic incest", where you aren't sufficiently exposed to a diverse enough set of ideas and end up too narrowly focused or clouded in your judgment. Ways to overcome it include: moving (in academics, accepting a fellowship at other institutions; as a professional, change jobs); observation (keeping abreast of the latest trends outside your immediate environment and field); community participation (attending conferences, symposia, and similar things; or even forums like this). But ultimately it requires a recognition that your views, knowledge, and beliefs are, necessarily, constrained by your experiences and that you sometimes have to seek out novel experiences or the experiences of others to change yourself. If you lack the drive for either, or too great a confidence in your own present state, then you're going to be stuck.
- naringas 6y agothere's something that I just don't get about golang modules, I don't know what it is but they don't click with me it might well be something tacit that golangers learn from each other directly? or otherwise it's so obvious and simple and I just haven't realized? it feels like it's the kind of thing that just clicks one day possibly after a few years
- Thaxll 6y agoIt's pretty clear and well explained there: https://github.com/golang/go/wiki/Modules https://github.com/golang/go/wiki/Modules On a day to day operation, you need to know 2/3 commands that's it. - go mod init - go mod tidy - go get xxx
- bootlooped 6y agoI use `go mod vendor` a lot.
- ArmandGrillet 6y agoIMO the documentation isn't there yet, it lacks a Q&A webpage or something short and giving the main how-tos in a succinct manner: How to create a module? How to bump a package version? How to use a specific package's branch? What to do if there is a conflict because a package used by another package messed up its compatibility? The current first Google result when looking for "go modules" is https://blog.golang.org/using-go-modules https://blog.golang.org/using-go-modules which is a very verbose post starting with "This post is part 1 in a series of 5 parts". Not very engaging. In one year I've accumulated a few snippets that I use regularly and get used to `go mod` but it was painful at the beginning. I wish I could have contributed to this topic but Go is most of the time a language that only requires to read pkg.go.dev and the main blog posts to know what's next and this Go modules workflow is, like gopls, something that doesn't currently have a good place to be documented AFAIK. Edit: Thaxll's comment gives a very nice URL to learn about Go modules, still too much for a beginner IMHO but it has a Q&A part that goes very deep.
- m0th87 6y agoYou're not alone. I've been writing go since before 1.0, and can handle modules/packages in _any other language_ fine, yet go's regularly leaves me stumped. A little while ago we were hitting yet another issue with dependencies related to version pinning at work. I tried to divine what the disparate versions mean, and came up with this flow chart from a legalistic reading of stack overflow issues [1] and the go modules doc [2]: has the package opted into go modules (i.e. is there a go.mod?) yes does the repository use semver? yes is the major version <= 1? yes => normal versioning, e.g. github.com/stretchr/testify v1.3.0 no => when importing, we need add /vN at the end, e.g., import "github.com/my/mod/v4", and in go.mod it will behave like github.com/my/mod/v4 v4.1.0 no => pseudo-versioning will be used, e.g. github.com/kolo/xmlrpc v0.0.0-20190717152603-07c4ee3fd181 no does the repository use semver? yes is the major version <= 1? yes => pseudo-versioning will be used, e.g. github.com/kolo/xmlrpc v0.0.0-20190717152603-07c4ee3fd181 no => version will be marked as incompatible, e.g. github.com/zeromq/goczmq v4.1.0+incompatible no => ??? I still don't know if this is right. But that's not really the point. I think the go authors painted themselves into a corner with such an anemic initial release. Now that they're trying to address shortcomings, the complexity is blowing up to handle the very large/diverse ecosystem. It reminds me a lot of java around 1.4 when they added in generics. In the long-run it was the right move, but man was it painful. 1: https://stackoverflow.com/a/57372286 https://stackoverflow.com/a/57372286 2: https://github.com/golang/go/wiki/Modules https://github.com/golang/go/wiki/Modules
- poorman 6y ago> When you run into a case where you want to add a method to an existing interface, you may be able to follow this strategy. Start by creating a new interface with your new method, or identify an existing interface with the new method. Next, identify the relevant functions that need to support it, type check for the second interface, and add code that uses it. I like this pattern, and I have used it. The issue I have is when I look at the generated godoc, the original interface is in the signature. I don't know the function accepts the new interface as well without either digging into the code or a comment for the function.
- ElCampechano 6y agoHow did you solve the pattern of not having generics in the language?
- swagonomixxx 6y agoTry this: https://www.reddit.com/r/rust/comments/5penft/parallelizing_enjarify_in_go_and_rust/dcsgk7n/ https://www.reddit.com/r/rust/comments/5penft/parallelizing_...
- anonymoushn 6y agoIt seems like this convention invites us to create this sort of issue more times: https://news.ycombinator.com/item?id=6060351 https://news.ycombinator.com/item?id=6060351
- enjoylife 6y agoI use Go daily and one of the aspects which is always a challenge is equality checking. From the post, > There is one subtle way a new field can break user code unexpectedly. If all the field types in a struct are comparable—meaning values of those types can be compared with == and != and used as a map key—then the overall struct type is comparable too. In this case, adding a new field of uncomparable type will make the overall struct type non-comparable, breaking any code that compares values of that struct type. From experience, this is non-trivial to avoid in a long lived codebase. It’s telling that 99% of the codegen tools used in Go, e.g. protobuf, thrift, Gorm, etc. all output an isEqual method of some kind.
- apta 6y agoAnyone who's worked on a large project and code base will appreciate the features proper mature languages like Java or C# offer. It's ironic that golang was supposedly designed for "programming in the large", but it's just a very bad experience for non-trivial code bases.
- kitd 6y agoJava has an equals() method. What features are you thinking of?
- pjmlp 6y ago- Type safe enumerations - Generics (lets see if flyweight design actually makes it) - package names that don't depend on source locations - binary packages for proper encapsulation and even faster builds - both points together allow for components eco-system to thrive - more knobs to turn on when performance matters - a proper graphical debugging experience
- kitd 6y agoI was really thinking about the problem described by GP. But in any case: - Type safe enumerations - Generics (lets see if flyweight design actually makes it) I'll give you both those. Generics would be the killer feature for me. - package names that don't depend on source locations Java package structure mirrors the file system, no? - binary packages for proper encapsulation and even faster builds Go package binaries are cached after build. In any case, build speed is rarely an issue for Go. - both points together allow for components eco-system to thrive One of the most impressive thing about Go IMO is the speed at which the ecosystem has grown and is growing. - more knobs to turn on when performance matters I have found it takes a lot longer to reach the need for performance knobs in Go than in Java. - a proper graphical debugging experience Not sure what you mean by "proper" and why it has to be graphical, but VSCode + Golang extension gives me enough insight into what's going on under the covers at debug time, equivalent to eg Eclipse + Java debugger. Just for the record, I love Java and have been programming in it for over 20 years. It has many great features and the ecosystem is unsurpassed. Go is a much more practical language though IMHO. The time to value is much shorter. Setting up a module project is one command. Writing & running tests, complete with coverage and performance, is supported out of the box by the basic "go" command. The standard lib is broad and deep. And as mentioned the ecosystem is growing fast.
- anonymoushn 6y agoHas something changed such that adding an exported field to a struct is now safe in the case of clients who are using struct embedding? Previously: https://blog.merovius.de/2015/07/29/backwards-compatibility-in-go.html https://blog.merovius.de/2015/07/29/backwards-compatibility-...
- jbamsterdam 6y agoNothing's changed, and that's still an issue. We didn't address it because the post was long enough, but as Merovius himself says, it's called out in the Go 1 compatibility promise. So the Go standard library doesn't worry about it, and neither should you. It's also extremely rare; I've never seen it in 10 years of Go programming.
- kstenerud 6y agoWhat the go team fails to realize is that they haven't actually reduced complexity in their design of go; they've simply MOVED it. And it's been moved in unintuitive ways as they've painted themselves into corners that should have been avoidable with some careful planning. We now have: - magic file names - magic directory names - magic comments - magic env vars - API design requires gymnastics and juggling since recursive imports are disallowed - A ton of useful code in the standard library remains unlayered and unexposed for no good reason (pprof, for example) - Requiring full paths rather than relative imports complicates everything when you're fetching it from a different source location (replace directives and magic ENV vars). - Slice implementation means that you need to do double pointer storage to keep track of the same slice from multiple places - Creating arrays with a dynamically defined length is horribly overcomplicated - Their policy on warnings and instead calling every little thing an error is terrible and wastes a ton of developer time. - Lack of runtime const value support is just sloppy. - Why do go source files even need a package declaration? It serves no useful purpose since the compiler already knows enough to spit out an error if you get it wrong. - Refactoring across packages is nearly impossible to automate because there's no way to declare adherence to an interface. - No importing of test code from other test code (which means that the import code in the compiler has increased complexity to enforce this silly rule). - Stack overflows are STILL impossible to track down (after a DECADE!) because it only prints the top of the stack, which is useless in such a case. - Linting is too opinionated and is all-or-nothing. ------------------ I could go on. I really wanted to like go, and many parts of it do actually feel nice. But even after a decade it's a huge mess. I've spent more time writing workarounds for go than for any other language in my 25 year career. Hell, I even modified the damn compiler out of frustration! [1] [1] https://github.com/kstenerud/go#the-go-programming-language https://github.com/kstenerud/go#the-go-programming-language
- dan_hawkins 6y ago> - Why do go source files even need a package declaration? It serves no useful purpose since the compiler already knows enough to spit out an error if you get it wrong. I know about one use for it: package mypackage_test allows you to exclude test code from builds. I agree with the rest of your points though :)