9 ms·
Go and Versioning: Minimal Version Selection
- kyrra 9y agoThe table-of-contents for these posts are here: https://research.swtch.com/vgo https://research.swtch.com/vgo Though part-4 hasn't been added to it just yet. This is the next post in the series discussed here: https://news.ycombinator.com/item?id=16421966 https://news.ycombinator.com/item?id=16421966
- pknopf 9y agoOne thing I don't like is that sometimes when I reference a go library, sometimes it's vendored dependencies are exposed. This forces me to constantly keep my vendor file in sync with theirs. Is there something we can do about forcing a check to ensure things aren't out of sync? Maybe a module can define dependencies as internet/external? If a dependency is marked as external, the referencing library must reference that same exact dependency/library version?
- bbatha 9y agoVendoring is not supported by vgo so it’s not an issue
- pknopf 9y agoIf library authors can expose types of their dependencies, it still is an issue. Consumers must use the same exact version.
- bbatha 9y agoThat’s untrue. You will only be able import the library once and that version will be common across your all of your dependencies. Only major version breaks change this equation, but as explained in the third article those imports will have different names and are effectively different libraries.
- deleted 9y ago[deleted]
- pknopf 9y agoHmm, can I reference multiple versions of the same library? Not that I'd want to, just wondering how it works.
- Skinney 9y agoOnly different major versions (that is, you can reference a/v1 and a/v2 in the same project, because the import path is different).
- bbatha 9y agovgo will guarantee that you only receive a single resolved minor version for any given major of a package. Each major will be imported as a different name, e.g. `import “registry.io/my/pkg”` for v1, `import “registry.io/my/pkg/v2”, etc. Because vendoring is disallowed by vgo that means that you don’t need to worry about issues like `import “registry.io/pkg”` conflicting with `import “yourlib/vendor/registry.io/pkg”`.
- stubish 9y agoMajor versions of the same library are available, referenced by different paths. It works if they don't reference any global resources outside of their own package. If they do reference any global resources (such as the using the standard flag package, paths on the filesystem, mutexes etc.), then things can fail badly. Yes, this means if one of your dependencies pulls in a different major version of a package than used by other dependencies, then they can end up duelling over resources.
- y0ssar1an 9y agodep will ignore the library's vendor directory entirely. You don't need to do any of that work yourself. dep will do it for you. https://github.com/golang/dep https://github.com/golang/dep
- bbatha 9y agoI still remain unconvinced at the lack of lockfiles. I’m in complete agreement with rsc the minimum version algorithm produces more reliable builds, ie version selection is deterministic. However, lockfiles do more than select versions deterministically, they also hash the packages so that you can be guaranteed that version 1.1 today is the same as 1.1 tomorrow. I also have philosophical concerns about selecting the minimum version: people release patch versions for a reason. I hope vgo only uses the minimum for minor versions not patches (even then I still prefer the newest minor — most of the fixes I release are not back ported as patches). However, I don’t think this algorithm is a bad choice and it seems to fit the go philosophy nicely as it’s simple and to make it “better” gets into many thornier issues that may even involve language changes ala project jigsaw for Java.
- uluyol 9y agoI don't see why hashing is incompatible with the minimum version algorithm. As far as I can tell, the algorithm will always pick versions that are specified in one module file unless a version that doesn't exist was added by hand. If the algorithm always selects a version specified in modules, then that's a perfectly good place to put hash values. This should probably be addressed explicitly, but it's only been what, 1 day, since vgo was announced?
- bbatha 9y agoIt’s not. But it’s not in the current tool. And as this article devoted a long section to why they’re not required for reproducibility I suspect the go team isn’t planning on added one. It’s especially important to give this feedback early in the process so this critical functionality of the tool is not omitted.
- bradfitz 9y agoSee Part 5: https://research.swtch.com/vgo-repro https://research.swtch.com/vgo-repro
- 9y ago
- shurcooL 9y agoOne of my biggest fights is getting people to use latest versions of high-quality software. More often than not, it avoids problems that are already resolved at tip. Helps discover newly introduced issues quickly and before it's a lot of trouble to fix them. Overall, I think it amortizes the cost of software development and reduces it on average, even if there are more smaller bursts where fixing work needs to be done, it's often trivial to do so. To that end, I even created Go Package Store [0] to make updating packages in one's GOPATH easier and more transparent. I use it at least once every few days to update my dependencies and deal with anything that comes up (rarely; most often I just need to leave a review comment on the fresh CL/PR). It's a great experience for me. Despite that, I have a feeling I might like MVS better than the traditional manifest+lock systems that try to pick the latest versions at project start time, but then keep them frozen there until people explicitly update. It makes people less likely to update on their own because they know a new project would start off with latest versions of dependencies anyway. With this system, you couldn't rely on that, and instead you have to update yourself explicitly from the start. I hope this knowledge gets people to take on that responsibility more often and makes it a good habit. Don't update when you need build stability, but when you have a quiet period and are able to update for some API changes, do so and reduce your tech debt for the future. > More than anything else, I wanted to find a version selection algorithm that was understandable. Predictable. Boring. Where other systems instead seem to optimize for displays of raw flexibility and power, minimal version selection aims to be invisible. I hope it succeeds. I'm keeping an open mind and looking forward to seeing how it works out. I have a feeling I might like it despite it seemingly optimizing for the opposite of what I fight for. I'm hopeful. [0] https://github.com/shurcooL/Go-Package-Store#readme https://github.com/shurcooL/Go-Package-Store#readme
- snarfy 9y agoIt's OK as long as you are in charge of when the updates happen. Your system is good in that it is a manually triggered process. It becomes a problem when a full rebuild includes deleting external dependencies, downloading latest sources, and rebuilding everything. When that happens your build environment is a constant moving target. The problem grows exponentially with the number of dependencies you have. When you have a project with a 1000+ dependencies, all pulled latest, nothing will ever compile, let alone your own code.
- mseepgood 9y agoI think we're witnessing a revolution in dependency management similar to gofmt.
- sagichmal 9y agoDefinitely not. Nothing here is particularly novel, `vgo` just stakes out a different position on all of the relevant spectrums than most (all) other package managers.
- xyzzy_plugh 9y agoI, too, am unfamiliar with prior art here. Are you able enlighten us?
- mseepgood 9y agoThe same could be said about 'gofmt' in relation to all the pretty printers that existed before, yet it was revolutionary.
- sagichmal 9y agoThat's a stretch.
- pcwalton 9y agogofmt is only revolutionary in that it refuses to solve the hard problems that pretty printers for other languages solve. Specifically, it doesn't wrap long lines--ever--even though making visually pleasing line breaking choices is pretty much the hardest part of pretty printing (look at how clang-format deals with it, for instance). This design decision would be a non-starter for almost all other languages, where long lines occasionally happen. In other words, gofmt showed that it's easy to write a pretty printer if you ignore line breaking and instead force everyone to accept long lines. This is not a particularly interesting, and certainly not revolutionary, result. (Not coincidentally, I have similar concerns regarding this Go versioning proposal and lockfiles.)
- lboasso 9y ago> Note that F 1.1 requires G 1.1, but G 1.1 also requires F 1.1. Declaring this kind of cycle can be important when singleton functionality moves from one module to another. Our algorithms must not assume the module requirement graph is acyclic. I thought we cannot have cycles in Go packages. The Go spec says "The importing of packages, by construction, guarantees that there can be no cyclic initialization dependencies." What am I missing?
- abbiya 9y agoHas no one ever come up Minimal Version Selection before ?
- sseth 9y agoThe minimal version selection addresses a problem that other package managers do not seem to : For transitive dependencies, it allows projects to use the version that direct dependency was actually tested with. This sounds like a good thing to have. What would be nice would be a command to upgrade to latest of the direct dependencies - and then transitively pull in the latest version of packages they have been tested with.
- jhoechtl 9y agoAs I understand it right now, vgo get -u (As a reminiscence to go get -u) should do what you want. I might missinterpret your need though. https://research.swtch.com/vgo-tour https://research.swtch.com/vgo-tour
- sseth 9y agoI believe this will get the latest of all the dependencies, not just the direct ones. If my module A depends on B and C, which both in turn depend on D, i want the latest of B and C only, and then transitively pull in the minimal version D compatible with B and C. edit : I can of course do it one by one explicitly for B and C. But it would be nice to qualify e.g. : vgo get -u -directonly