26 ms·

A Proposal for Package Versioning in Go

• pdeuchler 9y ago
  How many iterations of this do we have to go through? Go hasn't had mature dependency management since its inception and this constant thrash is really starting to make things difficult. For all practical purposes dependency management for programming languages is a solved problem, with many open source examples. The only explanation for this constant re-invention that I can come up with (and that's shared among others I know) is that Google doesn't care/need a dependency tool because of their mono repo. Which is rather frustrating since we get features (cough aliases cough) forced down our throats when Google suddenly finds a need for them.
  • schmichael 9y ago
    > Go hasn't had mature dependency management since its inception and this constant thrash is really starting to make things difficult. What constant thrash? In the 6 years since Go 1.0 the only dependency management related change I'm aware of is the addition of `vendor/`. Update: sounds like "thrash" refers to the variety of community vendoring tools. I guess of the few I've used they all felt fairly similar/interchangeable, so it didn't feel like thrashing changes.
    • notheguyouthink 9y ago
      There have been many packages for dependency management, which is not "official" thrash, but thrash nonetheless. Granted, mostly it's been thrash over the last few years. Especially bad since `go dep` was hailed as the king, and then recently abandoned entirely.
    • TheDong 9y ago
      The blog post mentions some of this. It claims " For a long time, we believed that the problem of package versioning would be best solved by an add-on tool, and we encouraged people to create one. The Go community created many tools with different approaches" These many tools were the thrash, including godep, gb, glide, dep, and more. Each of these expressed versions with different manifests, many of them could import other format's, but not all. All-in-all it has been a mess. Sure, the go team has not officially done much, but that's because their stance has been borderline negligent on the topic.
    • benesch 9y ago
      The constant thrash caused by the lack of a standard package management tool—even a de facto standard. Just look at the various package managers that have come and gone over the years: 1. https://github.com/tools/godep https://github.com/tools/godep 2. https://github.com/kardianos/govendor https://github.com/kardianos/govendor 3. https://github.com/robfig/glock https://github.com/robfig/glock 4. https://github.com/rogpeppe/govers https://github.com/rogpeppe/govers 5. https://github.com/Masterminds/glide https://github.com/Masterminds/glide 6. https://github.com/golang/dep https://github.com/golang/dep I'm sure I'm forgetting a few. And just when it seemed the Go team might be ready to standardize around Dep (6), they threw this wrench in the works. I'm of two minds about vgo. I think it has some interesting ideas, but Dep works today, is widely adopted, and has nearly all the features you'd expect from a modern package manager.
      • randomdata 9y ago
        > The constant thrash caused by the lack of a standard package management tool There is a standard, and has been one from the early days. The catch is that the standard is not loved by all, so others have created competing package management systems to fit their own needs.
        • kasey_junk 9y ago
          What is it?
          • lox 9y ago
            go get.
            • kasey_junk 9y ago
              Does that finally work with private repos?
              • sethammons 9y ago
                It always has. You have to edit your gitconfig. On mobile, so I can't paste mine. A Google search should get you there. It did for me like 5 years ago.
    • lobster_johnson 9y ago
      Between "go get", Godep, Govendor, Glide, gb, Dep, and this — and I'm actually omitting a bunch of other, less popular utilities [1] — there's certainly been churn. There's definitely a feeling among Go developers that this problem should have been solved much earlier, and that the Go team's years-long refusal to address it caused the proliferation of mediocre tools (looking at you, Glide) that in many ways made the whole situation worse. When Dep came along, a lot of people breathed a sigh a of relief, because we finally have something okay, and we can go back to being productive instead of fighting dependency management problems all day. [1] https://github.com/golang/go/wiki/PackageManagementTools https://github.com/golang/go/wiki/PackageManagementTools
    • majewsky 9y ago
      > In the 6 years since Go 1.0 the only dependency management related change I'm aware of is the addition of `vendor/`. This. Sure, new tools have appeared, but there's literally nothing wrong if you decide to just stick with godep or whatever you happen to be using.
  • merb 9y ago
    +1 I actually also liked dep with the vendoring approach, it is just better to ci a repo with vendored dependencies (no more Broken Building because of Internet/Proxy or github Problems). And in Go it was/is Even simple to upgrade them. Sadly go is slowly moving away from that Part. the new approach is Bad, because they still pull Sources from Github, this Makes their approach unsuitable. E: damn iPhone autocorrect
    • notheguyouthink 9y ago
      I thought they were adding support for vendoring? Was it abandoned again?
      • saghm 9y ago
        I think GP is referring to the fact that dep may not remain the blessed solution for long; recently, Russ Cox posted a number of articles suggesting looking into alternatives https://research.swtch.com/vgo https://research.swtch.com/vgo
    • pdeuchler 9y ago
      Agreed. Dep had its warts (20 minutes for a very simple dep ensure...), but it solved the 80% use case in a way that was least objectionable for the majority. The fact that they can't (won't?) even refactor dep to suit their new ideas or try and revitalize the gps project just tells me that they don't really care about solving this for end users and just want to have fun writing a fancy dependency graph solver. Which is cool and all, but at least give me a stable API first.
      • nicpottier 9y ago
        > just want to have fun writing a fancy dependency graph solver There is no graph solver, that's the whole point.
        • kibwen 9y ago
          Other package managers, including the ones mentioned in the OP, don't have SAT solvers either. This isn't a new feature of dependency resolution algorithms.
  • GiorgioG 9y ago
    Sometimes thrashing is good. Consider the pile of crap that is NuGet in the .NET world. Sure there's Paket, but it has nowhere near the adoption/support of the Microsoft blessed NuGet package manager. At least the Golang team can cherry pick the best ideas & lessons-learned from these community-driven package managers.
    • mattherman 9y ago
      What are some of your specific complaints about NuGet? I've never disliked it that much. For the most part it has "just worked" for me.
      • GiorgioG 9y ago
        In large projects, NuGet takes FOREVER to resolve package dependencies - I mean 4+ minutes on a new i7 Thinkpad w/16gb RAM/1TB ssd. Heaven-forbid you want to upgrade a package that exists in all 130 projects in the solution (not my call to have that many projects) - you may as well take a long lunch. I will try to make that the last task of the day so I can let it run for as long as it needs to. The VS UI for NuGet is terribly buggy.
        • pjmlp 9y ago
          Uauu! 4+ minutes. Try to build a C++ or Android project and then go for lunch instead. NuGET is great.
          • GiorgioG 9y ago
            That's 4 minutes to resolve one NuGet package's dependencies - not 4 minutes for a build.
            • pjmlp 9y ago
              Really? I never had it take that long.
  • elliotlarson 9y ago
    No kidding. I'm so impressed with most of Go, but this is just silly. I feel like Rust started out with solid dependency management in place, at least it's been there from the early days. I work a bit with Node and Ruby too. Both have solid dependency management in place. I mean there are warts, but generally the problem is solved. Actually, now that I think about it, Yahuda Katz has had a hand in working out dependency management for all of those technologies, Rust, Ruby and Node. Maybe the Go guys should bring in Yahuda to help out. If nothing else, he could probably help them get out of the paralysis of analysis loop they seem to be in here.
    • sitkack 9y ago
      Rust is good but not great. Being able to import two major versions of a lib into the same compilation unit (am I understanding this correctly?) is a huge win. Rust can have transitive deps that only differ by version number, but they can't come in contact with each other (don't cross the versions). Reread the post, the authors make some great, mature realizations that I hope other languages will listen to. I don't use the language, but I use languages that Go has affected and this is a great thing.
      • msie 9y ago
        > import two major versions of a lib into the same compilation unit When does this situation come up? What is a compilation unit in this case?
        • eat_veggies 9y ago
          If the newer version of some library has some breaking changes, you can migrate bit by bit instead of doing it all in one go.
          • smaddox 9y ago
            If this is the only motivation for multiple dependency versions in the same compilation unit (crate), I'm not convinced. You would be trading off the simplicity of each crate specifying a range of acceptable dependency versions for specifying `N` ranges of acceptable dependencies, and requiring one of each. Much better to enforce one version per compilation unit, and if you want to get complex with your dependencies, then break your project into multiple compilation units.
  • mseepgood 9y ago
    > For all practical purposes dependency management for programming languages is a solved problem It's obviously not solved satisfactory.
    • pcwalton 9y ago
      Yes, it is. Everything can be improved, but I see very few complaints with Cargo, and, most importantly, not complaints that would be solved with minimal version selection. (Most complaints I see are about the lack of package namespacing.)
      • crtc 9y ago
        > I see very few complaints If you live in a filter bubble (survivorship bias).
      • weberc2 9y ago
        I agree that Cargo is awesome and I would be happy to see Go follow suit, but I interpret "package management is solved" to mean "the industry has standardized on a single scheme". There does seem to be convergence in this space, but it's a relatively new phenomenon.
        • sagichmal 9y ago
          Yes, the industry has absolutely standardized on a single scheme: manifest plus lock as separate declarations, SAT solver, single dependency version per compilation unit.
          • weberc2 9y ago
            If that's the case, it's a pretty recent development. Python and JS didn't have this until very recently. Yeah, Go is a bit late to the party, but hardly worth the dramatic criticism in this thread.
          • clhodapp 9y ago
            That may be what things are converging on but in my opinion, any system that imposes a "single dependency version per compilation unit" policy is not a "real" dependency management solution. That approach creates a tremendous amount of friction because it pretty much ensures dependency hell if there are transitive dependencies with separate release cycles (and there very much should be able to be). Edit: I just realized that we may have a different definition of "compilation unit". If you mean "single program" then my point stands. However, if you just meant some sectioned-off part of the program, then I think the drawbacks of that are far less. Basically, I think that it is useful to be able to have access to multiple versions of a datatype (e.g. if you want to write a converter from old -> new) but that use case is far more fringe.
  • weberc2 9y ago
    I agree that the thrashing is a nuisance, but the 'reinvention' has nothing to do with Google except that Go didn't ship with a package manager by default because it wasn't useful to Google for the monorepo reason you cited. Regarding "features forced down our throats", I think that has only happened with aliases, and the community pushed back to the effect that the feature addition was halted until the community had time to properly review it. Mostly I think that was a one-off problem. As for "it's a solved problem", is it? Until perhaps the last year, pip, npm, and most other package managers have been plagued with serious issues. I'm not fond of the situation in Go, but I think it's not unreasonable to try something different and less binding than a package manager until a solution emerges which solves the problems faced by other package managers.
  • 0xdeadbeefbabe 9y ago
    I'm ok for now putting everything in one GOPATH as if I were google. > How many iterations of this do we have to go through? Fewer iterations than Mr. Cox has gone through. Isn't it great?
  • geodel 9y ago
    > forced down our throats when Google suddenly finds a need for them You can say that about Go itself. And there are already other/better alternatives. So why use Go?
    • dstroot 9y ago
      A language is valued by its programmer bench, tooling, available libraries, as well as the language itself. Honest question - what is better than go right now?
      • jerf 9y ago
        You'd have to specify the use case of interest. I don't think there's anything that uniformly dominates Go, but that's terribly interesting since there are so many dimensions of interest for a programming language that there aren't any languages in use that uniformly dominate some other language in use. (You can get uniform domination if you include unused language or esolangs, but who cares.) But there are certainly many use cases for which Go is not the best choice, not in the top 3-5, or straight-up not a viable choice.
      • baq 9y ago
        C, c++, c#, Java, Python, Lua, lisp, Ada... Go has its good sides, but it's just a tool. It's not the best, because tools aren't supposed to be the best. Tools are supposed to be useful for what they're made for.
        • ilovecars2 9y ago
          I have a hard time with using Java to write micro-services. Even though lots of packages, such as Undertow, support non blocking I/O, a lot of the rest of the stack and the programmers that use it don’t know how to do NIO. What happens then is that the SRE/Ops team needs to configure large thread pools which consume lots of RAM just because multiplexing is I/O so difficult. In comparison I can just start goroutines and Elixir processes for fast and simple I/O parallelism almost without thinking about it.
  • _ph_ 9y ago
    Actually I think the process shows a strength of the Go community. The initial lack of a dependency management tool certainly had to do with the practises at Google - as Google employees, the processes of Google certainly shaped the needs of the Go creators. But I am very happy, that they did not pick a random management system, just because they "had to have one". We can see enough bad examples in the wild. They left this aspect out of the language standard, to tackle the challenge later, with proper consideration. As far as I understand all details, Russ Cox has presented a very thoroughly worked out spec, and it seems to solve a lot of problems, other dependency management tools have. It is also completely compatible with Go so far and also fully optional.
    • pdeuchler 9y ago
      >> But I am very happy, that they did not pick a random management system, just because they "had to have one" But they have, multiple times? First it was just giving people links to godep when people complained in github issues. Then it was a tacit "we like this" about gps and glide. Then it was a pretty-much-official-blessing of dep. Now it's a super-official-now-we-mean-it implementation of vgo. I recognize this as a pattern because I do it all the time, software engineers constantly re-write things from the ground up when they don't care about actually solving the problem and just want to play around with cool new ideas. As far as the vgo spec, I'm not against it per se... I like most of the general conclusions, but none of it is new. We've had these ideas in the Go world for a while. Most of them have even been implemented! Why can't we just slowly transition an existing tool? Or maybe implement a common library that individual tools can use to solve the problem in their own way? Oh wait we already did that and it got abandoned. Until we see a tool with a stable API that solves the 80% use case and lasts for over a year I'm not going to get excited about the new flashy thing.
      • ko82jeipp 9y ago
        Welcome to IT as a career. Having started in the 90s, I have been through these same conversations regarding Adobe, Oracle (oh lawd Oracle) MS, Dell, Apple to a lesser extent. “Vendor the industry currently relies on heavily is changing things or yanking the rug again?” The free market works by big players yanking every one else around. Guh.
  • Thaxll 9y ago
    "Go hasn't had mature dependency management since its inception" go dep is perfectly fine for most use case.
    • endymi0n 9y ago
      Agreed, but compared to Go itself, dep is incredibly recent, still in Alpha, not widely used yet and only the last in a series of incredibly many homegrown approaches that created lots of chaos and churn. And when everybody thought the rollercoaster was finally over, vgo came around the corner. It’s a great proposal, no doubt, but it‘s fair to say dependency management and generics were the two largest unsolved daily encountered problems with Go in real life. Let‘s just hope the community gets an Epiphany about the latter one as well soon...
  • zeveb 9y ago
    > The only explanation for this constant re-invention that I can come up with (and that's shared among others I know) is that Google doesn't care/need a dependency tool because of their mono repo. That's a pretty good argument for using a monorepo, don't you think? A single organisation's code should all be in a single repo, which means that a single version of a dependency is used within the organisation, and that updating to a new version of that dependency is a single, self-contained project.
    • ilovecars2 9y ago
      We use a monorepo at our place of work and it’s great, so far. We also use govendor to manage external dependencies in the monorepo. We are rigorous about keeping packages up to date so have never had to run v1 and v2 of a package at the same time.
  • bobbyi_settv 9y ago
    It's far from a solved problem. Many ecosystems haven't gotten past the "single dependency file and lock file" system where an entire project has to use a single version of each dependency. There are people splitting their project into a thousand "microservices" and making them interoperate over HTTP instead of regular function calls so that each part of their project can have its own dependencies.
    • rickycook 9y ago
      also so that a crash doesn’t stop everything like a BSOD, also so that you can use the language most appropriate for the task at hand because assuming a single language is good for everything is just wrong, also to be able to incrementally migrate things to new PAAS/build systems/etc in the future so you don’t do big bang... and plenty more multiple library versions is one of the lowest priorities when doing micro services id say this is not to say that people don’t significantly over use micro services, but there are plenty of hard problems they solve
    • JepZ 9y ago
      Yes, its kinda funny, I mean package managers are probably the ones with the most solid experience in dependency management and when I look at them I can see many different concepts at work there. Some like for example pacman are bold and are not afraid to break a system during upgrade (as it is very unlikely that the upgrade will fail) and others put an abnormal amount of effort to ensure an always functional system at any time (especially on source based distributions, e.g. paludis). And while I have been using development snapshots for years with my system and didn't have more bugs to cope with compared to other systems, I can understand that developers in corporate contexts need to be able to specify dependencies with versions. So I appreciate that proposal very much and hope it will lead to a better solution for every body involved by providing a way to use versioned dependencies without introducing too much overhead for those who don't need versioned dependencies.
      • flukus 9y ago
        Pacman, apt-get, et al also have something most language specific package managers don't, maintainers. Maintainers back port security fixes and are careful not to break binary compatibility. A random $lang specific package is unlikely to have either, if you expect a bug fix you have to upgrade to the latest version. Developers that don't want to provide this sort of stability typically hate distro package managers. No package manager can fix a cultural problem.
  • sethammons 9y ago
    > For all practical purposes dependency management for programming languages is a solved problem Hey Phil! It's been a long time. C'mon man, when we worked together, are you telling me you never ran into the never-resolving dependency map with berkshelf and chef? I know I did many times. Locks and a manifest did not solve that. It sounds like the proposed solution should prevent that.
  • ngrilly 9y ago
    Have you read the post? It provides answers to your claim that "dependency management is a solved problem".
  • nvarsj 9y ago
    > How many iterations of this do we have to go through? Many, I think, based on what the author of dep says (https://sdboyer.io/blog/vgo-and-dep/ https://sdboyer.io/blog/vgo-and-dep/). I don't get why rsc would just throw away the code and work that was the culmination of years of community work. Isn't rsc largely responsible for creating this problem in the first place? I'm doubtful he's going to fix the dependency problem from scratch. There's a certain pigheadedness to a lot of golang decisions, and I'm afraid this is another one of them...
    • shabbyrobe 9y ago
      I definitely see this with Go decisions too, but I think that same "pigheadedness" has also been a driving factor behind a lot of what makes Go so effective to work with. They've thrown out an awful lot of stuff, which sometimes makes me bristle but I usually have to admit they were right. I think they threw out a bit too much, but that's only a problem if the gaps don't get filled or the compromises aren't adequately explained. What's left is a very pragmatic selection of extremely useful tools that can be used to cut a tonne of working code very, very quickly. I can't say I always like it, but I benefit tremendously from it.
      • nvarsj 9y ago
        I really like Go overall, especially for systems work. But there's various quirks that hurt my day to day productivity, and I think they only exist due to stubbornness. For example, why do unused variables and imports, a style issue, cause a compiler error? This is a constant pain if commenting out lines of code during debugging or prototyping. On the other hand, the compiler doesn't care if I ignore or forget to check error values - which is almost always incorrect behavior.
        • shabbyrobe 9y ago
          Yeah for sure, it's definitely got its warts. I suspect the import thing was far easier to code as an error than to compute whether they need to be pruned. Gotta love those go build times though! With vim-go and goimports, it's handled automatically for me on save so I don't find that an issue any more. Unused variables, on the other hand...
  • mjevans 9y ago
    What are the "benefits" of this approach? To re-iterate a different post of mine against this question (implicitly, what are the benefits of NOT having library versions): * The only version to test against is HEAD * No fossilization of security or stability issues * Public libraries must support all Uppercased (exposed) declarations; that is the library interface. * If the build breaks in an odd way: update go, then: go get -u all * Simplicity, there's only one supported version and only one version to go get and develop against. * Also, why would building against an old version _ever_ be necessary?
  • saturn_vk 9y ago
    But it has had mature dependency management tools for a long time. They were similar to cargo and npm.
• TheDong 9y ago
  Sam boyer had a take on this which is worth reading considering he is the current developer of dep: https://sdboyer.io/blog/vgo-and-dep/ https://sdboyer.io/blog/vgo-and-dep/
  • i2cpsi 9y ago
    To preface this I have not looked into the internal details of vgo, so it could be the world's most interesting solution to this problem. I feel like this is a punch in the gut for a lot of people that spent tons of time looking at ways to solve dependencies in Go. From his post, it looks like Russ never really looked into dep closely as he gave it his "blessing" during gophercon. It seems like whenever there is an issue that gets brought in a proposal (not always but it seems like a lot) it gets shot down by the "higher ups". Not until it gets their attention, or it becomes a big deal to someone (Cloudfare) then is it important. Look don't get me wrong, vgo could be a great technical solution, but disregarding the work of those who actually cared about this issue before you did is a big mistake. It will alienate people in the community who care on improving the ecosystem for those outside of Google. I want to thank all of those involved in dep, it's a great tool that solved my dependency problems! Without you, we would not be having the discussion about vgo and a better way to dependency management in Go.
    • dilap 9y ago
      It's tough, but sometimes you need to throw away work. It's better ultimately in the long run. I think Russ has been doing a pretty good job of giving recognition and credit to the dep folks for their exploratory work.
    • derekperkins 9y ago
      Saying that dep and prior art has been ignored or not "looked into" is unfair. Russ is very clear about where dep and similar tools fall short of a perfect solution. Not only that, but Google has contracted Sam to help with vgo, since he presumably has the most Go experience in the matter. I'm sure it's disheartening for people who worked on dep. I've had a great experience with it too, especially after the .4.1 release. While not everyone agrees, one of my favorite things about Go is that every decision is well thought out in terms of its effects on the entire ecosystem. Just because some other language or toolset does something isn't a good enough reason to force it into Go. I trust the overlords, they've been good to me so far. :)
• nine_k 9y ago
  In short: they follow the Rust's approach. Great! Update: no, I'm wrong.
  • TheDong 9y ago
    That is inaccurate. In short, Russ Cox has been bashing cargo's approach of preferring newer versions and lockfiles and a SAT solver (or really an approximation of one). Apparently making the dependency-manager's code simpler is a goal worth removing developer's ability to be expressive and removing security updates.
    • Conan_Kudo 9y ago
      I'd be surprised if the Go developers ever care to do this right. Google doesn't use any of this, and all their projects do extensive vendoring with heavy modifications (effectively forking them). Personally, I despise their approach to developing software, and on principle, I won't really develop software in Go because their attitude permeates through language and software development mechanisms.
    • tcheard 9y ago
      > Apparently making the dependency-manager's code simpler is a goal That isn't the goal at all. The goal of Minimal Version Selection is to ensure that the version selected is closest to that which was tested by the app and its dependencies. > worth removing developer's ability to be expressive and removing security updates. And that isn't true either. It allows the developer of both the app and its dependencies to express the version they have tested against, and still allow the developer to receive security updates by explicitly increasing the version, allowing them to test that the update doesn't create problems while doing so. It allows for version pinning without the need for a lock file.
  • notheguyouthink 9y ago
    I love Rust.. but isn't this approach very different from Rust? Rust doesn't even have the concept of import paths, which is almost entirely how Go is doing versioning here. Sure, they have verifiable builds, but even JavaScript (Yarn/etc) has had that for years.
• the_mitsuhiko 9y ago
  I want to know what they consider the limitations of the cargo/rust approach. It has been really nice to use as a user.
  • strkek 9y ago
    To me, the one thing I'd like to change about Cargo is that required initial clone of a 100MB+ git repository before you can even install something. IIRC this was because a libgit2 issue preventing them from doing a shallow clone though, so there's no way around it for now. Disclaimer: I use both Go and Rust on a daily basis and think both are nice in their own way.
    • the_mitsuhiko 9y ago
      That’s more of an implementation detail though. The design does not demand that. I assume that no concern initially when the ecosystem was tiny.
      • ngrilly 9y ago
        I agree, but it's not a detail in practice. My feeling is that Rust is very attentive to theoretical details, and Go to practical details. Of course it's an over-simplification, and both approaches are pertinent and complementary ;-)
        • kibwen 9y ago
          That's the exact opposite of what the grandparent is talking about, though. Cargo was the one saying "this works for now, we'll fix libgit2 later", which is firmly in the practical camp. vgo is the one saying "we can't emulate other package managers because SAT solvers are slow", which ignores that in practice they're not, valuing strictly theoretical considerations instead (and, in practice, Cargo doesn't even use a SAT solver anyway, so they didn't do their homework).
          • ngrilly 9y ago
• nemo1618 9y ago
  I'm excited to see Go trying something new here. Something just feels clunky about the current "manifest+lockfile" approach. There are a couple of ideas in vgo that I really like, e.g. "incompatible versions must have different import paths." I think rsc is justified when he compares these conventions to upper-casing exported identifiers in Go packages: for a while it seemed weird, but now it feels natural and obvious. In other words, Go has a history of breaking from conventional approaches, with surprisingly consistent positive outcomes. Some bikeshedding is inevitable if this proposal is accepted, but I hope that the main ideas are preserved.
  • wwwigham 9y ago
    > I'm excited to see Go trying something new here. "Semantic import versioning" is equivalent to versioned APIs in the web world, no? Though I have never seen anyone utilize the same concepts for language-level packages; it certainly maps well!
    • Shoothe 9y ago
      For the record Roy Fielding is against using version numbers in URLs: https://www.infoq.com/articles/roy-fielding-on-versioning/ https://www.infoq.com/articles/roy-fielding-on-versioning/
      • sitkack 9y ago
        Roy Fielding isn't a deity. The version would be encoded in the 'Accepts' content type header. There is little distinction, one is in-band (in the path) and one is out-of-band (in the header), but the information is still represented.
        • dordoka 9y ago
          He is not a deity, of course, he's "just" the one that invented REST. I guess what you mean is that he's not right about everything he says. Well, what he exactly says is that URLs should not include versioning, because URLs are the interfaces names and REST estates that interface names should not be versioned, as that implies a breaking change in the API. It's just the wrong place for versioning in the REST way. But he is not against versioning, as you say, you can use the Accept header. You could also use a custom header if you might, but the canonical way would be the Accept header.
• fwip 9y ago
  My initial reaction to Minimal Version Selection (MVS) is concern, that developers won't get security updates and bug fix patches in their dependencies applied. But dependency software has been trending toward the use of lock files for a while now - and without explicit developer intent, those won't get bugfixes either. I think I'm mostly concerned about how this affects transitive dependencies. My package Foo depends on Bar_1.3.0, which depends on Baz_3.2.4. If Baz gets a security update to Baz_3.2.5, either I need to add an explicit dependency "Baz_3.2.5" to Foo, or wait for Bar to release 1.3.1 that depends on Baz_3.2.5. If go adds tooling to identify and make these transitive dependency upgrades as easy as "npm update", then I will be a little bit less uneasy.
  • xiaq 9y ago
    Exactly. MVS may sounds weird at first, but in practice it works identically to having a Lockfile (which is becoming the standard these days), just simpler.
  • skybrian 9y ago
    I don't think libraries have to do this, just applications. As a library owner you're not responsible for making sure downstream gets all the latest security patches. They can do it themselves at will, by running "vgo get -u". https://research.swtch.com/vgo-tour https://research.swtch.com/vgo-tour
    • mfer 9y ago
      The example there dealt with direct dependencies but what about transitive (dependencies of dependencies)? If you have a module that asks for an old version of a dependency what will upgrade that? minimal version selection will keep with the old transitive versions. That's where there concern comes in.
      • deleted 9y ago
        [deleted]
      • skybrian 9y ago
        I think the -u flag might already be transitive? Haven't tried it, though.
• twhb 9y ago
  Why not just put the full version number in the import path? Both the manifest (dependency section) and lock files become unnecessary. It’s DRY. Dependencies are specified where they’re used, improving componentization. Upgrading and forgetting to update one location is easily fixable: tooling already scans go code for a list of imports, modify this to warn on version differences. Or even to update them. Git history gets “cluttered”, but shouldn’t it be? The behavior of values in a file is changing. This constitutes a change of requirements on the file’s code, or at least needs a moment’s review to decide the code needn’t change. Seeing that change in history would make tracking down any bugs it causes easier. Besides, we’re talking more files changed in a single edit, not more edits. Semantic versioning is a qualitative description, not a guarantee. Due to edge-case use or human error, every minor or patch update may be a breaking change. It would be better to have a layer of human interpretation between semantic versions and code changes, rather than tooling that assumes them to always be correct.
  • zamalek 9y ago
    I like the idea too. Something along the lines of import "fmt#1.2.3". The problem is that you do repeat yourself: you have to sync the version across any files that use it.
    • twhb 9y ago
      It's not repetition if it sometimes varies, as the article discusses. It's certainly a common case that they're all the same, and we can handle that by augmenting go, which already scans the project for import lines, to warn on different versions, or even update them in the import lines.
  • skybrian 9y ago
    It doesn't scale. If you have a->b->c and b version-bumps every time c does, then this also means a has to version-bump. So basically changing one file means everything that depends on it, even indirectly, has to change. Furthermore, if you have diamond dependencies, a might end up version-bumping more than once. Regular edits would get lost in a sea of version-bumps.
• Keats 9y ago
  It says it is reproducible but is depending on mutable git repositories? In an earlier post they mentioned proxy servers to have some kind of cache, will there be an official one?
  • strkek 9y ago
    Well, you'll always have to trust some point on being immutable. A git repository can mutate, but so can a central registry or a proxy. Once a version is cached in a registry or a proxy, nothing stops people with enough privileges from modifying that cached version. So really the only thing that changes is just who you trust.
    • Keats 9y ago
      crates.io is immutable, you can't edit/delete published versions. And npm is as well finally. > So really the only thing that changes is just who you trust. With an immutable central registry, I only need to trust the organization running it. Without it, my build depends on the maintainers of all my dependencies (including transitive) to build. In the case of Go that would be trusting Google versus trusting an unknown number of random persons. Even recently in Go there was this event: https://www.reddit.com/r/golang/comments/7vv9zz/popular_lib_gobindata_removed_from_github_or_why/ https://www.reddit.com/r/golang/comments/7vv9zz/popular_lib_... where someone deleted their account and someone else signed up and created another repository with the same name. I believe the Minimal Version System will ensure that no one gets automatically updated to a new version of potentially bad code there though.
      • zbentley 9y ago
        If you want an artifact repository for Go, several exist with varying levels of immutability/reliability/etc. However, Go also has an answer to the situation of "I want dependencies directly from the community without a caching/authorizing intermediary, but don't want to be a victim of someone rewriting their history on GitHub/account-squatting/whatever", and that answer is vendoring plus checking diffs on package upgrade. That's often less convenient than using a trustable caching registry, but it's comparing apples and oranges: a cache like that requires a centralized solution at present, though usable distributed ones might come about at some point; vendoring just requires some of your disk space and a git clone.
• munificent 9y ago
  > It is of course possible to build systems that use semantic versioning without semantic import versioning, but only by giving up either partial code upgrades or import uniqueness. Cargo allows partial code upgrades by giving up import uniqueness: a given import path can have different meanings in different parts of a large build. Dep ensures import uniqueness by giving up partial code upgrades: all packages involved in a large build must find a single agreed-upon version of a given dependency, raising the possibility that large programs will be unbuildable. ... Semantic import versioning lets us avoid the choice and keep both instead. It's a useful exercise to be critical of existing systems and see if there are opportunities to improve that they missed. That's how progress happens. At the same time, there is a common fallacy (especially among very smart people) that this might be an instance of. Even if it isn't and the Go folks came up with something brilliant, I think it's worth talking about because it occurs a lot elsewhere. I'll call it the "Missing Third Feature". Cox's claim boils down to. "X gives you A but not B. Y gives you B but not A. I've just come up with Z which gives you both A and B, so it's superior to both." In many cases, though, the reality is. "X gives you A and C but not B. Y gives you B and C but not A. I've just come up with Z which gives you A and B, but not C (because I'm likely not even aware that C exists)." I could be wrong, but if I had to guess, C is that vgo has no ability to express compatibility across multiple major versions. Let's say my package foo depends on bar. The maintainers of bar add a parameter to a function in it as well as fixing a number of other bugs. That signature change is a breaking change, so they rev foo to v2. My package bar does not call that function. In order to get the bug fixes, though, I have to change all of my imports to use foo/v2. Anyone using bar must either fix all of their imports of foo to v2, or end up with two versions of foo in their application (which might cause weird behavior if values from one foo end up flowing to another). The key problem is that from a package's own perspective, "breaking change" is defined conservatively — any change that could possibly break even a single user is a breaking change and necessitates a major version bump. From a package consumer's perspective, many nominally breaking changes do not actually break that particular consumer. Package managers like Cargo let you express that gracefully. My package foo can say, "I depend on bar and work with bar >1.2.3 <3.0.0" if I know that foo isn't impacted by any of the potential breakages in bar 2.0.0 or 3.0.0. That in turn lets my package be used in a greater number of version contexts without causing undo pain to my consumers. This may not turn out to be a big deal. It's hard to tell. But my general inclination when I'm designing a system is that if my idea seems unilaterally better than the competition in all axes, then I strongly suspect those systems have a feature that I am oblivious to and that mine lacks and I try to figure out what it is they know that I don't.
• mfer 9y ago
  Minimal Version Selection This is the piece that is very different from other dependency managers and is worth people looking at. Instead of trying to get the latest version it's going to try to get the oldest. If you want to update to newer version of transitive (dependencies of dependencies) that your dependencies have not updated for you'll need to start tracking those yourself. There's an issue touching on this at https://github.com/golang/go/issues/24500 https://github.com/golang/go/issues/24500 Other packages managers use the opposite of minimal version selection. Many of them even have a don't make me thing command to update the whole dependency tree (e.g., `npm update`). What do folks think of the implications of MVS, especially for transitive dependency management?
  • bishop_mandible 9y ago
    > Other packages managers use the opposite of minimal version selection. Many of them even have a don't make me thing command to update the whole dependency tree (e.g., `npm update`). vgo does have that too: vgo get -u
    • mfer 9y ago
      That will work for direct dependencies but not dependencies of dependencies. See https://github.com/golang/go/issues/24500 https://github.com/golang/go/issues/24500
      • nicpottier 9y ago
        Correct, it will get the newest version that is actually tested against your dependencies, which is by design. Why would you want to pull a version that is newer than the author of the library you depend on has actually tested with? You can of course force this in vgo, but having the default use the versions specified by the authors makes a whole lot more sense than just using the newest.
        • mfer 9y ago
          > Why would you want to pull a version that is newer than the author of the library you depend on has actually tested with? 1. To install a security update or bug fix update you need in a transitive dependency that the author of the dependency you're using hasn't updated to. 2. To use the same workflows across all my dependency management tools (npm, cargo, composer, bundler, and the rest of the lot follow the same patters and vgo goes against the patterns used by the others) There are two reasons.
• colek42 9y ago
  We have just moved to using dep, yes, it exposed some issues in our dependency tree, but it works now and I don't think our organization will be looking to move anytime soon.
• teabee89 9y ago
  To all the skeptics, please try it out and provide constructive feedback. Experiment with the corner cases you think won't work. Judging just by the blogpost won't help anyone. Personally, I'm very excited and impressed by the ability of the Go team to innovate by diving deep and understanding every aspect of the problem at hand and the existing solutions, instead of just blindly adopting whatever already exists.
  • deleted 9y ago
    [deleted]
  • bigdubs 9y ago
    The lack of support for private repos is still a dealbreaker for both `dep` and `vgo`.
    • JepZ 9y ago
      How is it different from using 'go get'?
      • Groxx 9y ago
        `go get` has so many other blockers for enterprise use that "support for mirrors" barely registers. But yes: it's no different, `go get`'s lack of mirror support is also a blocker for tons of businesses.
      • bigdubs 9y ago
        govendor anyway will use local versions of deps if you have them in your gopath
    • teabee89 9y ago
      I wanted to find out, so I looked at vgo's source code, and found that vgo does indeed have some solution to this using Github Access Tokens and ~/.netrc: https://github.com/golang/vgo/blob/b6ca6ae975e2b066c002388a896833851a9e54a9/vendor/cmd/go/internal/web2/web.go#L271 https://github.com/golang/vgo/blob/b6ca6ae975e2b066c002388a8...
• jasonwatkinspdx 9y ago
  Really glad to see the change of course on this issue.
• bishop_mandible 9y ago
  I feel like this is the package management equivalent to the shift from geocentrism to heliocentrism. Yes, you can approximately predict the movements of the planets with the geocentric model and lots of epicycles (Bundler/npm model), but it suddenly makes so much more sense and it's so much simpler if you assume the sun as the center of the solar system.
• ramses0 9y ago
  Much love to the author for clearly thinking through. As GIT was to DVCS, and SemVer to individual package versioning, I feel this proposal is "the one true way" as a starting point by which all future progress will be judged.
  • strkek 9y ago
    I don't feel like Git is "the one true way" to DVCS though. For example using SHA1 as hashing algorithm doesn't seem like a good idea. Or calculating "packfiles" ad-hoc on clone operations, that's wasteful and makes cloning big repos on unstable connections a PITA. But it's certainly true that Git is the "current best practice" in the DVCS landscape.
  • gkya 9y ago
    > As GIT was to DVCS... ? Git did not create a novel concept. For one, it was written nearly contemporaneously to Mercurial, which is DVCS too and uses a very similar if not identical model. SemVer is a similar story too. It was not like before it people were using /dev/random to number their releases. I really like the fact that Go is catching up in this area, but I fail to see the revolution everybody's talking about here.
    • geodel 9y ago
      I think 99% of comments are wtf... they should have copied package management xyz 5 years ago.
      • mseepgood 9y ago
        Which is how you don't find better solutions.
• room271 9y ago
  Yes, Rich Hickey (amongst others I am sure) has talked about precisely this model of backwards compatibility - i.e. if you break the contract you have to rename the thing. This feels far better than the current model used in most languages. If you've ever had struggles creating an uberjar you know this pain.
  • DougBTX 9y ago
    Well, the article is suggesting to put the major version number in the name, so that “change the name” and “bump the major version number” are equivelant actions, which is a little more subtle than just a plain rename.
  • stephen 9y ago
    Agreed, after years of JVM dependency hell (which happens in any language), I think "never make breaking changes" should be non-negotiable for publicly-published code. E.g. repo managers like Maven central/etc. should use binary API analysis to reject any jar upload that has breaking changes. My only hesitation is that, AFAIK, semantic import versioning has never been tried at scale, so having to constantly bump imports from "com.foo.v1" to "com.foo.v2", and deal with "app1 wants to pass com.foo.v1 objects to app2, but it expects com.foo.v2 objects" might introduce more pain than expected. Granted, right now app1/app2 are blithely passing around "com.foo" objects that may/may not be compatible, but if it's an 80/20 thing, or 99/1 thing, and most of the time you get lucky and it works, perhaps that's good enough. But would be great to have go be the first community to try this at scale and see how it goes. I like it.
  • faitswulff 9y ago
    Russ actually mentions Rich Hickey's keynote talk, Spec-ulation (https://www.youtube.com/watch?v=oyLBGkS5ICk https://www.youtube.com/watch?v=oyLBGkS5ICk) in one of his blog posts about vgo
• msoad 9y ago
  Isn't yarn locking system a proof that semantic versioning has failed?
  • zbentley 9y ago
    Not at all. Fingerprinted lockfile systems (yarn or otherwise) provide separate benefits from semver: - The potential for deterministic builds. Saying "my system will work after update because all of the updates are minor versions" is a big benefit provided by semver, but saying "it will work after update because the bits of the built files are the same" is a lot more assurance. "No Trespassing" signs haven't "failed" because door locks and fences exist; it's just different levels of effort/assurance. - Security/package verification. - A speed advantage in dependency fetching. Using a lockfile, you no longer have to ask the registry for something that matches your semver pattern, or make multiple round trips. For a single package that advantage is negligible, but across multiple packages with shared dependencies, having a lockfile improves the ability to do parallel package downloads and gives your system knowledge of which dependencies are shared much earlier in the process. - Auditability/visibility. If you check your lockfile in and someone else is having problems reproducing a bug, being able to diff lockfiles makes it very easy to see if your environments differ. Since the full tree is serialized in a predictable place, it's easy to diagnose situations like "you're using a different version of package X, which is causing weird behavior because it's depended on by a package we use". That semver's fault; people put '>' without '<' in semver strings all the time, and that's no more a failing of semver than misspelings are a failing of your text editor. Lockfiles also make it easy to identify the opposite scenario: "our installations are identical; must be something else on your system that's causing the problem--maybe check your environment/kernel rev?" Edits: the soul of wit.
• solidsnack9000 9y ago
  The explicit `v2` in import paths is a case where Go has abandoned conventional abstraction and information hiding in an area where maybe there didn't need to be any in the first place.
• billytetrud 9y ago
  How has go not solved this when it's had npm as an example for 6 years?
  • ovao 9y ago
    It’s explained in the post. To summarize: the Go team felt it would be best if users developed the tooling to solve the problem rather than having an official solution. This resulted in too many solutions, which is itself a non-solution. Pretty clearly this was a mistake by the Go team, and the blog post says as much.
• cdelsolar 9y ago
  I literally just switched to dep.
  • sgarman 9y ago
    Sounds like this is still a ways out right?
  • tcheard 9y ago
    dep is still the officially recommended way to go for now. Russ Cox has mentioned in his blog posts on vgo that they will be working hard to ease the transition from dep to the built in versioning tools.
    • ospider 9y ago
      Which means at some point, we have to switch from dep to vgo. although the transaction could be smooth, but the methodology behind it must have changed, and that would not be a smooth transaction for our mindset.
• solatic 9y ago
  I don't get this. The foundation upon which semantic versioning is built is that the entire semantic version is handled outside the code itself, which allows different packages to communicate how they adhere to a common standard compared to each other. For instance, if I depend on A at 1.0.2, and I depend on B at 2.3.0, even if A and B have nothing to do with one another, as a consumer of A and B, I have a common contract with both of them, which makes it easier to resolve my dependency requirements. In this case, it doesn't matter if B's author never de-facto issues patch releases, because my dependency resolver tool treats them the same. If I break that foundational assumption by putting the major version directly in the code, i.e. "semantic import versioning", what benefit do I have to track minor and patch separately? Why not just replace them both with a timestamp? As a library packager, I fit one of the following situations: * I don't bother with minor or patch versions at all, every release is denoted as possibly breaking. Timestamps mean nothing to me since every major release is effectively a timestamp anyway. * I don't bother with patch versions, only minor versions. Same as before, each minor version is effectively a timestamp anyway. * I don't bother with minor versions, only patch versions. Also effectively a timestamp. * I bother with both minor and patch versions, but after I increase a minor version number, I never issue new patches for old minor versions, e.g. 1.0.0 -> 1.0.1 -> 1.0.2 -> 1.1.0 -> 1.1.1 -> 1.2.0. This is effectively a linear release pattern and therefore can be replaced with timestamps (if your software depends on 1.0.2, and the library packager issues 1.1.0, which is not supposed to break compatibility with 1.0.2, then why wouldn't you upgrade?). * I bother with both minor and patch versions, and I'm committed to releasing patches for old minor versions for a certain amount of time, and therefore my linear release history conflicts with timestamps, i.e. if I release in the order of 1.0.0 -> 1.0.1 -> 1.1.0 -> 1.0.2, despite 1.0.2 having a later timestamp than 1.1.0, consumers of 1.1.0 should not "upgrade" to 1.0.2 since it may break compatibility with features introduced in 1.1.0. Implicit in this versioning practice isn't just the willingness to patch old versions of the software, but the recognition that at least a subset of your users desire to reduce risk as much as possible by avoiding even seemingly non-breaking changes and yet not avoiding patch changes. Is that even rational? Should dependency management for statically-linked binaries (not dynamically linked binaries where patch version updates can be introduced into an environment with a pre-existing binary, not networked APIs where rolling back minor versions can impact other systems using that API, not to mention the effects on backing datastores...), the statically-linked binaries in the end either will or will not compile and pass a test suite, so should dependency management for Go even bother to support this use case? Why not have "semantic import versioning" with different major versions supporting different import paths, and then keep track of the timestamps of each dependency as it was used, with some "go dependencies update" checking for new timestamped versions (remember, none of which should result in breaking changes), keep track of which timestamped versions are used in version control and allow builds without updating the timestamps. This way a) gets you reproduceable builds, b) makes it easy to check if updates will (inadvertently, through a mistake of the library packager) break your build, and roll back if necessary, c) takes advantage of Go's property that it creates statically-linked binaries to ensure that such builds are safe in spite of the reduction in resolution vis-a-vis melding minor and patch versions into one. Dependency resolution is then a trivial matter of compiling a list of all major versions semantically imported in the codebase, throwing them in a file matched with what were at some point the latest timestamp, and optionally checking for and updating the timestamps in the dependency resolution file. If no code references a given major version anymore, remove it and its timestamp from the dependency resolution file.
• Communitivity 9y ago
  I just finished reading the actual proposal. I am very happy to see this addressed in Go. Every time I've pitched Go the versioning has been a sticking point, despite the many great features of Go. A potential problem I see with this proposal though is that when I verify everything works, and is secure, with version 2 of a package I am automatically upgraded to version 2.1 because the import path is 'my/thing/v2/sub/package' which will grab 2.1 when it's available, if I am understanding it right. At that point I am using a dependency I have not run through my additional checks (e.g., static analysis), without knowing it. This is complicated further by the proposal's suggestion that only the major version be allowed in semantic import paths. How then can I fix the version to exactly what I want, with exactly the amount of flexibility I want? I am not sure, but I think that this issue is not completely addressed by the concept of a 'high-fidelity' build, as discussed in the proposal. If the minimally compatible version based on dependencies is 2.1, but I want to use 2.2 then the high-fidelity build won't do as it would be stuck at 2.1. It is significantly more complex to handle, but I'd much rather see something along the lines of 'my/thing/$v/2/1#sub.package'. This also lends itself to things like 'my/thing/$v/latest#sub.package', 'my/thing/$v/2/latest#sub.package', and 'my/thing/$v/2/0#sub.package'. A less complex approach might be to mandate major, minor, and patch version numbers in the semantic import path, but allow 'x' to be used where the author just wants the latest. Examples would then be: 'my/thing/v2.1.1/sub/package'. This also lends itself to things like 'my/thing/v2.1.x/sub/package', 'my/thing/v2.x.x/sub/package', and 'my/thing/v2.0.x/sub/package' I'll leave you with what I think is a relevant quote from Cool URIs Don't Change, by Tim Berners-Lee. "It is the the duty of a Webmaster to allocate URIs which you will be able to stand by in 2 years, in 20 years, in 200 years. This needs thought, and organization, and commitment. URIs change when there is some information in them which changes. It is critical how you design them. (What, design a URI? I have to design URIs? Yes, you have to think about it.). Designing mostly means leaving information out. The creation date of the document - the date the URI is issued - is one thing which will not change. It is very useful for separating requests which use a new system from those which use an old system. That is one thing with which it is good to start a URI. If a document is in any way dated, even though it will be of interest for generations, then the date is a good starter. The only exception is a page which is deliberately a "latest" page for, for example, the whole organization or a large part of it. http://www.pathfinder.com/money/moneydaily/latest/ http://www.pathfinder.com/money/moneydaily/latest/ is the latest "Money daily" column in "Money" magazine. The main reason for not needing the date in this URI is that there is no reason for the persistence of the URI to outlast the magazine. The concept of "today's Money" vanishes if Money goes out of production. If you want to link to the content, you would link to it where it appears separately in the archives as http://www.pathfinder.com/money/moneydaily/1998/981212.moneyonline.html" http://www.pathfinder.com/money/moneydaily/1998/981212.money... -- Cool URIs Don't Change, by Tim Berners-Lee
• jrochkind1 9y ago
  > exemplified by Rust’s Cargo, with tagged semantic versions, a manifest, a lock file, and a SAT solver to decide which versions to use. I think Cargo got this from ruby's bundler, the first modern tool to do all this? Why does cargo get the credit! haha.
  • swsieber 9y ago
    I think it's because Cargo was the blessed solution baked into the paradigm from the beginner. Rust get's credit for a lot of things it shouldn't (well, not credit for being the first), because it pulls many well done things together nicely. It might not be novel in a lot of areas, but is seems like it's got a lot of polish. Or, perhaps it's brought up as example because Rust occupies a space a lot more similar to Go than Ruby does.
  • munificent 9y ago
    Yehuda Katz wrote both, so he gets the credit either way. :)
  • dbaupp 9y ago
    The right people are getting the credit, at least: the team that originally built bundler also created cargo.
  • deleted 9y ago
    [deleted]
• erikb 9y ago
  What I don't understand is why Go is not simply copying something from another community. And yes it will be incompatible with the "old way", but what is the "old way" with such a young language anyways. But then at least you don't need to learn the basics again that have been solved a hundred times already. Just copy&paste and then move on to other topics. And while you are at it, also add real exception handling instead of asking people to reraise exceptions under each and every function call. And instead teach people to actually handle most of the exceptions from underlying libraries, maybe reraising a more context-appropriate one on the current abstraction level.
  • crispinb 9y ago
    > What I don't understand is why Go is not simply copying something from another community The proposal explains this pretty clearly, using Cargo as an example, and listing what the Go team sees as its shortcomings (at least for golang). Is there an alternative extant approach that you think they've missed? > And while you are at it, also add real exception handling They're not going to do this. There are philosophical disagreements between advocates of different approaches here, and Go has very clearly picked its side. https://blog.golang.org/errors-are-values https://blog.golang.org/errors-are-values is a good exposition of the case. If it doesn't convince (or at least mollify) you, you probably should either suck it up, or avoid Go where you can.
    • kasey_junk 9y ago
      It would be way more persuasive if they didn't fall back to panic/recover semantics that are exceptions.
      • ngrilly 9y ago
        panic/recover is a bit different from exceptions because it can be used as a control structure (like if/for/switch) inside a function. Rust has something similar with panic!.
        • 9y ago
• mappu 9y ago
  MVS requires import path compatibility. For semver-major changes the /v2/-style workaround is OK, except when a package is still v0. What happens in vgo if MY_APP depends on A and B, which both depend on different, incompatible v0 tags of C? I understand that A and B can declare which v0 they want in go.mod, but it can't be solved for MY_APP. This unfortunately isn't hypothetical...
• JepZ 9y ago
  I am very happy that Russ Cox et al. are still brave enough to propose new ideas when they see something not working as intended.
• nstart 9y ago
  Having just built my first project in Go, I'd be happy if some form of standardization is introduced. Everything I read was mostly along the lines of "well there's this community standard process we use that's not official but it might as well be". But when I started installing packages, it was clear that the result of this lack of focus on package versioning meant that people were simply pulling packages from master or the latest release on github depending on which tool you used. Examples of how I had to deal with packages: Yaml parsing - Used gopkg.in/yaml.v2 . What that means? I don't know. v2 could be one git hash today, and another tomorrow. Slack SDK - Used nlopes/slack. dep ensure -add dumps me on to the latest release of Github. That was fine although I didn't realise that was what happened. The latest release is 12 commits behind master at the time of this writing A CLI framework - Used urfave/cli. Same thing with dep ensure -add. Except I realised that the last release to Github was eons ago. August 2017 IIRC. The code I needed in particular had been released in October 2017 and since then had many commits to master. I just added a constraint to my Gopkg.toml file to lock the pulling to master. The point is that in all 3 of these examples, each person had an entirely different way of doing things. All 3 libraries are mature in terms of functionality and the people behind them being experienced go devs. I'm not entirely sure where the "Independently, semantic versioning has become the de facto standard for describing software versions in many language communities, including the Go community." statement comes from (the including the Go community bit). Because that was not my experience. If this tool introduces some clarity and helps push the community towards developing and releasing more sensibly I'll be really happy. Otherwise I foresee that instead of using semantic versioning, I'll be locking to different git commit hashes to choose which API version I want.
  • favadi 9y ago
    > Yaml parsing - Used gopkg.in/yaml.v2 . What that means? I don't know. v2 could be one git hash today, and another tomorrow. I'm sure that you can use a fixed git hash if you want. > Slack SDK - Used nlopes/slack. dep ensure -add dumps me on to the latest release of Github. That was fine although I didn't realise that was what happened. The latest release is 12 commits behind master at the time of this writing Do you expect the author to tag a release every time he pushed to master? And you can use master branch if you want. > A CLI framework - Used urfave/cli. Same thing with dep ensure -add. Except I realised that the last release to Github was eons ago. August 2017 IIRC. The code I needed in particular had been released in October 2017 and since then had many commits to master. I just added a constraint to my Gopkg.toml file to lock the pulling to master. It is barely 6 months old! How often you expect the author to release new version?
• Groxx 9y ago
  I'm honestly confused about why "this package manager is a SAT solver" is being trotted out as a bad thing. Repeatedly. Having used multiple such package managers in the past: the runtime is utterly dominated by time-to-download or even simply disk access, not time-to-compute. Compute time has uniformly been far below human-visible times - e.g. a few hundred dependencies resolves in `dep` in around 100ms (for the solver) on my machine. SAT is not a problem at all. Yes, you can construct a worst-case scenario for it that will chew up a ton of CPU. In practice it simply doesn't happen, and trying to defend against it is both a waste of effort and leads to crippled decision-making.
  • icholy 9y ago
    I think it has more to do with the complexity. Minimum version selection is trivial to implement and understand. The processing time is a side-note.
    • Groxx 9y ago
      MVS is less complex to code, definitely agreed. But given how well studied and broadly implemented SAT is, I don't really think that's a useful distinction to its end goal - be a useful build system. Chucking the whole transitive-version-respecting thing is even simpler[1] but it's terrible and Go shouldn't do that just because it's simple. And for "understandability" in a conceptual sense, personally SAT has always felt simple and predictable to me - "find something within all bounds or err" is something we do all the time by hand. [1] e.g. pip essentially ignores version constraints if something's already installed or some other lib already mentioned it. it's absolutely terrible and causes many problems in anything long-lived.
      • dilap 9y ago
        A non-SAT system is understandable in the sense that you can, as a human, look at it and understand what the solution is should be. That's kind of cool. It also seems within the realm of imaginable that having an easy-enough-to-implement scheme that you're not falling back on writing heuristic backtracking search might result in more tools being written. Obviously, you are right that being NP-complete is clearly not a showstopper or a huuuge problem, in practice. Still, seems nice to avoid if you can! By the way, that's interesting to hear about pip. I do my best to avoid python, but I inevitably end up reluctantly wanting to use some tool written in it, and inevitably it breaks. What a fucking shit-ass ecosystem.
• snarfy 9y ago
  I was recently tasked with learning Go. Overall I liked it, but the lack of versioning on packages was the deal breaker for using it for any big project. The bigger the project, the more dependencies, and the more things that can go wrong upstream.
• deafcalculus 9y ago
  Why won't the simple approach of not allowing packages to break API compatibility work? Fixed some bugs or added new functions to a library foo? Keep calling it foo. Made an incompatible change? Call it foo2. At least, this encourages backward compatibility. Incrementing the version number is an easier decision to make than explicitly creating a whole new thing. One of the awesome features of Go is how stable the language and standard libs are. Why not expect the same of the community?
• twic 9y ago
  Okay, so i'm reading the minimal version selection algorithm article [1], and there's either a flaw, or something i don't get. Here's a modified version of the example graph, written as pidgin DOT [2]: A -> B1.2 A -> C1.2 B1.2 -> D1.3 C1.2 -> D1.4 D1.3 -> X1.1 D1.4 -> Y1.1 The key thing here is that D1.3 depended on X1.1, but the new D1.4 depends on Y1.1 instead. I guess X is old and busted, and Y is the new hotness. What is the build list for A? Russ says: The rough build list for M is also just the list of all modules reachable in the requirement graph starting at M and following arrows. And: Simplify the rough build list to produce the final build list, by keeping only the newest version of any listed module. The list of all modules reachable from A is B1.2, C1.2, D1.3, D1.4, X1.1, and Y1.1, so that's the rough build list. The list of the newest versions of each module is B1.2, C1.2, D1.4, X1.1, and Y1.1, so that's the build list. The build list contains X1.1, even though it is not needed. Really? [1] https://research.swtch.com/vgo-mvs https://research.swtch.com/vgo-mvs [2] https://www.graphviz.org/doc/info/lang.html https://www.graphviz.org/doc/info/lang.html
  • StefanKarpinski 9y ago
    I believe that implicit in this scheme is that you cannot remove dependencies. It seems possible to add new dependencies, however. Changing dependencies can be seen as removing one and adding the other, but you can only do one, not the other. This limitation could probably be patched up by resolving a set of versions and then noticing that X1.1 is unreachable and discarding it. The trouble with that fix would be if X1.1 forces some other package to an unnecessarily high version even though it ends up being discarded.