16 ms·

How many iterations of this do we have to go through? Go hasn't had mature dependency management since its inception and this constant thrash is really starting

How many iterations of this do we have to go through? Go hasn't had mature dependency management since its inception and this constant thrash is really starting to make things difficult. For all practical purposes dependency management for programming languages is a solved problem, with many open source examples. The only explanation for this constant re-invention that I can come up with (and that's shared among others I know) is that Google doesn't care/need a dependency tool because of their mono repo. Which is rather frustrating since we get features (cough aliases cough) forced down our throats when Google suddenly finds a need for them.

schmichael 9y ago
> Go hasn't had mature dependency management since its inception and this constant thrash is really starting to make things difficult. What constant thrash? In the 6 years since Go 1.0 the only dependency management related change I'm aware of is the addition of `vendor/`. Update: sounds like "thrash" refers to the variety of community vendoring tools. I guess of the few I've used they all felt fairly similar/interchangeable, so it didn't feel like thrashing changes.
- notheguyouthink 9y ago
  There have been many packages for dependency management, which is not "official" thrash, but thrash nonetheless. Granted, mostly it's been thrash over the last few years. Especially bad since `go dep` was hailed as the king, and then recently abandoned entirely.
- TheDong 9y ago
  The blog post mentions some of this. It claims " For a long time, we believed that the problem of package versioning would be best solved by an add-on tool, and we encouraged people to create one. The Go community created many tools with different approaches" These many tools were the thrash, including godep, gb, glide, dep, and more. Each of these expressed versions with different manifests, many of them could import other format's, but not all. All-in-all it has been a mess. Sure, the go team has not officially done much, but that's because their stance has been borderline negligent on the topic.
- benesch 9y ago
  The constant thrash caused by the lack of a standard package management tool—even a de facto standard. Just look at the various package managers that have come and gone over the years: 1. https://github.com/tools/godep https://github.com/tools/godep 2. https://github.com/kardianos/govendor https://github.com/kardianos/govendor 3. https://github.com/robfig/glock https://github.com/robfig/glock 4. https://github.com/rogpeppe/govers https://github.com/rogpeppe/govers 5. https://github.com/Masterminds/glide https://github.com/Masterminds/glide 6. https://github.com/golang/dep https://github.com/golang/dep I'm sure I'm forgetting a few. And just when it seemed the Go team might be ready to standardize around Dep (6), they threw this wrench in the works. I'm of two minds about vgo. I think it has some interesting ideas, but Dep works today, is widely adopted, and has nearly all the features you'd expect from a modern package manager.
  - randomdata 9y ago
    > The constant thrash caused by the lack of a standard package management tool There is a standard, and has been one from the early days. The catch is that the standard is not loved by all, so others have created competing package management systems to fit their own needs.
    - kasey_junk 9y ago
      What is it?
      - lox 9y ago
        go get.
        kasey_junk 9y ago
        Does that finally work with private repos?
        sethammons 9y ago
        It always has. You have to edit your gitconfig. On mobile, so I can't paste mine. A Google search should get you there. It did for me like 5 years ago.
- lobster_johnson 9y ago
  Between "go get", Godep, Govendor, Glide, gb, Dep, and this — and I'm actually omitting a bunch of other, less popular utilities [1] — there's certainly been churn. There's definitely a feeling among Go developers that this problem should have been solved much earlier, and that the Go team's years-long refusal to address it caused the proliferation of mediocre tools (looking at you, Glide) that in many ways made the whole situation worse. When Dep came along, a lot of people breathed a sigh a of relief, because we finally have something okay, and we can go back to being productive instead of fighting dependency management problems all day. [1] https://github.com/golang/go/wiki/PackageManagementTools https://github.com/golang/go/wiki/PackageManagementTools
- majewsky 9y ago
  > In the 6 years since Go 1.0 the only dependency management related change I'm aware of is the addition of `vendor/`. This. Sure, new tools have appeared, but there's literally nothing wrong if you decide to just stick with godep or whatever you happen to be using.
merb 9y ago
+1 I actually also liked dep with the vendoring approach, it is just better to ci a repo with vendored dependencies (no more Broken Building because of Internet/Proxy or github Problems). And in Go it was/is Even simple to upgrade them. Sadly go is slowly moving away from that Part. the new approach is Bad, because they still pull Sources from Github, this Makes their approach unsuitable. E: damn iPhone autocorrect
- notheguyouthink 9y ago
  I thought they were adding support for vendoring? Was it abandoned again?
  - saghm 9y ago
    I think GP is referring to the fact that dep may not remain the blessed solution for long; recently, Russ Cox posted a number of articles suggesting looking into alternatives https://research.swtch.com/vgo https://research.swtch.com/vgo
- pdeuchler 9y ago
  Agreed. Dep had its warts (20 minutes for a very simple dep ensure...), but it solved the 80% use case in a way that was least objectionable for the majority. The fact that they can't (won't?) even refactor dep to suit their new ideas or try and revitalize the gps project just tells me that they don't really care about solving this for end users and just want to have fun writing a fancy dependency graph solver. Which is cool and all, but at least give me a stable API first.
  - nicpottier 9y ago
    > just want to have fun writing a fancy dependency graph solver There is no graph solver, that's the whole point.
    - kibwen 9y ago
      Other package managers, including the ones mentioned in the OP, don't have SAT solvers either. This isn't a new feature of dependency resolution algorithms.
GiorgioG 9y ago
Sometimes thrashing is good. Consider the pile of crap that is NuGet in the .NET world. Sure there's Paket, but it has nowhere near the adoption/support of the Microsoft blessed NuGet package manager. At least the Golang team can cherry pick the best ideas & lessons-learned from these community-driven package managers.
- mattherman 9y ago
  What are some of your specific complaints about NuGet? I've never disliked it that much. For the most part it has "just worked" for me.
  - GiorgioG 9y ago
    In large projects, NuGet takes FOREVER to resolve package dependencies - I mean 4+ minutes on a new i7 Thinkpad w/16gb RAM/1TB ssd. Heaven-forbid you want to upgrade a package that exists in all 130 projects in the solution (not my call to have that many projects) - you may as well take a long lunch. I will try to make that the last task of the day so I can let it run for as long as it needs to. The VS UI for NuGet is terribly buggy.
    - pjmlp 9y ago
      Uauu! 4+ minutes. Try to build a C++ or Android project and then go for lunch instead. NuGET is great.
      - GiorgioG 9y ago
        That's 4 minutes to resolve one NuGet package's dependencies - not 4 minutes for a build.
        pjmlp 9y ago
        Really? I never had it take that long.
elliotlarson 9y ago
No kidding. I'm so impressed with most of Go, but this is just silly. I feel like Rust started out with solid dependency management in place, at least it's been there from the early days. I work a bit with Node and Ruby too. Both have solid dependency management in place. I mean there are warts, but generally the problem is solved. Actually, now that I think about it, Yahuda Katz has had a hand in working out dependency management for all of those technologies, Rust, Ruby and Node. Maybe the Go guys should bring in Yahuda to help out. If nothing else, he could probably help them get out of the paralysis of analysis loop they seem to be in here.
- sitkack 9y ago
  Rust is good but not great. Being able to import two major versions of a lib into the same compilation unit (am I understanding this correctly?) is a huge win. Rust can have transitive deps that only differ by version number, but they can't come in contact with each other (don't cross the versions). Reread the post, the authors make some great, mature realizations that I hope other languages will listen to. I don't use the language, but I use languages that Go has affected and this is a great thing.
  - msie 9y ago
    > import two major versions of a lib into the same compilation unit When does this situation come up? What is a compilation unit in this case?
    - eat_veggies 9y ago
      If the newer version of some library has some breaking changes, you can migrate bit by bit instead of doing it all in one go.
      - smaddox 9y ago
        If this is the only motivation for multiple dependency versions in the same compilation unit (crate), I'm not convinced. You would be trading off the simplicity of each crate specifying a range of acceptable dependency versions for specifying `N` ranges of acceptable dependencies, and requiring one of each. Much better to enforce one version per compilation unit, and if you want to get complex with your dependencies, then break your project into multiple compilation units.
mseepgood 9y ago
> For all practical purposes dependency management for programming languages is a solved problem It's obviously not solved satisfactory.
- pcwalton 9y ago
  Yes, it is. Everything can be improved, but I see very few complaints with Cargo, and, most importantly, not complaints that would be solved with minimal version selection. (Most complaints I see are about the lack of package namespacing.)
  - crtc 9y ago
    > I see very few complaints If you live in a filter bubble (survivorship bias).
  - weberc2 9y ago
    I agree that Cargo is awesome and I would be happy to see Go follow suit, but I interpret "package management is solved" to mean "the industry has standardized on a single scheme". There does seem to be convergence in this space, but it's a relatively new phenomenon.
    - sagichmal 9y ago
      Yes, the industry has absolutely standardized on a single scheme: manifest plus lock as separate declarations, SAT solver, single dependency version per compilation unit.
      - weberc2 9y ago
        If that's the case, it's a pretty recent development. Python and JS didn't have this until very recently. Yeah, Go is a bit late to the party, but hardly worth the dramatic criticism in this thread.
      - clhodapp 9y ago
        That may be what things are converging on but in my opinion, any system that imposes a "single dependency version per compilation unit" policy is not a "real" dependency management solution. That approach creates a tremendous amount of friction because it pretty much ensures dependency hell if there are transitive dependencies with separate release cycles (and there very much should be able to be). Edit: I just realized that we may have a different definition of "compilation unit". If you mean "single program" then my point stands. However, if you just meant some sectioned-off part of the program, then I think the drawbacks of that are far less. Basically, I think that it is useful to be able to have access to multiple versions of a datatype (e.g. if you want to write a converter from old -> new) but that use case is far more fringe.
weberc2 9y ago
I agree that the thrashing is a nuisance, but the 'reinvention' has nothing to do with Google except that Go didn't ship with a package manager by default because it wasn't useful to Google for the monorepo reason you cited. Regarding "features forced down our throats", I think that has only happened with aliases, and the community pushed back to the effect that the feature addition was halted until the community had time to properly review it. Mostly I think that was a one-off problem. As for "it's a solved problem", is it? Until perhaps the last year, pip, npm, and most other package managers have been plagued with serious issues. I'm not fond of the situation in Go, but I think it's not unreasonable to try something different and less binding than a package manager until a solution emerges which solves the problems faced by other package managers.
0xdeadbeefbabe 9y ago
I'm ok for now putting everything in one GOPATH as if I were google. > How many iterations of this do we have to go through? Fewer iterations than Mr. Cox has gone through. Isn't it great?
geodel 9y ago
> forced down our throats when Google suddenly finds a need for them You can say that about Go itself. And there are already other/better alternatives. So why use Go?
- dstroot 9y ago
  A language is valued by its programmer bench, tooling, available libraries, as well as the language itself. Honest question - what is better than go right now?
  - jerf 9y ago
    You'd have to specify the use case of interest. I don't think there's anything that uniformly dominates Go, but that's terribly interesting since there are so many dimensions of interest for a programming language that there aren't any languages in use that uniformly dominate some other language in use. (You can get uniform domination if you include unused language or esolangs, but who cares.) But there are certainly many use cases for which Go is not the best choice, not in the top 3-5, or straight-up not a viable choice.
  - baq 9y ago
    C, c++, c#, Java, Python, Lua, lisp, Ada... Go has its good sides, but it's just a tool. It's not the best, because tools aren't supposed to be the best. Tools are supposed to be useful for what they're made for.
    - ilovecars2 9y ago
      I have a hard time with using Java to write micro-services. Even though lots of packages, such as Undertow, support non blocking I/O, a lot of the rest of the stack and the programmers that use it don’t know how to do NIO. What happens then is that the SRE/Ops team needs to configure large thread pools which consume lots of RAM just because multiplexing is I/O so difficult. In comparison I can just start goroutines and Elixir processes for fast and simple I/O parallelism almost without thinking about it.
_ph_ 9y ago
Actually I think the process shows a strength of the Go community. The initial lack of a dependency management tool certainly had to do with the practises at Google - as Google employees, the processes of Google certainly shaped the needs of the Go creators. But I am very happy, that they did not pick a random management system, just because they "had to have one". We can see enough bad examples in the wild. They left this aspect out of the language standard, to tackle the challenge later, with proper consideration. As far as I understand all details, Russ Cox has presented a very thoroughly worked out spec, and it seems to solve a lot of problems, other dependency management tools have. It is also completely compatible with Go so far and also fully optional.
- pdeuchler 9y ago
  >> But I am very happy, that they did not pick a random management system, just because they "had to have one" But they have, multiple times? First it was just giving people links to godep when people complained in github issues. Then it was a tacit "we like this" about gps and glide. Then it was a pretty-much-official-blessing of dep. Now it's a super-official-now-we-mean-it implementation of vgo. I recognize this as a pattern because I do it all the time, software engineers constantly re-write things from the ground up when they don't care about actually solving the problem and just want to play around with cool new ideas. As far as the vgo spec, I'm not against it per se... I like most of the general conclusions, but none of it is new. We've had these ideas in the Go world for a while. Most of them have even been implemented! Why can't we just slowly transition an existing tool? Or maybe implement a common library that individual tools can use to solve the problem in their own way? Oh wait we already did that and it got abandoned. Until we see a tool with a stable API that solves the 80% use case and lasts for over a year I'm not going to get excited about the new flashy thing.
  - ko82jeipp 9y ago
    Welcome to IT as a career. Having started in the 90s, I have been through these same conversations regarding Adobe, Oracle (oh lawd Oracle) MS, Dell, Apple to a lesser extent. “Vendor the industry currently relies on heavily is changing things or yanking the rug again?” The free market works by big players yanking every one else around. Guh.
Thaxll 9y ago
"Go hasn't had mature dependency management since its inception" go dep is perfectly fine for most use case.
- endymi0n 9y ago
  Agreed, but compared to Go itself, dep is incredibly recent, still in Alpha, not widely used yet and only the last in a series of incredibly many homegrown approaches that created lots of chaos and churn. And when everybody thought the rollercoaster was finally over, vgo came around the corner. It’s a great proposal, no doubt, but it‘s fair to say dependency management and generics were the two largest unsolved daily encountered problems with Go in real life. Let‘s just hope the community gets an Epiphany about the latter one as well soon...
zeveb 9y ago
> The only explanation for this constant re-invention that I can come up with (and that's shared among others I know) is that Google doesn't care/need a dependency tool because of their mono repo. That's a pretty good argument for using a monorepo, don't you think? A single organisation's code should all be in a single repo, which means that a single version of a dependency is used within the organisation, and that updating to a new version of that dependency is a single, self-contained project.
- ilovecars2 9y ago
  We use a monorepo at our place of work and it’s great, so far. We also use govendor to manage external dependencies in the monorepo. We are rigorous about keeping packages up to date so have never had to run v1 and v2 of a package at the same time.
bobbyi_settv 9y ago
It's far from a solved problem. Many ecosystems haven't gotten past the "single dependency file and lock file" system where an entire project has to use a single version of each dependency. There are people splitting their project into a thousand "microservices" and making them interoperate over HTTP instead of regular function calls so that each part of their project can have its own dependencies.
- rickycook 9y ago
  also so that a crash doesn’t stop everything like a BSOD, also so that you can use the language most appropriate for the task at hand because assuming a single language is good for everything is just wrong, also to be able to incrementally migrate things to new PAAS/build systems/etc in the future so you don’t do big bang... and plenty more multiple library versions is one of the lowest priorities when doing micro services id say this is not to say that people don’t significantly over use micro services, but there are plenty of hard problems they solve
- JepZ 9y ago
  Yes, its kinda funny, I mean package managers are probably the ones with the most solid experience in dependency management and when I look at them I can see many different concepts at work there. Some like for example pacman are bold and are not afraid to break a system during upgrade (as it is very unlikely that the upgrade will fail) and others put an abnormal amount of effort to ensure an always functional system at any time (especially on source based distributions, e.g. paludis). And while I have been using development snapshots for years with my system and didn't have more bugs to cope with compared to other systems, I can understand that developers in corporate contexts need to be able to specify dependencies with versions. So I appreciate that proposal very much and hope it will lead to a better solution for every body involved by providing a way to use versioned dependencies without introducing too much overhead for those who don't need versioned dependencies.
  - flukus 9y ago
    Pacman, apt-get, et al also have something most language specific package managers don't, maintainers. Maintainers back port security fixes and are careful not to break binary compatibility. A random $lang specific package is unlikely to have either, if you expect a bug fix you have to upgrade to the latest version. Developers that don't want to provide this sort of stability typically hate distro package managers. No package manager can fix a cultural problem.
sethammons 9y ago
> For all practical purposes dependency management for programming languages is a solved problem Hey Phil! It's been a long time. C'mon man, when we worked together, are you telling me you never ran into the never-resolving dependency map with berkshelf and chef? I know I did many times. Locks and a manifest did not solve that. It sounds like the proposed solution should prevent that.
ngrilly 9y ago
Have you read the post? It provides answers to your claim that "dependency management is a solved problem".
nvarsj 9y ago
> How many iterations of this do we have to go through? Many, I think, based on what the author of dep says (https://sdboyer.io/blog/vgo-and-dep/ https://sdboyer.io/blog/vgo-and-dep/). I don't get why rsc would just throw away the code and work that was the culmination of years of community work. Isn't rsc largely responsible for creating this problem in the first place? I'm doubtful he's going to fix the dependency problem from scratch. There's a certain pigheadedness to a lot of golang decisions, and I'm afraid this is another one of them...
- shabbyrobe 9y ago
  I definitely see this with Go decisions too, but I think that same "pigheadedness" has also been a driving factor behind a lot of what makes Go so effective to work with. They've thrown out an awful lot of stuff, which sometimes makes me bristle but I usually have to admit they were right. I think they threw out a bit too much, but that's only a problem if the gaps don't get filled or the compromises aren't adequately explained. What's left is a very pragmatic selection of extremely useful tools that can be used to cut a tonne of working code very, very quickly. I can't say I always like it, but I benefit tremendously from it.
  - nvarsj 9y ago
    I really like Go overall, especially for systems work. But there's various quirks that hurt my day to day productivity, and I think they only exist due to stubbornness. For example, why do unused variables and imports, a style issue, cause a compiler error? This is a constant pain if commenting out lines of code during debugging or prototyping. On the other hand, the compiler doesn't care if I ignore or forget to check error values - which is almost always incorrect behavior.
    - shabbyrobe 9y ago
      Yeah for sure, it's definitely got its warts. I suspect the import thing was far easier to code as an error than to compute whether they need to be pruned. Gotta love those go build times though! With vim-go and goimports, it's handled automatically for me on save so I don't find that an issue any more. Unused variables, on the other hand...
mjevans 9y ago
What are the "benefits" of this approach? To re-iterate a different post of mine against this question (implicitly, what are the benefits of NOT having library versions): * The only version to test against is HEAD * No fossilization of security or stability issues * Public libraries must support all Uppercased (exposed) declarations; that is the library interface. * If the build breaks in an odd way: update go, then: go get -u all * Simplicity, there's only one supported version and only one version to go get and develop against. * Also, why would building against an old version _ever_ be necessary?
saturn_vk 9y ago
But it has had mature dependency management tools for a long time. They were similar to cargo and npm.

schmichael 9y ago

> Go hasn't had mature dependency management since its inception and this constant thrash is really starting to make things difficult. What constant thrash? In the 6 years since Go 1.0 the only dependency management related change I'm aware of is the addition of `vendor/`. Update: sounds like "thrash" refers to the variety of community vendoring tools. I guess of the few I've used they all felt fairly similar/interchangeable, so it didn't feel like thrashing changes.

notheguyouthink 9y ago
There have been many packages for dependency management, which is not "official" thrash, but thrash nonetheless. Granted, mostly it's been thrash over the last few years. Especially bad since `go dep` was hailed as the king, and then recently abandoned entirely.
TheDong 9y ago
The blog post mentions some of this. It claims " For a long time, we believed that the problem of package versioning would be best solved by an add-on tool, and we encouraged people to create one. The Go community created many tools with different approaches" These many tools were the thrash, including godep, gb, glide, dep, and more. Each of these expressed versions with different manifests, many of them could import other format's, but not all. All-in-all it has been a mess. Sure, the go team has not officially done much, but that's because their stance has been borderline negligent on the topic.
benesch 9y ago
The constant thrash caused by the lack of a standard package management tool—even a de facto standard. Just look at the various package managers that have come and gone over the years: 1. https://github.com/tools/godep https://github.com/tools/godep 2. https://github.com/kardianos/govendor https://github.com/kardianos/govendor 3. https://github.com/robfig/glock https://github.com/robfig/glock 4. https://github.com/rogpeppe/govers https://github.com/rogpeppe/govers 5. https://github.com/Masterminds/glide https://github.com/Masterminds/glide 6. https://github.com/golang/dep https://github.com/golang/dep I'm sure I'm forgetting a few. And just when it seemed the Go team might be ready to standardize around Dep (6), they threw this wrench in the works. I'm of two minds about vgo. I think it has some interesting ideas, but Dep works today, is widely adopted, and has nearly all the features you'd expect from a modern package manager.
- randomdata 9y ago
  > The constant thrash caused by the lack of a standard package management tool There is a standard, and has been one from the early days. The catch is that the standard is not loved by all, so others have created competing package management systems to fit their own needs.
  - kasey_junk 9y ago
    What is it?
    - lox 9y ago
      go get.
      - kasey_junk 9y ago
        Does that finally work with private repos?
        sethammons 9y ago
        It always has. You have to edit your gitconfig. On mobile, so I can't paste mine. A Google search should get you there. It did for me like 5 years ago.
lobster_johnson 9y ago
Between "go get", Godep, Govendor, Glide, gb, Dep, and this — and I'm actually omitting a bunch of other, less popular utilities [1] — there's certainly been churn. There's definitely a feeling among Go developers that this problem should have been solved much earlier, and that the Go team's years-long refusal to address it caused the proliferation of mediocre tools (looking at you, Glide) that in many ways made the whole situation worse. When Dep came along, a lot of people breathed a sigh a of relief, because we finally have something okay, and we can go back to being productive instead of fighting dependency management problems all day. [1] https://github.com/golang/go/wiki/PackageManagementTools https://github.com/golang/go/wiki/PackageManagementTools
majewsky 9y ago
> In the 6 years since Go 1.0 the only dependency management related change I'm aware of is the addition of `vendor/`. This. Sure, new tools have appeared, but there's literally nothing wrong if you decide to just stick with godep or whatever you happen to be using.

merb 9y ago

+1 I actually also liked dep with the vendoring approach, it is just better to ci a repo with vendored dependencies (no more Broken Building because of Internet/Proxy or github Problems). And in Go it was/is Even simple to upgrade them. Sadly go is slowly moving away from that Part. the new approach is Bad, because they still pull Sources from Github, this Makes their approach unsuitable. E: damn iPhone autocorrect

notheguyouthink 9y ago
I thought they were adding support for vendoring? Was it abandoned again?
- saghm 9y ago
  I think GP is referring to the fact that dep may not remain the blessed solution for long; recently, Russ Cox posted a number of articles suggesting looking into alternatives https://research.swtch.com/vgo https://research.swtch.com/vgo
pdeuchler 9y ago
Agreed. Dep had its warts (20 minutes for a very simple dep ensure...), but it solved the 80% use case in a way that was least objectionable for the majority. The fact that they can't (won't?) even refactor dep to suit their new ideas or try and revitalize the gps project just tells me that they don't really care about solving this for end users and just want to have fun writing a fancy dependency graph solver. Which is cool and all, but at least give me a stable API first.
- nicpottier 9y ago
  > just want to have fun writing a fancy dependency graph solver There is no graph solver, that's the whole point.
  - kibwen 9y ago
    Other package managers, including the ones mentioned in the OP, don't have SAT solvers either. This isn't a new feature of dependency resolution algorithms.

GiorgioG 9y ago

Sometimes thrashing is good. Consider the pile of crap that is NuGet in the .NET world. Sure there's Paket, but it has nowhere near the adoption/support of the Microsoft blessed NuGet package manager. At least the Golang team can cherry pick the best ideas & lessons-learned from these community-driven package managers.

mattherman 9y ago
What are some of your specific complaints about NuGet? I've never disliked it that much. For the most part it has "just worked" for me.
- GiorgioG 9y ago
  In large projects, NuGet takes FOREVER to resolve package dependencies - I mean 4+ minutes on a new i7 Thinkpad w/16gb RAM/1TB ssd. Heaven-forbid you want to upgrade a package that exists in all 130 projects in the solution (not my call to have that many projects) - you may as well take a long lunch. I will try to make that the last task of the day so I can let it run for as long as it needs to. The VS UI for NuGet is terribly buggy.
  - pjmlp 9y ago
    Uauu! 4+ minutes. Try to build a C++ or Android project and then go for lunch instead. NuGET is great.
    - GiorgioG 9y ago
      That's 4 minutes to resolve one NuGet package's dependencies - not 4 minutes for a build.
      - pjmlp 9y ago
        Really? I never had it take that long.

elliotlarson 9y ago

No kidding. I'm so impressed with most of Go, but this is just silly. I feel like Rust started out with solid dependency management in place, at least it's been there from the early days. I work a bit with Node and Ruby too. Both have solid dependency management in place. I mean there are warts, but generally the problem is solved. Actually, now that I think about it, Yahuda Katz has had a hand in working out dependency management for all of those technologies, Rust, Ruby and Node. Maybe the Go guys should bring in Yahuda to help out. If nothing else, he could probably help them get out of the paralysis of analysis loop they seem to be in here.

sitkack 9y ago
Rust is good but not great. Being able to import two major versions of a lib into the same compilation unit (am I understanding this correctly?) is a huge win. Rust can have transitive deps that only differ by version number, but they can't come in contact with each other (don't cross the versions). Reread the post, the authors make some great, mature realizations that I hope other languages will listen to. I don't use the language, but I use languages that Go has affected and this is a great thing.
- msie 9y ago
  > import two major versions of a lib into the same compilation unit When does this situation come up? What is a compilation unit in this case?
  - eat_veggies 9y ago
    If the newer version of some library has some breaking changes, you can migrate bit by bit instead of doing it all in one go.
    - smaddox 9y ago
      If this is the only motivation for multiple dependency versions in the same compilation unit (crate), I'm not convinced. You would be trading off the simplicity of each crate specifying a range of acceptable dependency versions for specifying `N` ranges of acceptable dependencies, and requiring one of each. Much better to enforce one version per compilation unit, and if you want to get complex with your dependencies, then break your project into multiple compilation units.

mseepgood 9y ago

> For all practical purposes dependency management for programming languages is a solved problem It's obviously not solved satisfactory.

pcwalton 9y ago
Yes, it is. Everything can be improved, but I see very few complaints with Cargo, and, most importantly, not complaints that would be solved with minimal version selection. (Most complaints I see are about the lack of package namespacing.)
- crtc 9y ago
  > I see very few complaints If you live in a filter bubble (survivorship bias).
- weberc2 9y ago
  I agree that Cargo is awesome and I would be happy to see Go follow suit, but I interpret "package management is solved" to mean "the industry has standardized on a single scheme". There does seem to be convergence in this space, but it's a relatively new phenomenon.
  - sagichmal 9y ago
    Yes, the industry has absolutely standardized on a single scheme: manifest plus lock as separate declarations, SAT solver, single dependency version per compilation unit.
    - weberc2 9y ago
      If that's the case, it's a pretty recent development. Python and JS didn't have this until very recently. Yeah, Go is a bit late to the party, but hardly worth the dramatic criticism in this thread.
    - clhodapp 9y ago
      That may be what things are converging on but in my opinion, any system that imposes a "single dependency version per compilation unit" policy is not a "real" dependency management solution. That approach creates a tremendous amount of friction because it pretty much ensures dependency hell if there are transitive dependencies with separate release cycles (and there very much should be able to be). Edit: I just realized that we may have a different definition of "compilation unit". If you mean "single program" then my point stands. However, if you just meant some sectioned-off part of the program, then I think the drawbacks of that are far less. Basically, I think that it is useful to be able to have access to multiple versions of a datatype (e.g. if you want to write a converter from old -> new) but that use case is far more fringe.

weberc2 9y ago

I agree that the thrashing is a nuisance, but the 'reinvention' has nothing to do with Google except that Go didn't ship with a package manager by default because it wasn't useful to Google for the monorepo reason you cited. Regarding "features forced down our throats", I think that has only happened with aliases, and the community pushed back to the effect that the feature addition was halted until the community had time to properly review it. Mostly I think that was a one-off problem. As for "it's a solved problem", is it? Until perhaps the last year, pip, npm, and most other package managers have been plagued with serious issues. I'm not fond of the situation in Go, but I think it's not unreasonable to try something different and less binding than a package manager until a solution emerges which solves the problems faced by other package managers.

0xdeadbeefbabe 9y ago

I'm ok for now putting everything in one GOPATH as if I were google. > How many iterations of this do we have to go through? Fewer iterations than Mr. Cox has gone through. Isn't it great?

geodel 9y ago

> forced down our throats when Google suddenly finds a need for them You can say that about Go itself. And there are already other/better alternatives. So why use Go?

dstroot 9y ago
A language is valued by its programmer bench, tooling, available libraries, as well as the language itself. Honest question - what is better than go right now?
- jerf 9y ago
  You'd have to specify the use case of interest. I don't think there's anything that uniformly dominates Go, but that's terribly interesting since there are so many dimensions of interest for a programming language that there aren't any languages in use that uniformly dominate some other language in use. (You can get uniform domination if you include unused language or esolangs, but who cares.) But there are certainly many use cases for which Go is not the best choice, not in the top 3-5, or straight-up not a viable choice.
- baq 9y ago
  C, c++, c#, Java, Python, Lua, lisp, Ada... Go has its good sides, but it's just a tool. It's not the best, because tools aren't supposed to be the best. Tools are supposed to be useful for what they're made for.
  - ilovecars2 9y ago
    I have a hard time with using Java to write micro-services. Even though lots of packages, such as Undertow, support non blocking I/O, a lot of the rest of the stack and the programmers that use it don’t know how to do NIO. What happens then is that the SRE/Ops team needs to configure large thread pools which consume lots of RAM just because multiplexing is I/O so difficult. In comparison I can just start goroutines and Elixir processes for fast and simple I/O parallelism almost without thinking about it.

_ph_ 9y ago

Actually I think the process shows a strength of the Go community. The initial lack of a dependency management tool certainly had to do with the practises at Google - as Google employees, the processes of Google certainly shaped the needs of the Go creators. But I am very happy, that they did not pick a random management system, just because they "had to have one". We can see enough bad examples in the wild. They left this aspect out of the language standard, to tackle the challenge later, with proper consideration. As far as I understand all details, Russ Cox has presented a very thoroughly worked out spec, and it seems to solve a lot of problems, other dependency management tools have. It is also completely compatible with Go so far and also fully optional.

pdeuchler 9y ago
>> But I am very happy, that they did not pick a random management system, just because they "had to have one" But they have, multiple times? First it was just giving people links to godep when people complained in github issues. Then it was a tacit "we like this" about gps and glide. Then it was a pretty-much-official-blessing of dep. Now it's a super-official-now-we-mean-it implementation of vgo. I recognize this as a pattern because I do it all the time, software engineers constantly re-write things from the ground up when they don't care about actually solving the problem and just want to play around with cool new ideas. As far as the vgo spec, I'm not against it per se... I like most of the general conclusions, but none of it is new. We've had these ideas in the Go world for a while. Most of them have even been implemented! Why can't we just slowly transition an existing tool? Or maybe implement a common library that individual tools can use to solve the problem in their own way? Oh wait we already did that and it got abandoned. Until we see a tool with a stable API that solves the 80% use case and lasts for over a year I'm not going to get excited about the new flashy thing.
- ko82jeipp 9y ago
  Welcome to IT as a career. Having started in the 90s, I have been through these same conversations regarding Adobe, Oracle (oh lawd Oracle) MS, Dell, Apple to a lesser extent. “Vendor the industry currently relies on heavily is changing things or yanking the rug again?” The free market works by big players yanking every one else around. Guh.

Thaxll 9y ago

"Go hasn't had mature dependency management since its inception" go dep is perfectly fine for most use case.

endymi0n 9y ago
Agreed, but compared to Go itself, dep is incredibly recent, still in Alpha, not widely used yet and only the last in a series of incredibly many homegrown approaches that created lots of chaos and churn. And when everybody thought the rollercoaster was finally over, vgo came around the corner. It’s a great proposal, no doubt, but it‘s fair to say dependency management and generics were the two largest unsolved daily encountered problems with Go in real life. Let‘s just hope the community gets an Epiphany about the latter one as well soon...

zeveb 9y ago

> The only explanation for this constant re-invention that I can come up with (and that's shared among others I know) is that Google doesn't care/need a dependency tool because of their mono repo. That's a pretty good argument for using a monorepo, don't you think? A single organisation's code should all be in a single repo, which means that a single version of a dependency is used within the organisation, and that updating to a new version of that dependency is a single, self-contained project.

ilovecars2 9y ago
We use a monorepo at our place of work and it’s great, so far. We also use govendor to manage external dependencies in the monorepo. We are rigorous about keeping packages up to date so have never had to run v1 and v2 of a package at the same time.

bobbyi_settv 9y ago

It's far from a solved problem. Many ecosystems haven't gotten past the "single dependency file and lock file" system where an entire project has to use a single version of each dependency. There are people splitting their project into a thousand "microservices" and making them interoperate over HTTP instead of regular function calls so that each part of their project can have its own dependencies.

rickycook 9y ago
also so that a crash doesn’t stop everything like a BSOD, also so that you can use the language most appropriate for the task at hand because assuming a single language is good for everything is just wrong, also to be able to incrementally migrate things to new PAAS/build systems/etc in the future so you don’t do big bang... and plenty more multiple library versions is one of the lowest priorities when doing micro services id say this is not to say that people don’t significantly over use micro services, but there are plenty of hard problems they solve
JepZ 9y ago
Yes, its kinda funny, I mean package managers are probably the ones with the most solid experience in dependency management and when I look at them I can see many different concepts at work there. Some like for example pacman are bold and are not afraid to break a system during upgrade (as it is very unlikely that the upgrade will fail) and others put an abnormal amount of effort to ensure an always functional system at any time (especially on source based distributions, e.g. paludis). And while I have been using development snapshots for years with my system and didn't have more bugs to cope with compared to other systems, I can understand that developers in corporate contexts need to be able to specify dependencies with versions. So I appreciate that proposal very much and hope it will lead to a better solution for every body involved by providing a way to use versioned dependencies without introducing too much overhead for those who don't need versioned dependencies.
- flukus 9y ago
  Pacman, apt-get, et al also have something most language specific package managers don't, maintainers. Maintainers back port security fixes and are careful not to break binary compatibility. A random $lang specific package is unlikely to have either, if you expect a bug fix you have to upgrade to the latest version. Developers that don't want to provide this sort of stability typically hate distro package managers. No package manager can fix a cultural problem.

sethammons 9y ago

> For all practical purposes dependency management for programming languages is a solved problem Hey Phil! It's been a long time. C'mon man, when we worked together, are you telling me you never ran into the never-resolving dependency map with berkshelf and chef? I know I did many times. Locks and a manifest did not solve that. It sounds like the proposed solution should prevent that.

ngrilly 9y ago

Have you read the post? It provides answers to your claim that "dependency management is a solved problem".

nvarsj 9y ago

> How many iterations of this do we have to go through? Many, I think, based on what the author of dep says (https://sdboyer.io/blog/vgo-and-dep/ https://sdboyer.io/blog/vgo-and-dep/). I don't get why rsc would just throw away the code and work that was the culmination of years of community work. Isn't rsc largely responsible for creating this problem in the first place? I'm doubtful he's going to fix the dependency problem from scratch. There's a certain pigheadedness to a lot of golang decisions, and I'm afraid this is another one of them...

shabbyrobe 9y ago
I definitely see this with Go decisions too, but I think that same "pigheadedness" has also been a driving factor behind a lot of what makes Go so effective to work with. They've thrown out an awful lot of stuff, which sometimes makes me bristle but I usually have to admit they were right. I think they threw out a bit too much, but that's only a problem if the gaps don't get filled or the compromises aren't adequately explained. What's left is a very pragmatic selection of extremely useful tools that can be used to cut a tonne of working code very, very quickly. I can't say I always like it, but I benefit tremendously from it.
- nvarsj 9y ago
  I really like Go overall, especially for systems work. But there's various quirks that hurt my day to day productivity, and I think they only exist due to stubbornness. For example, why do unused variables and imports, a style issue, cause a compiler error? This is a constant pain if commenting out lines of code during debugging or prototyping. On the other hand, the compiler doesn't care if I ignore or forget to check error values - which is almost always incorrect behavior.
  - shabbyrobe 9y ago
    Yeah for sure, it's definitely got its warts. I suspect the import thing was far easier to code as an error than to compute whether they need to be pruned. Gotta love those go build times though! With vim-go and goimports, it's handled automatically for me on save so I don't find that an issue any more. Unused variables, on the other hand...

mjevans 9y ago

What are the "benefits" of this approach? To re-iterate a different post of mine against this question (implicitly, what are the benefits of NOT having library versions): * The only version to test against is HEAD * No fossilization of security or stability issues * Public libraries must support all Uppercased (exposed) declarations; that is the library interface. * If the build breaks in an odd way: update go, then: go get -u all * Simplicity, there's only one supported version and only one version to go get and develop against. * Also, why would building against an old version _ever_ be necessary?

saturn_vk 9y ago

But it has had mature dependency management tools for a long time. They were similar to cargo and npm.