4 ms·

For those who missed it, Sam Boyer (maintainer of dep) wrote a detailed post about why he thinks vgo (or rather Minimum Version Selection) is inadequate[0]. Th

by adwhit 8y ago

For those who missed it, Sam Boyer (maintainer of dep) wrote a detailed post about why he thinks vgo (or rather Minimum Version Selection) is inadequate[0]. The key argument is With dep, it’s usually easy to point to failures - they’re explicit, verbose (and, currently, often difficult to understand, and printed out at the end of a dep ensure run. The primary failure mode in vgo, however, is silent false positives - a vgo {get,test,run,build} command changes your dependency graph, and exits 0. Maybe everything’s fine, maybe it isn’t, but it’s incumbent upon you to take additional steps to understand that your build is broken. I haven't been following this debate very closely, and this post doesn't make it clear - have these points been addressed? Is there still room to maneuver, or is the design mostly settled now? I know the post states that any design flaws will be fixed, but it sounds very much like the more typical lock-file + solver solution has been definitively decided against. [0] https://sdboyer.io/vgo/failure-modes/ https://sdboyer.io/vgo/failure-modes/

tmpz22 8y ago
I just hope they figure out a solution which satisfies the community for the long run. Maybe they have already with vgo, but this is starting to feel vary Javascript-y the way people have been pushed from $OLD_PACKAGE_MANAGER -> dep -> vgo.
- jrs95 8y ago
  Had vgo not included MVS I think they likely would have. But there's a lot of disagreement over the design of vgo which could have been avoided had it just done what almost every other recent package manager has done. This is going to make it more painful than it ought to be, but ultimately since this will be the official solution I think it's likely that it will be adopted and that the churn will end since third party tools are no longer necessary.
  - stubish 8y ago
    Third party tools will still be necessary, to work around short comings. I imagine a tool to update all your imports to the latest release will be needed by the majority who don't need reproducible builds, so you get versions with known bugs fixed. MVS seems to be essentially pinning to the oldest possible version, and having that entrenched in your software is going to suck the big tech debt when you do need to update. Hope none of the known bugs are data loss or security related, and good luck monitoring each and every one of your explicit and transient dependencies for updates on a larger project. A tool to update your dependencies to the latest version when convenient or for automated CI tests seems required to me.
    - kaeshiwaza 8y ago
      Look at https://research.swtch.com/vgo-tour https://research.swtch.com/vgo-tour (Upgrading) vgo list -m -u will show you which newer releases of your dependencies. Then you can upgrade one dependencies vgo get xxx or all vgo get -u
      - stubish 8y ago
        Excellent, that seems to address my misgivings. Thanks.
- ljm 8y ago
  Quite the contrary: NPM's reputation in these circles is less than stellar and it's been incumbent in the JS ecosystem ever since Node became a thing. Well, with a few attempts at competition along the way before those working with the browser jumped on board, but nobody is pushing anybody to use anything but NPM and that attitude hasn't changed for years. This story isn't unique to Go: both Python and Ruby have had their own ordeals with dependency management, there was an article about CMake just the other day... Maybe Go's mistake is searching for the one-true dependency resolution system and deprecating everything along the way until something good-enough turns up. Maybe it'd be better that developers are encouraged to use dep while vgo is still in proposal stage so that there is an easy migration path from one standard to another, as opposed to immediately rendering obsolete. I don't really know, neither do I know why Javascript is the scapegoat for this kind of shit, it's practically a rite of passage for a language gaining increasing mind-share.
  - weberc2 8y ago
    > Maybe Go's mistake is searching for the one-true dependency resolution system and deprecating everything along the way until something good-enough turns up. Maybe it'd be better that developers are encouraged to use dep while vgo is still in proposal stage so that there is an easy migration path from one standard to another, as opposed to immediately rendering obsolete. The odd thing is I get the feeling that their search for the one-true system is causing them to repeat every other package manager's mistakes. I'm likely misinformed, but I get the feeling that they think they can do the same thing other package managers have considered or tried, but it will work for them somehow. I don't get the feeling that they seriously considered the criticism of VGo's approach. It smells like hubris. Also, there's Cargo, which is lauded by all, but the proposal doesn't seem to consider that perhaps the Cargo folks had a reason to make their dependency resolution scheme complicated. Again, this smacks of hubris. I'm happy to be persuaded otherwise, and it would really only take a link to a thread in which some VGo proponent thoughtfully addresses Sam's criticism and the "why not copy Cargo?" criticism (and no, the "Cargo's dep resolution scheme is overly complicated" rationale from the proposal doesn't constitute).
    - ljm 8y ago
TheSwordsman 8y ago
The concerns haven't been addressed. Sam's latest comment for context: - https://github.com/golang/go/issues/24301#issuecomment-392549295 https://github.com/golang/go/issues/24301#issuecomment-39254...
nicpottier 8y ago
They haven't been addressed, but then again they exist in `dep` as well and most other package managers, just in different forms. Russ Cox had a good talk about this which is worth a watch: https://www.youtube.com/watch?v=F8nrpe0XWRg&feature=youtu.be&t=25m34s https://www.youtube.com/watch?v=F8nrpe0XWRg&feature=youtu.be...
- Groxx 8y ago
  >They haven't been addressed, but then again they exist in `dep` as well and most other package managers, just in different forms. The whole premise of the blog post is that vgo has additional failure modes that other ones do not. Right at the top, the first item in the list of "what this post covers" is: >MVS has all the same failure modes, plus more, minus pathological SAT.
skybrian 8y ago
It's worth pointing out that when working in a monorepo, there are no version numbers and no package system to tell you that upstream broke you, or you broke something downstream. So how do you find bugs? You run tests. The same can be true if you're working with package management. Most bugs aren't found by the dependency tracking system anyway. Having good tests will tell you about incompatibilities that upstream didn't even know about. If you're not using exactly the same versions that the maintainer used, you need to run tests. Maybe people currently don't run tests often enough? But this suggests a different approach to software robustness than comparing version numbers.
- lobster_johnson 8y ago
  Testing and package management aren't mutually exclusive. If you're in a monorepo, all your code evolves in lockstep. A core tenet of versioned package management is to get reproducible builds that only need to break once you choose to evolve forward in time (i.e. upgrade). Historically, most languages (C, C++, pre-Maven Java) haven't had package management at all, and so dependencies have typically been managed by vendoring the code (or JAR files). JAR files worked okay, but vendoring incurs maintenance overhead that isn't acceptable in today's environment. git submodules are theoretically a solution, but also high-maintenance.
  - skybrian 8y ago
    Yes, you are right. And I believe vgo is supposed to guarantee reproducible builds just like other package systems. However, when you upgrade a dependency, it's still possible that you're using a particular combination of library versions that have never been tested before. Some incompatibilities can be prevented by looking at version constraints. But you're not left with no error detection if the package system fails to detect an incompatibility; in the end, what matters is that the code compiles and the tests pass.
    - oceanswave 8y ago
      This methodology places way too much emphasis on the breadth of the tests into a test centric view— say you had a dep that had an SSL vulnerability - most of the time you’re not going to be checking for this type of thing at the level of your app, and doing so - but you bet you need to ensure that you are using the version of the dep that has the vulnerability fixed

by adwhit 8y ago

tmpz22 8y ago

I just hope they figure out a solution which satisfies the community for the long run. Maybe they have already with vgo, but this is starting to feel vary Javascript-y the way people have been pushed from $OLD_PACKAGE_MANAGER -> dep -> vgo.

jrs95 8y ago
Had vgo not included MVS I think they likely would have. But there's a lot of disagreement over the design of vgo which could have been avoided had it just done what almost every other recent package manager has done. This is going to make it more painful than it ought to be, but ultimately since this will be the official solution I think it's likely that it will be adopted and that the churn will end since third party tools are no longer necessary.
- stubish 8y ago
  Third party tools will still be necessary, to work around short comings. I imagine a tool to update all your imports to the latest release will be needed by the majority who don't need reproducible builds, so you get versions with known bugs fixed. MVS seems to be essentially pinning to the oldest possible version, and having that entrenched in your software is going to suck the big tech debt when you do need to update. Hope none of the known bugs are data loss or security related, and good luck monitoring each and every one of your explicit and transient dependencies for updates on a larger project. A tool to update your dependencies to the latest version when convenient or for automated CI tests seems required to me.
  - kaeshiwaza 8y ago
    Look at https://research.swtch.com/vgo-tour https://research.swtch.com/vgo-tour (Upgrading) vgo list -m -u will show you which newer releases of your dependencies. Then you can upgrade one dependencies vgo get xxx or all vgo get -u
    - stubish 8y ago
      Excellent, that seems to address my misgivings. Thanks.
ljm 8y ago
Quite the contrary: NPM's reputation in these circles is less than stellar and it's been incumbent in the JS ecosystem ever since Node became a thing. Well, with a few attempts at competition along the way before those working with the browser jumped on board, but nobody is pushing anybody to use anything but NPM and that attitude hasn't changed for years. This story isn't unique to Go: both Python and Ruby have had their own ordeals with dependency management, there was an article about CMake just the other day... Maybe Go's mistake is searching for the one-true dependency resolution system and deprecating everything along the way until something good-enough turns up. Maybe it'd be better that developers are encouraged to use dep while vgo is still in proposal stage so that there is an easy migration path from one standard to another, as opposed to immediately rendering obsolete. I don't really know, neither do I know why Javascript is the scapegoat for this kind of shit, it's practically a rite of passage for a language gaining increasing mind-share.
- weberc2 8y ago
  > Maybe Go's mistake is searching for the one-true dependency resolution system and deprecating everything along the way until something good-enough turns up. Maybe it'd be better that developers are encouraged to use dep while vgo is still in proposal stage so that there is an easy migration path from one standard to another, as opposed to immediately rendering obsolete. The odd thing is I get the feeling that their search for the one-true system is causing them to repeat every other package manager's mistakes. I'm likely misinformed, but I get the feeling that they think they can do the same thing other package managers have considered or tried, but it will work for them somehow. I don't get the feeling that they seriously considered the criticism of VGo's approach. It smells like hubris. Also, there's Cargo, which is lauded by all, but the proposal doesn't seem to consider that perhaps the Cargo folks had a reason to make their dependency resolution scheme complicated. Again, this smacks of hubris. I'm happy to be persuaded otherwise, and it would really only take a link to a thread in which some VGo proponent thoughtfully addresses Sam's criticism and the "why not copy Cargo?" criticism (and no, the "Cargo's dep resolution scheme is overly complicated" rationale from the proposal doesn't constitute).
  - ljm 8y ago

TheSwordsman 8y ago

The concerns haven't been addressed. Sam's latest comment for context: - https://github.com/golang/go/issues/24301#issuecomment-392549295 https://github.com/golang/go/issues/24301#issuecomment-39254...

nicpottier 8y ago

They haven't been addressed, but then again they exist in `dep` as well and most other package managers, just in different forms. Russ Cox had a good talk about this which is worth a watch: https://www.youtube.com/watch?v=F8nrpe0XWRg&feature=youtu.be&t=25m34s https://www.youtube.com/watch?v=F8nrpe0XWRg&feature=youtu.be...

Groxx 8y ago
>They haven't been addressed, but then again they exist in `dep` as well and most other package managers, just in different forms. The whole premise of the blog post is that vgo has additional failure modes that other ones do not. Right at the top, the first item in the list of "what this post covers" is: >MVS has all the same failure modes, plus more, minus pathological SAT.

skybrian 8y ago

It's worth pointing out that when working in a monorepo, there are no version numbers and no package system to tell you that upstream broke you, or you broke something downstream. So how do you find bugs? You run tests. The same can be true if you're working with package management. Most bugs aren't found by the dependency tracking system anyway. Having good tests will tell you about incompatibilities that upstream didn't even know about. If you're not using exactly the same versions that the maintainer used, you need to run tests. Maybe people currently don't run tests often enough? But this suggests a different approach to software robustness than comparing version numbers.

lobster_johnson 8y ago
Testing and package management aren't mutually exclusive. If you're in a monorepo, all your code evolves in lockstep. A core tenet of versioned package management is to get reproducible builds that only need to break once you choose to evolve forward in time (i.e. upgrade). Historically, most languages (C, C++, pre-Maven Java) haven't had package management at all, and so dependencies have typically been managed by vendoring the code (or JAR files). JAR files worked okay, but vendoring incurs maintenance overhead that isn't acceptable in today's environment. git submodules are theoretically a solution, but also high-maintenance.
- skybrian 8y ago
  Yes, you are right. And I believe vgo is supposed to guarantee reproducible builds just like other package systems. However, when you upgrade a dependency, it's still possible that you're using a particular combination of library versions that have never been tested before. Some incompatibilities can be prevented by looking at version constraints. But you're not left with no error detection if the package system fails to detect an incompatibility; in the end, what matters is that the code compiles and the tests pass.
  - oceanswave 8y ago
    This methodology places way too much emphasis on the breadth of the tests into a test centric view— say you had a dep that had an SSL vulnerability - most of the time you’re not going to be checking for this type of thing at the level of your app, and doing so - but you bet you need to ensure that you are using the version of the dep that has the vulnerability fixed