7 ms·

Minimal Version Selection This is the piece that is very different from other dependency managers and is worth people looking at. Instead of trying to get the

by mfer 9y ago

Minimal Version Selection This is the piece that is very different from other dependency managers and is worth people looking at. Instead of trying to get the latest version it's going to try to get the oldest. If you want to update to newer version of transitive (dependencies of dependencies) that your dependencies have not updated for you'll need to start tracking those yourself. There's an issue touching on this at https://github.com/golang/go/issues/24500 https://github.com/golang/go/issues/24500 Other packages managers use the opposite of minimal version selection. Many of them even have a don't make me thing command to update the whole dependency tree (e.g., `npm update`). What do folks think of the implications of MVS, especially for transitive dependency management?

bishop_mandible 9y ago
> Other packages managers use the opposite of minimal version selection. Many of them even have a don't make me thing command to update the whole dependency tree (e.g., `npm update`). vgo does have that too: vgo get -u
- mfer 9y ago
  That will work for direct dependencies but not dependencies of dependencies. See https://github.com/golang/go/issues/24500 https://github.com/golang/go/issues/24500
  - nicpottier 9y ago
    Correct, it will get the newest version that is actually tested against your dependencies, which is by design. Why would you want to pull a version that is newer than the author of the library you depend on has actually tested with? You can of course force this in vgo, but having the default use the versions specified by the authors makes a whole lot more sense than just using the newest.
    - mfer 9y ago
      > Why would you want to pull a version that is newer than the author of the library you depend on has actually tested with? 1. To install a security update or bug fix update you need in a transitive dependency that the author of the dependency you're using hasn't updated to. 2. To use the same workflows across all my dependency management tools (npm, cargo, composer, bundler, and the rest of the lot follow the same patters and vgo goes against the patterns used by the others) There are two reasons.
      - stouset 9y ago
        3. To apply bugfixes that affect you, but weren't exposed by your direct dependency's tests.
      - nicpottier 9y ago
        You can do 1), you just have make that an explicit dependency of your own. It just won't grab newest by default for sub-dependencies. So yes, you can pull the latest. 2) seems like a bit of a silly reason, if we all wanted to make everything work the same all the time we wouldn't make much progress or try anything new. Whether vgo's approach is correct or not we don't know yet, but saying that it isn't familiar isn't a good enough reason to not try it out. To me, vgo matches what we already do in our python projects with a lot of dependencies. Pin everything in a freeze and upgrade on a schedule when we need to. We have seen far too many failures doing it any other way. (IE, using the "latest" of everything, which often either breaks semantic versioning and actually breaks or has subtle bugs that didn't exist before)
        Groxx 9y ago
        Using something like pip-compile, or literally `pip freeze > requirements.txt`?
        nicpottier 9y ago
        The latter yes.
    - deleted 9y ago
  - sseth 9y ago
    I believe that vgo get -u works for both direct and indirect dependencies. See the vgo tour : https://research.swtch.com/vgo-tour https://research.swtch.com/vgo-tour The indirect dependency (rsc.io/sampler - which is a dependency of rsc.io/quote), is also upgraded to the latest version v1.99.99 when vgo get -u is done.
stouset 9y ago
> What do folks think of the implications of MVS, especially for transitive dependency management? I can't think of more than a handful of times over the past twelve years (primarily Ruby and Rust) where a newer package broke an existing one, and the majority of those times the problem was a new major version that wasn't appropriately accounted for (e.g., the author should have depended upon '~> 2.x.x' instead of '>= 2.x.x'). I'd much rather have that problem addressed. Assume semver, and make the easiest way of specifying a dependency cap it to a major version. On the other hand, I regularly deal with the consequences of upgrading infrequently — when you do upgrade, it's a nightmare. As a rule of thumb, the more frequently you update your dependencies, the less net pain you experience. Upgrading once a week is relatively painless. Upgrading once a year (or even less frequently) is a nightmare due to the sheer volume of changes coming in at once. Also, having transitioned to the security side of things over the past few years, encouraging stale dependencies just means that security patches will never get incorporated. A stable project that releases mostly security fixes and few feature changes will — in practice — never see its minimum version bumped in projects that depend upon it. My prediction is that this will just result in security patches not being applied and the problem of upgrading dependencies will be made generally worse. I will be happy, though, if I am proven wrong.
- adrianmonk 9y ago
  According to my reading of it, there is an update-all operation. It has just been separated out in an attempt to give the user control over initiating it. But I agree that procrastinating on updates creates problems. (At some point, the entire point of version pinning is to allow you the choice to defer, but it's still a problem.) I think it might be worth exploring reporting as a solution. Everyone knows that old versions are a problem, but what if I had tools to tell me how bad or good the situation is for my project at this moment? And what if they ran by default either periodically or as part of my build or both? Examples of stuff it could tell me: * Am I one minor version behind on this one library and it doesn't matter? * Or am I a major version behind on this other library and I'm using code that isn't even supported or maintained? * Are there security fixes that I haven't taken? * Are there libraries that have security fixes but no release is available yet? * How about a list of libraries that I'm not on the latest minor version of AND the latest version has been available for more than 2 weeks? (Maybe I don't want to fall behind but I don't want to be a guinea pig either.) * Or a list of libraries that I'm not on the latest major version of and a newer major version has been available for 6 months? Since this is important, it would be great to have real visibility into it. Right now, every build I've ever done, this is just something that people track in their heads and just assume they have a good handle on. Doing regular releases and always taking the latest version of everything helps somewhat, but sometimes a release gets canceled. Or maybe there's a system that isn't being regularly worked on and doesn't have regular releases, yet dependencies are being updated, it is behind, but by how much?
deckar01 9y ago
> Instead of trying to get the latest version it's going to try to get the oldest. No, minimal refers to the number of dependency changes between upgrades, not the version numbers of the dependencies. Dependency conflicts initially resolve to the higher of the two versions. https://research.swtch.com/vgo-mvs https://research.swtch.com/vgo-mvs section " Algorithm 1: Construct Build List" In the example, a dependency is upgraded and no longer needs one of the shared transient dependencies. Instead of downgrading it to the version declared by the unchanged dependency, it keeps the higher version that you were already using. When upgrading, transient dependencies are never downgraded, only added (or removed). https://research.swtch.com/vgo-mvs https://research.swtch.com/vgo-mvs section " Algorithm 3. Upgrade One Module"
- mfer 9y ago
  Let's say you have the following picture... App --> Dependency A --> Dependency C (at version 1.2.3) --> Dependency B --> Dependency C (at version 1.3.4) The latest release of Dependency C is 1.4.5. Which version would be used after running `vgo get -u`? As you note it would use the newer version of those specified (version 1.3.4). The newer releases after those explicitly specified are not used.
  - sseth 9y ago
    I believe (having tried the tour) that you will get v1.4.5 at the App level, if you do vgo get -u. If you just do plain vgo get, you will get 1.3.4, because of the minimum version approach.
resonantjacket5 9y ago
Mvs upside sounds like that the dependencies of dependencies won't need change. However the downside is that how would you upgrade security patches into older packages? They would always fetch the older dependency and never get upgrades. There's a reason why npm install was made the way it was though it went too far in the other direction with always getting latest dependency.
- sseth 9y ago
  As Russ points out, the current approach of pulling the latest leads to a situation where a package managers can be quite sloppy. For example, a package may have a stated dependency A@1.1.2. But because everyone is pulling the latest, it may turn out that the package actually no longer works with that version of A anymore. The vgo approach will encourage package managers to update their dependency versions to what they have actually tested with, which is very useful information for consumers of the package.
IshKebab 9y ago
> Instead of trying to get the latest version it's going to try to get the oldest. Err, no. For each library it uses the newest version of the library that is used by any of its dependents (including transitive dependents). See https://research.swtch.com/vgo-mvs https://research.swtch.com/vgo-mvs Especially the line: Simplify the rough build list to produce the final build list, by keeping only the newest version of any listed module. Different major versions are considered different libraries essentially. To be honest I thought this was all obvious and I'm sure this approach has been recommended by Go people in the past. It has another nice advantage over Rust's system - you have have pre-releases of major versions. Rust doesn't really have a way to do that except for releases before version 1.0. In other words when you want to do some incompatible changes and release version 2 of your library there's no good way to put test versions of it on Crates.io.
- steveklabnik 9y ago
  > It has another nice advantage over Rust's system - you have have pre-releases of major versions. Rust doesn't really have a way to do that except for releases before version 1.0. Semver has a specific way to indicate that a feature is a pre-release, and people use it. The second post on /r/rust is advertising a pre-release of the next major version of rand: https://crates.io/crates/rand/0.5.0-pre.0 https://crates.io/crates/rand/0.5.0-pre.0 (Yes, this is before 1.0, but 1.5.0-pre.0 would work just fine too!)
  - IshKebab 9y ago
    Do you have a reference for that? It isn't mentioned here: https://doc.rust-lang.org/cargo/reference/manifest.html#the-version-field https://doc.rust-lang.org/cargo/reference/manifest.html#the-... I found this bug report and it seems like it has some issues: https://github.com/rust-lang/cargo/issues/2222 https://github.com/rust-lang/cargo/issues/2222
    - steveklabnik 9y ago
      Reference: I maintain the semver library. I’m on my phone, so I can’t link you to the docs right now. I’ll add a link tomorrow.

by mfer 9y ago

bishop_mandible 9y ago

> Other packages managers use the opposite of minimal version selection. Many of them even have a don't make me thing command to update the whole dependency tree (e.g., `npm update`). vgo does have that too: vgo get -u

mfer 9y ago
That will work for direct dependencies but not dependencies of dependencies. See https://github.com/golang/go/issues/24500 https://github.com/golang/go/issues/24500
- nicpottier 9y ago
  Correct, it will get the newest version that is actually tested against your dependencies, which is by design. Why would you want to pull a version that is newer than the author of the library you depend on has actually tested with? You can of course force this in vgo, but having the default use the versions specified by the authors makes a whole lot more sense than just using the newest.
  - mfer 9y ago
    > Why would you want to pull a version that is newer than the author of the library you depend on has actually tested with? 1. To install a security update or bug fix update you need in a transitive dependency that the author of the dependency you're using hasn't updated to. 2. To use the same workflows across all my dependency management tools (npm, cargo, composer, bundler, and the rest of the lot follow the same patters and vgo goes against the patterns used by the others) There are two reasons.
    - stouset 9y ago
      3. To apply bugfixes that affect you, but weren't exposed by your direct dependency's tests.
    - nicpottier 9y ago
      You can do 1), you just have make that an explicit dependency of your own. It just won't grab newest by default for sub-dependencies. So yes, you can pull the latest. 2) seems like a bit of a silly reason, if we all wanted to make everything work the same all the time we wouldn't make much progress or try anything new. Whether vgo's approach is correct or not we don't know yet, but saying that it isn't familiar isn't a good enough reason to not try it out. To me, vgo matches what we already do in our python projects with a lot of dependencies. Pin everything in a freeze and upgrade on a schedule when we need to. We have seen far too many failures doing it any other way. (IE, using the "latest" of everything, which often either breaks semantic versioning and actually breaks or has subtle bugs that didn't exist before)
      - Groxx 9y ago
        Using something like pip-compile, or literally `pip freeze > requirements.txt`?
        nicpottier 9y ago
        The latter yes.
  - deleted 9y ago
- sseth 9y ago
  I believe that vgo get -u works for both direct and indirect dependencies. See the vgo tour : https://research.swtch.com/vgo-tour https://research.swtch.com/vgo-tour The indirect dependency (rsc.io/sampler - which is a dependency of rsc.io/quote), is also upgraded to the latest version v1.99.99 when vgo get -u is done.

stouset 9y ago

> What do folks think of the implications of MVS, especially for transitive dependency management? I can't think of more than a handful of times over the past twelve years (primarily Ruby and Rust) where a newer package broke an existing one, and the majority of those times the problem was a new major version that wasn't appropriately accounted for (e.g., the author should have depended upon '~> 2.x.x' instead of '>= 2.x.x'). I'd much rather have that problem addressed. Assume semver, and make the easiest way of specifying a dependency cap it to a major version. On the other hand, I regularly deal with the consequences of upgrading infrequently — when you do upgrade, it's a nightmare. As a rule of thumb, the more frequently you update your dependencies, the less net pain you experience. Upgrading once a week is relatively painless. Upgrading once a year (or even less frequently) is a nightmare due to the sheer volume of changes coming in at once. Also, having transitioned to the security side of things over the past few years, encouraging stale dependencies just means that security patches will never get incorporated. A stable project that releases mostly security fixes and few feature changes will — in practice — never see its minimum version bumped in projects that depend upon it. My prediction is that this will just result in security patches not being applied and the problem of upgrading dependencies will be made generally worse. I will be happy, though, if I am proven wrong.

adrianmonk 9y ago
According to my reading of it, there is an update-all operation. It has just been separated out in an attempt to give the user control over initiating it. But I agree that procrastinating on updates creates problems. (At some point, the entire point of version pinning is to allow you the choice to defer, but it's still a problem.) I think it might be worth exploring reporting as a solution. Everyone knows that old versions are a problem, but what if I had tools to tell me how bad or good the situation is for my project at this moment? And what if they ran by default either periodically or as part of my build or both? Examples of stuff it could tell me: * Am I one minor version behind on this one library and it doesn't matter? * Or am I a major version behind on this other library and I'm using code that isn't even supported or maintained? * Are there security fixes that I haven't taken? * Are there libraries that have security fixes but no release is available yet? * How about a list of libraries that I'm not on the latest minor version of AND the latest version has been available for more than 2 weeks? (Maybe I don't want to fall behind but I don't want to be a guinea pig either.) * Or a list of libraries that I'm not on the latest major version of and a newer major version has been available for 6 months? Since this is important, it would be great to have real visibility into it. Right now, every build I've ever done, this is just something that people track in their heads and just assume they have a good handle on. Doing regular releases and always taking the latest version of everything helps somewhat, but sometimes a release gets canceled. Or maybe there's a system that isn't being regularly worked on and doesn't have regular releases, yet dependencies are being updated, it is behind, but by how much?

deckar01 9y ago

> Instead of trying to get the latest version it's going to try to get the oldest. No, minimal refers to the number of dependency changes between upgrades, not the version numbers of the dependencies. Dependency conflicts initially resolve to the higher of the two versions. https://research.swtch.com/vgo-mvs https://research.swtch.com/vgo-mvs section " Algorithm 1: Construct Build List" In the example, a dependency is upgraded and no longer needs one of the shared transient dependencies. Instead of downgrading it to the version declared by the unchanged dependency, it keeps the higher version that you were already using. When upgrading, transient dependencies are never downgraded, only added (or removed). https://research.swtch.com/vgo-mvs https://research.swtch.com/vgo-mvs section " Algorithm 3. Upgrade One Module"

mfer 9y ago
Let's say you have the following picture... App --> Dependency A --> Dependency C (at version 1.2.3) --> Dependency B --> Dependency C (at version 1.3.4) The latest release of Dependency C is 1.4.5. Which version would be used after running `vgo get -u`? As you note it would use the newer version of those specified (version 1.3.4). The newer releases after those explicitly specified are not used.
- sseth 9y ago
  I believe (having tried the tour) that you will get v1.4.5 at the App level, if you do vgo get -u. If you just do plain vgo get, you will get 1.3.4, because of the minimum version approach.

resonantjacket5 9y ago

Mvs upside sounds like that the dependencies of dependencies won't need change. However the downside is that how would you upgrade security patches into older packages? They would always fetch the older dependency and never get upgrades. There's a reason why npm install was made the way it was though it went too far in the other direction with always getting latest dependency.

sseth 9y ago
As Russ points out, the current approach of pulling the latest leads to a situation where a package managers can be quite sloppy. For example, a package may have a stated dependency A@1.1.2. But because everyone is pulling the latest, it may turn out that the package actually no longer works with that version of A anymore. The vgo approach will encourage package managers to update their dependency versions to what they have actually tested with, which is very useful information for consumers of the package.

IshKebab 9y ago

> Instead of trying to get the latest version it's going to try to get the oldest. Err, no. For each library it uses the newest version of the library that is used by any of its dependents (including transitive dependents). See https://research.swtch.com/vgo-mvs https://research.swtch.com/vgo-mvs Especially the line: Simplify the rough build list to produce the final build list, by keeping only the newest version of any listed module. Different major versions are considered different libraries essentially. To be honest I thought this was all obvious and I'm sure this approach has been recommended by Go people in the past. It has another nice advantage over Rust's system - you have have pre-releases of major versions. Rust doesn't really have a way to do that except for releases before version 1.0. In other words when you want to do some incompatible changes and release version 2 of your library there's no good way to put test versions of it on Crates.io.

steveklabnik 9y ago
> It has another nice advantage over Rust's system - you have have pre-releases of major versions. Rust doesn't really have a way to do that except for releases before version 1.0. Semver has a specific way to indicate that a feature is a pre-release, and people use it. The second post on /r/rust is advertising a pre-release of the next major version of rand: https://crates.io/crates/rand/0.5.0-pre.0 https://crates.io/crates/rand/0.5.0-pre.0 (Yes, this is before 1.0, but 1.5.0-pre.0 would work just fine too!)
- IshKebab 9y ago
  Do you have a reference for that? It isn't mentioned here: https://doc.rust-lang.org/cargo/reference/manifest.html#the-version-field https://doc.rust-lang.org/cargo/reference/manifest.html#the-... I found this bug report and it seems like it has some issues: https://github.com/rust-lang/cargo/issues/2222 https://github.com/rust-lang/cargo/issues/2222
  - steveklabnik 9y ago
    Reference: I maintain the semver library. I’m on my phone, so I can’t link you to the docs right now. I’ll add a link tomorrow.