7 ms·

Here's a problem MSV guards against: a new checkin in a dependency introduces a massive security hole and it silently gets pulled into your project. The "secur

by kjksf 8y ago

Here's a problem MSV guards against: a new checkin in a dependency introduces a massive security hole and it silently gets pulled into your project. The "security issue" argument is bogus because while new code can fix bugs it can also introduce new bugs. Every competent dev shop has code reviews. A code review exists because you don't trust your coworkers to not introduce bugs. But somehow you're ok with people you don't know to effectively check in code into your code base without as much as running a smoke test.

Jyaif 8y ago
There's a difference between an old version where everybody know about the vulnerabilities because they were patched, and a new version where pretty much nobody knows any vulnerability (otherwise they would have been patched).
- jessaustin 8y ago
  Everyone who knows about the old vulns can avoid them easily with minimum version, just by setting their dependency versions properly. The new unknown vulns about which no one knows are avoided automatically. Re-reading the thread, maybe this is what you meant? I'm not sure, possibly because this is also what 'kjksf meant.
  - jacques_chester 8y ago
    The arguments are directed past each other. Minimal selection breaks down when you ignore it and thereby allow known bugs to pile up. Maximal selection breaks down when you ignore it and thereby allow unknown, untracked dependencies to pile up. The answer for both is that version updating should trigger CI activity and then be automatically applied when tested. Dependency tools should be as strictly dumb as possible. This gap exists because dependency management tools blossomed and matured in a manual developer-at-a-keyboard context before the kind of automation that makes their clever bits less relevant or necessary.
    - weberc2 8y ago
      Vulnerability scanning favors old versions. If you have this, you’re probably in a (considerably?) safer place with MVS than max selection.
      - jacques_chester 8y ago
        > Vulnerability scanning favors old versions. I don't follow. Could you elaborate?
        weberc2 8y ago
        Vuln scanning depends on vulns having been discovered. These vulns are more likely to be discovered in old versions than new versions.
        jacques_chester 8y ago
        I agree. But I think we're disagreeing on what policy follows from that.
jacques_chester 8y ago
> The "security issue" argument is bogus because while new code can fix bugs it can also introduce new bugs. Old versions only ever increase their known-bug count. Only a new version can reduce a known-bug count. It follows that the longer you stick to a version, the more problems you are guaranteed to be wilfully ignoring. There are other reasons that automatic version selection at execution time (whether minimal or maximal) are problematic. But the idea that security is "bogus" is a bit of a stretch.
- weberc2 8y ago
  I think you’ve forgotten that patches exist.
  - jacques_chester 8y ago
    I don't know why you think so.
    - weberc2 8y ago
      Because you seem to think the choice is between grabbing the latest and greatest (“Hey look, no bugs found in yesterday’s release!”) and an old version with lots of bugs. The alternative is patched versions, which your comment makes no mention of. Perhaps patches were what you meant by “new versions”, but then that doesn’t fit in the context of the conversation.
      - jacques_chester 8y ago
        > Perhaps patches were what you meant by “new versions” I thought that was obvious from context. I guess not. I'm used to the idea that latest versions are safer, mostly because I have worked in and around systems intended to create that assurance. Where I work we track every upstream CVE patch that affects us until it has been released to customers. Large parts of this process are automated. On a good day the patch lands upstream and a fresh version of our own software rolls off the production line a few hours, fully baked and fully tested. My sensitivity to allowing versions to fall behind comes from knowing that a) it doesn't have to happen and b) that software that fell behind cannot reduce its bug or vulnerability count without being upgraded. New versions absolutely introduce new bugs, but on the balance of risks, a known vulnerability is more likely to exploited than an unknown one.
wereHamster 8y ago
> […] silently gets pulled into your project. The core issue that caused all of OPs problems was that Renovate Bot claimed to have only updated one dependency (@sourcegraph/codeintellify), but also updated another, unrelated one (sanitize-html). If you can't trust the tools around you not to lie to your face, all bets are off. It doesn't matter whether our tools follow MSV or the opposite (LSV – Latest Version Selection?). We should always pin our dependencies to exact versions, and explicitly update them as needed. Renovate Bot (and its alternatives) attempt to automate that part. They send you notifications or even open pull requests, you can inspect the changelog of your dependencies and decide for yourself if the update is necessary (eg. because of security reasons) or if you prefer not to update out of fear of breaking something. Right now it seems Renovate Bot sucks at that, and should be improved accordingly.
- nicksnyder 8y ago
  Renovate Bot didn't do anything special. It ran exactly the same npm command that I would have run on the command line (update the single dependency), which would have produced the exact same result.
  - wereHamster 8y ago
    > Renovate Bot didn't do anything special. Maybe it /should/ do something special though. Maybe it should stop using npm. If it says "I updated package X" and it really updated X /and/ Y, then it didn't do what it promised. Maybe it should say "I updated package X and maybe also others I don't know…" that would be more accurate to how it currently works.
    - nicksnyder 8y ago
      This pain isn’t really about Renovate Bot. You can pretend that it doesn’t exist or that your project isn’t using it. I should still be able to update a single dependency without updating unrelated things.

by kjksf 8y ago

Jyaif 8y ago

There's a difference between an old version where everybody know about the vulnerabilities because they were patched, and a new version where pretty much nobody knows any vulnerability (otherwise they would have been patched).

jessaustin 8y ago
Everyone who knows about the old vulns can avoid them easily with minimum version, just by setting their dependency versions properly. The new unknown vulns about which no one knows are avoided automatically. Re-reading the thread, maybe this is what you meant? I'm not sure, possibly because this is also what 'kjksf meant.
- jacques_chester 8y ago
  The arguments are directed past each other. Minimal selection breaks down when you ignore it and thereby allow known bugs to pile up. Maximal selection breaks down when you ignore it and thereby allow unknown, untracked dependencies to pile up. The answer for both is that version updating should trigger CI activity and then be automatically applied when tested. Dependency tools should be as strictly dumb as possible. This gap exists because dependency management tools blossomed and matured in a manual developer-at-a-keyboard context before the kind of automation that makes their clever bits less relevant or necessary.
  - weberc2 8y ago
    Vulnerability scanning favors old versions. If you have this, you’re probably in a (considerably?) safer place with MVS than max selection.
    - jacques_chester 8y ago
      > Vulnerability scanning favors old versions. I don't follow. Could you elaborate?
      - weberc2 8y ago
        Vuln scanning depends on vulns having been discovered. These vulns are more likely to be discovered in old versions than new versions.
        jacques_chester 8y ago
        I agree. But I think we're disagreeing on what policy follows from that.

jacques_chester 8y ago

> The "security issue" argument is bogus because while new code can fix bugs it can also introduce new bugs. Old versions only ever increase their known-bug count. Only a new version can reduce a known-bug count. It follows that the longer you stick to a version, the more problems you are guaranteed to be wilfully ignoring. There are other reasons that automatic version selection at execution time (whether minimal or maximal) are problematic. But the idea that security is "bogus" is a bit of a stretch.

weberc2 8y ago
I think you’ve forgotten that patches exist.
- jacques_chester 8y ago
  I don't know why you think so.
  - weberc2 8y ago
    Because you seem to think the choice is between grabbing the latest and greatest (“Hey look, no bugs found in yesterday’s release!”) and an old version with lots of bugs. The alternative is patched versions, which your comment makes no mention of. Perhaps patches were what you meant by “new versions”, but then that doesn’t fit in the context of the conversation.
    - jacques_chester 8y ago
      > Perhaps patches were what you meant by “new versions” I thought that was obvious from context. I guess not. I'm used to the idea that latest versions are safer, mostly because I have worked in and around systems intended to create that assurance. Where I work we track every upstream CVE patch that affects us until it has been released to customers. Large parts of this process are automated. On a good day the patch lands upstream and a fresh version of our own software rolls off the production line a few hours, fully baked and fully tested. My sensitivity to allowing versions to fall behind comes from knowing that a) it doesn't have to happen and b) that software that fell behind cannot reduce its bug or vulnerability count without being upgraded. New versions absolutely introduce new bugs, but on the balance of risks, a known vulnerability is more likely to exploited than an unknown one.

wereHamster 8y ago

> […] silently gets pulled into your project. The core issue that caused all of OPs problems was that Renovate Bot claimed to have only updated one dependency (@sourcegraph/codeintellify), but also updated another, unrelated one (sanitize-html). If you can't trust the tools around you not to lie to your face, all bets are off. It doesn't matter whether our tools follow MSV or the opposite (LSV – Latest Version Selection?). We should always pin our dependencies to exact versions, and explicitly update them as needed. Renovate Bot (and its alternatives) attempt to automate that part. They send you notifications or even open pull requests, you can inspect the changelog of your dependencies and decide for yourself if the update is necessary (eg. because of security reasons) or if you prefer not to update out of fear of breaking something. Right now it seems Renovate Bot sucks at that, and should be improved accordingly.

nicksnyder 8y ago
Renovate Bot didn't do anything special. It ran exactly the same npm command that I would have run on the command line (update the single dependency), which would have produced the exact same result.
- wereHamster 8y ago
  > Renovate Bot didn't do anything special. Maybe it /should/ do something special though. Maybe it should stop using npm. If it says "I updated package X" and it really updated X /and/ Y, then it didn't do what it promised. Maybe it should say "I updated package X and maybe also others I don't know…" that would be more accurate to how it currently works.
  - nicksnyder 8y ago
    This pain isn’t really about Renovate Bot. You can pretend that it doesn’t exist or that your project isn’t using it. I should still be able to update a single dependency without updating unrelated things.