6 ms·

This whole "automate your dependency updates" is gonna bit people in the ass sooner or later. No one except a few select I know, actually review their dependenc

This whole "automate your dependency updates" is gonna bit people in the ass sooner or later. No one except a few select I know, actually review their dependency changes. Most basically have some sort of automated flow where each update becomes a pull-request, and if passes the tests, the person merges it. Or they do "once every X" full updates of everything, and if the tests pass, they merge it. Wonder if we soon will have a service you simply connect to your repository and now this is fully automatic, with the commit to master happening automatically if the tests pass?

deleted 7y ago
[deleted]
myroon5 7y ago
Dependabot can be configured to automatically commit to master if the tests pass
- crgwbr 7y ago
  I believe the OPs point wasn’t that that’s what we need—it’s that that’s a bad idea. It’s insecure and bound to break things eventually.
- karlding 7y ago
  The point that the parent commenter is highlighting is that if somehow a malicious package version is introduced, since nobody actually is auditing the changes that are being pulled in, it is possible for the tests to pass and introduce a security vulnerability. They're suggesting that the contents of the actual diff in the packages should be reviewed.
deleted 7y ago
[deleted]
Eikon 7y ago
Not only that but also the security implications. I always though that is was really crazy that people are just willing to download megabytes of arbitrary code from sometimes unknown sources without doing any security audit whatsoever. And when people are doing audits, they most often just clone the repository without checking if the archive on whatever package manager is matching the source code. It’s a disaster waiting to happen.
jzoch 7y ago
Its supposed to be part of a larger process, a culture of constantly committing small changes and have many automated gates before you reach production (not just your CI instance). Real CI/CD. Heroku does something like this because they release to a staging instance, gather metrics, rollback if necessary, then release to a canary in production, repeat monitoring, then release to a wider and wider surface area. Obviously this does not fix security bugs introduced, but functionality wise it works well. Security tends to get better over time not worse too when it comes to dependencies i find.
- capableweb 7y ago
  Sure, I have no problem with having a automated process, but things like reviewing dependencies have to be manual when code has changed, how can you otherwise ensure there is nothing nasty hidden there? With the dawn of these automated services, it becomes easier to do what you can essentially call propagation-hacks, where one compromised module would trigger bunch of other packages to auto-update, which triggers more and which eventually gets run in your production environment. Also, not sure "security tends to get better over time". When a library is small and unused, no one really cares about reviewing the security as either it's not used by many to warrant it, or it's surface is so small it's not worth for people to try to find holes. As the software grows, it gets more users, and more reviews, which also finds more issues. Probably the security is the same as in the beginning, and depends more on the maintainers, then depending how long it existed. One example could be OpenSSL which everyone been assuming for a long time was totally fine.
  - londt8 7y ago
    I think reviewing all code in the dependencies is not reasonable. When i add new dependency to my projects i evaluate how trustworthy the maintainers seem.
    - capableweb 7y ago
      Instead of throwing your hands in the air and saying "Well, too many dependencies, can't review it", you can pull in less dependencies so you can manage to review it. Especially from sources where anyone can publish anything basically.
hermanradtke 7y ago
I wonder if there is any research on whether someone is better off not updating at all or automatically updating. I would bet that apps are better off automatically updating to remove published vulnerabilities than they are worrying about the handful of bitcoin wallet stealing backdoors. [1] It comes down to trust. Automatically updating something like https://www.npmjs.com/package/express https://www.npmjs.com/package/express is probably pretty safe. Automatically updating a relatively unknown package made by an unknown person is much less safe. Also, it goes beyond application dependency changes. What about your build server or that SaaS build pipeline you use? Key infrastructure gets hacked [2][3] and it turns out all the time you spent reviewing express updates should have instead been spent reviewing changes from somewhere else. [1]: https://github.com/dominictarr/event-stream/issues/116 https://github.com/dominictarr/event-stream/issues/116 [2]: https://blog.linuxmint.com/?p=2994 https://blog.linuxmint.com/?p=2994 [3]: https://blogs.adobe.com/security/2012/09/inappropriate-use-of-adobe-code-signing-certificate.html https://blogs.adobe.com/security/2012/09/inappropriate-use-o...
paulddraper 7y ago
I too am interested if frequently updates for the sake of updates leads to better stability or worse stability. I'm reminded of a single SemVer patch upgrade done for https://github.com/lifaon74/url-polyfill https://github.com/lifaon74/url-polyfill that then subtle broke my application but only in IE (where the polyfill is used) and only in production (because the URL is different). I knew about once the tickets started flooding in. I have since adopted more of the mindset "if it ain't broke don't fix it." I do run security checks (e.g. npm audit), and I still do upgrade, but I do it at a pace where I can afford good testing and careful attention. (IE is hard AF to set up tests for, though like all problems it can be solved with more money.)

deleted 7y ago

[deleted]

myroon5 7y ago

Dependabot can be configured to automatically commit to master if the tests pass

crgwbr 7y ago
I believe the OPs point wasn’t that that’s what we need—it’s that that’s a bad idea. It’s insecure and bound to break things eventually.
karlding 7y ago
The point that the parent commenter is highlighting is that if somehow a malicious package version is introduced, since nobody actually is auditing the changes that are being pulled in, it is possible for the tests to pass and introduce a security vulnerability. They're suggesting that the contents of the actual diff in the packages should be reviewed.

deleted 7y ago

[deleted]

Eikon 7y ago

Not only that but also the security implications. I always though that is was really crazy that people are just willing to download megabytes of arbitrary code from sometimes unknown sources without doing any security audit whatsoever. And when people are doing audits, they most often just clone the repository without checking if the archive on whatever package manager is matching the source code. It’s a disaster waiting to happen.

jzoch 7y ago

Its supposed to be part of a larger process, a culture of constantly committing small changes and have many automated gates before you reach production (not just your CI instance). Real CI/CD. Heroku does something like this because they release to a staging instance, gather metrics, rollback if necessary, then release to a canary in production, repeat monitoring, then release to a wider and wider surface area. Obviously this does not fix security bugs introduced, but functionality wise it works well. Security tends to get better over time not worse too when it comes to dependencies i find.

capableweb 7y ago
Sure, I have no problem with having a automated process, but things like reviewing dependencies have to be manual when code has changed, how can you otherwise ensure there is nothing nasty hidden there? With the dawn of these automated services, it becomes easier to do what you can essentially call propagation-hacks, where one compromised module would trigger bunch of other packages to auto-update, which triggers more and which eventually gets run in your production environment. Also, not sure "security tends to get better over time". When a library is small and unused, no one really cares about reviewing the security as either it's not used by many to warrant it, or it's surface is so small it's not worth for people to try to find holes. As the software grows, it gets more users, and more reviews, which also finds more issues. Probably the security is the same as in the beginning, and depends more on the maintainers, then depending how long it existed. One example could be OpenSSL which everyone been assuming for a long time was totally fine.
- londt8 7y ago
  I think reviewing all code in the dependencies is not reasonable. When i add new dependency to my projects i evaluate how trustworthy the maintainers seem.
  - capableweb 7y ago
    Instead of throwing your hands in the air and saying "Well, too many dependencies, can't review it", you can pull in less dependencies so you can manage to review it. Especially from sources where anyone can publish anything basically.

hermanradtke 7y ago

I wonder if there is any research on whether someone is better off not updating at all or automatically updating. I would bet that apps are better off automatically updating to remove published vulnerabilities than they are worrying about the handful of bitcoin wallet stealing backdoors. [1] It comes down to trust. Automatically updating something like https://www.npmjs.com/package/express https://www.npmjs.com/package/express is probably pretty safe. Automatically updating a relatively unknown package made by an unknown person is much less safe. Also, it goes beyond application dependency changes. What about your build server or that SaaS build pipeline you use? Key infrastructure gets hacked [2][3] and it turns out all the time you spent reviewing express updates should have instead been spent reviewing changes from somewhere else. [1]: https://github.com/dominictarr/event-stream/issues/116 https://github.com/dominictarr/event-stream/issues/116 [2]: https://blog.linuxmint.com/?p=2994 https://blog.linuxmint.com/?p=2994 [3]: https://blogs.adobe.com/security/2012/09/inappropriate-use-of-adobe-code-signing-certificate.html https://blogs.adobe.com/security/2012/09/inappropriate-use-o...

paulddraper 7y ago

I too am interested if frequently updates for the sake of updates leads to better stability or worse stability. I'm reminded of a single SemVer patch upgrade done for https://github.com/lifaon74/url-polyfill https://github.com/lifaon74/url-polyfill that then subtle broke my application but only in IE (where the polyfill is used) and only in production (because the URL is different). I knew about once the tickets started flooding in. I have since adopted more of the mindset "if it ain't broke don't fix it." I do run security checks (e.g. npm audit), and I still do upgrade, but I do it at a pace where I can afford good testing and careful attention. (IE is hard AF to set up tests for, though like all problems it can be solved with more money.)