5 ms·

One of the things that worries me about nodejs is the huge chain of dependencies. I'm not an expert on these things so it would be amazing if someone could corr

by gedrap 11y ago

One of the things that worries me about nodejs is the huge chain of dependencies. I'm not an expert on these things so it would be amazing if someone could correct me if I'm wrong. It's enough for one of the packages down the line to break compatibility and don't change the version correctly (i.e. bump up major version number bit), or have a slightly too loose version requirements and everything breaks down the line. Ok, if something gets broken it's relatively easy to notice given the test coverage is good enough. However, it's much much harder when it comes to security breaches (like the one described in the linked article), you might not notice it for a long long time. Anecdotal data but I tried to teach the interns to use yeoman when they were working on a small angularjs project and it just didn't work, because some dependency somewhere was broken. Happened to me as well and the solution was to try to update it a few days later (should have opened an issue, I know). I'm using npm shrinkwrap to avoid surprises but still.. It just doesn't feel right. I shouldn't be risking to break the project just by updating the dependencies, unless I've decided to update one of the dependencies to a new major version.

glenjamin 11y ago
I think https://nodesecurity.io/ https://nodesecurity.io/ will help you. Security flagging that can parse your dependency tree is a good solution to this in my opinion.
eloisant 11y ago
Yes, you should use npm shrinkwrap. It baffles me that automatically update your dependencies is considering the right thing to do. Practically that means that you can push a semicolon fix, your CI server will fetch a different (newer) version of a dependency and break something completely unrelated.
- tlrobinson 11y ago
  I agree shrinkwrap is necessary, and should probably be a more standard part of the npm workflow, but it's kind of annoying to use in practice. For one it generates huge files. Like 750KB on a project with a couple dozen dependencies: https://github.com/metabase/metabase/blob/master/npm-shrinkwrap.json https://github.com/metabase/metabase/blob/master/npm-shrinkw... Secondly, it's not deterministic and will generate huge diffs every time you run it even if nothing changes. Uber has a tool called npm-shrinkwrap that in theory is supposed to solve the latter, but I've never gotten it working on my current projects: https://github.com/uber/npm-shrinkwrap https://github.com/uber/npm-shrinkwrap
- Killswitch 11y ago
  > It baffles me that automatically update your dependencies is considering the right thing to do. The idea is to rely on semver. If you do ~1.3.4 in your dependency then if that dependency follows semver properly, you'll get 1.3.5 if it's out, and your stuff will still work, but you're getting bug fixes and patches without having to keep an eye on the sometimes hundreds of dependencies. Luckily tools like greenkeeper.io are around now. The drawback is many people don't follow semver, so I opt to appending --save-exact to all npm installs (actually have npm config set save-exact true)
  - gedrap 11y ago
    Exactly. In ideal world, semver would solve this issue really easily (probably not completely but to a large extent). However, there are so many dependencies in a typical nodejs project that I have hard time trusting that devs will follow semvar :)
raesene4 11y ago
If you want to be concerned about the npm chain of dependencies model, one thing you should probably think about is that there is no curation of code on npm, so a package could be malicious (e.g. if an attacker compromised a set of package maintainer creds from a dot file) and npm packages (like most package formats) have the ability to execute code on install...
- LunaSea 11y ago
  You mean like every package manager of every programming language ?
  - raesene4 11y ago
    Absolutely! Although that said not all of them store creds in dot files by default...
assafh 11y ago
Yes, shrinkwrapp is a good solution to lock down your dependency versions. Regarding the security breaches, this is a big problem because, as you mention, there are huge chains of dependencies, and also some of the vulnerabilities are not even in your direct dependency - which makes it hard to fix vulnerabilities. Snyk recently released stats that found that 14% of npm packages carry known vulnerabilities. Also 64% of our users have found a vulnerability in a private repo, so the chance that your package also contains a vulnerable dependency is quite high.(I work there btw) We released an open source cli tool that helps you find and fix vulnerabilities in your dependencies tree. You are welcome to try it - https://snyk.io https://snyk.io. You can also take a look at our open source vulnerabilities database with advisories and patches at https://github.com/Snyk/vulndb https://github.com/Snyk/vulndb.
- gedrap 11y ago
  Looks interesting, I will take a look :) after briefly skimming through the landing page, seems like a cool thing to add to CI process because, well, there's no reason not to automate that and hope that folks will run it from time to time.

by gedrap 11y ago

glenjamin 11y ago

I think https://nodesecurity.io/ https://nodesecurity.io/ will help you. Security flagging that can parse your dependency tree is a good solution to this in my opinion.

eloisant 11y ago

Yes, you should use npm shrinkwrap. It baffles me that automatically update your dependencies is considering the right thing to do. Practically that means that you can push a semicolon fix, your CI server will fetch a different (newer) version of a dependency and break something completely unrelated.

tlrobinson 11y ago
I agree shrinkwrap is necessary, and should probably be a more standard part of the npm workflow, but it's kind of annoying to use in practice. For one it generates huge files. Like 750KB on a project with a couple dozen dependencies: https://github.com/metabase/metabase/blob/master/npm-shrinkwrap.json https://github.com/metabase/metabase/blob/master/npm-shrinkw... Secondly, it's not deterministic and will generate huge diffs every time you run it even if nothing changes. Uber has a tool called npm-shrinkwrap that in theory is supposed to solve the latter, but I've never gotten it working on my current projects: https://github.com/uber/npm-shrinkwrap https://github.com/uber/npm-shrinkwrap
Killswitch 11y ago
> It baffles me that automatically update your dependencies is considering the right thing to do. The idea is to rely on semver. If you do ~1.3.4 in your dependency then if that dependency follows semver properly, you'll get 1.3.5 if it's out, and your stuff will still work, but you're getting bug fixes and patches without having to keep an eye on the sometimes hundreds of dependencies. Luckily tools like greenkeeper.io are around now. The drawback is many people don't follow semver, so I opt to appending --save-exact to all npm installs (actually have npm config set save-exact true)
- gedrap 11y ago
  Exactly. In ideal world, semver would solve this issue really easily (probably not completely but to a large extent). However, there are so many dependencies in a typical nodejs project that I have hard time trusting that devs will follow semvar :)

raesene4 11y ago

If you want to be concerned about the npm chain of dependencies model, one thing you should probably think about is that there is no curation of code on npm, so a package could be malicious (e.g. if an attacker compromised a set of package maintainer creds from a dot file) and npm packages (like most package formats) have the ability to execute code on install...

LunaSea 11y ago
You mean like every package manager of every programming language ?
- raesene4 11y ago
  Absolutely! Although that said not all of them store creds in dot files by default...

assafh 11y ago

Yes, shrinkwrapp is a good solution to lock down your dependency versions. Regarding the security breaches, this is a big problem because, as you mention, there are huge chains of dependencies, and also some of the vulnerabilities are not even in your direct dependency - which makes it hard to fix vulnerabilities. Snyk recently released stats that found that 14% of npm packages carry known vulnerabilities. Also 64% of our users have found a vulnerability in a private repo, so the chance that your package also contains a vulnerable dependency is quite high.(I work there btw) We released an open source cli tool that helps you find and fix vulnerabilities in your dependencies tree. You are welcome to try it - https://snyk.io https://snyk.io. You can also take a look at our open source vulnerabilities database with advisories and patches at https://github.com/Snyk/vulndb https://github.com/Snyk/vulndb.

gedrap 11y ago
Looks interesting, I will take a look :) after briefly skimming through the landing page, seems like a cool thing to add to CI process because, well, there's no reason not to automate that and hope that folks will run it from time to time.