5 ms·

> As long as each dependency is listed in package.json, anyone can create a working local copy of your app - including node_modules - by running npm install. B

by jasim 11y ago

> As long as each dependency is listed in package.json, anyone can create a working local copy of your app - including node_modules - by running npm install. But it is going to download the internet, will take forever, and if the network is patchy, any dependencies that weren't downloaded will remain unresolved, even if you reinstall the package, and you'll have to manually install them with the help of `npm list`, or nuke node_modules and start over, and pray that this time the network remains perfect for a few minutes at a stretch. (someone please tell me I've got it all wrong)

sisk 11y ago
You're not necessarily wrong (that a ton of modules will be downloaded) but there are two points worth mentioning. 1) you can install from cache with something like: npm install --cache-min 9999999 2) npm's infrastructure has scaled extremely well. Gone are the days of spotty uptime and slow resolution. I maintain an npm proxy for one of—if not the—single largest consumer of npm modules and other than the very occasional 502 blip, npm has been rock-solid.
- jasim 11y ago
  Thanks for clarifying. I'll try the cache-min flag. But I suspect that if the cache contains packages with unresolved dependencies, then they'll also be of no use, and have to be removed too. Also npm's servers has never given any trouble, the issue was always with my networks.
albinofrenchy 11y ago
Either way you are going to have to potentially download a lot of stuff; I'd rather not have that bloat integrated into the repo. Also consider if you come onto a project and they've upgraded a library version multiple times, when you clone from the repo you are going to grab every revision they've ever used and checked in. This is going to be much bigger and slower than the equivalent npm install for that package. If you are grabbing from an external repo, the spotty network is going to mess you up either way. If it's internal and your internet is that spotty, consider hosting a local npm cacheing server like this -- https://www.npmjs.com/package/sinopia https://www.npmjs.com/package/sinopia. My only issue with this technique is that 'npm start' and the rest aren't smart enough (please correct me if I'm wrong and there is a way to make it care!) to grab dependencies if you upgrade a dependency version and check that in.
- jasim 11y ago
  (bundling node_modules into git is definitely not an option)
- Pr3fix 11y ago
  If you're checking your `node_modules` into the repo you're doing it wrong :\ should just check in package.json.
k__ 11y ago
I found the versions a bigger problem. I install something --save and get "^1.0.0" into my package.json, which resolves to 1.0.0 Someone installs my stuff and on her machine it resolves to 1.0.3 and nothing works anymore, even if she cloned an exact copy of my repository etc. Next step is to lock down all versions in my package.json, so all wildcards are gone. But the packages I depend on still have wildcards and can break happily. This happens far too often for my taste and can (as far as I know) only be solved with third party software.
- hunterloftis 11y ago
  You can solve it with the npm shrinkwrap workflow mentioned in the article. It takes somewhat more effort, but locks down the full dependency tree. People have historically experienced a lot of issues with shrinkwrap in older versions of npm; I'm optimistic that npm 3 solves a lot of them.
  - k__ 11y ago
    Yes, I hope for npm3 too, but at the moment everyone in my team tells me it is far to slow to use it for anything :\
- mkozlows 11y ago
  This is what you want to happen. The alternative is, you shrinkwrap everything and lock it down, and your dependencies never change. Except that now it's been a year, and you want to update your dependencies, and it's this huge jump that requires so many changes that it's going to take a month of concentrated pain to do, so you never do it, and welcome to the world of legacy code. Take your dependency-update hits in small bursts on the regular, and only lock down (via shrinkwrap or Docker images) for QA/release builds.
  - iamsohungry 11y ago
    > This is what you want to happen. Uh, no, I don't want my dependencies to break out from under me, thanks. This is a major problem with the JS culture that I cannot stand. Too many JS devs can't be bothered to write software that doesn't suck, and so they try to instead tell their users their crappy problems are actually what they want. No, if you push changes to a library that break your library users' software without warning and a reasonable upgrade plan, you're doing a bad job. > The alternative is, you shrinkwrap everything and lock it down, and your dependencies never change. Or I use dependencies that upgrade their dependencies intentionally instead of whenever the `^` does it for them. The problem is that most of the JS packages out there are garbage software that doesn't do that. So it's necessary to be careful and only use dependencies which are well-maintained and reliable. ...which isn't unique to npm or a new problem, it's been well-known since all the way back when CPAN was new. Just because installing libraries is easy doesn't mean it's a good idea. > Except that now it's been a year, and you want to update your dependencies, and it's this huge jump that requires so many changes that it's going to take a month of concentrated pain to do, so you never do it, and welcome to the world of legacy code. Or you can do your job like a competent professional and don't wait a year to upgrade your dependencies. Again, fewer, more reliable dependencies helps with this; if you are using reasonable dependencies, they'll push few breaking changes and give warnings and reasonable upgrade plans.
draw_down 11y ago
Pick your poison, big dependencies or big repo + duplication. But the author is correct, the node way is to put deps in package.json and `npm i`.

by jasim 11y ago

sisk 11y ago

You're not necessarily wrong (that a ton of modules will be downloaded) but there are two points worth mentioning. 1) you can install from cache with something like: npm install --cache-min 9999999 2) npm's infrastructure has scaled extremely well. Gone are the days of spotty uptime and slow resolution. I maintain an npm proxy for one of—if not the—single largest consumer of npm modules and other than the very occasional 502 blip, npm has been rock-solid.

jasim 11y ago
Thanks for clarifying. I'll try the cache-min flag. But I suspect that if the cache contains packages with unresolved dependencies, then they'll also be of no use, and have to be removed too. Also npm's servers has never given any trouble, the issue was always with my networks.

albinofrenchy 11y ago

Either way you are going to have to potentially download a lot of stuff; I'd rather not have that bloat integrated into the repo. Also consider if you come onto a project and they've upgraded a library version multiple times, when you clone from the repo you are going to grab every revision they've ever used and checked in. This is going to be much bigger and slower than the equivalent npm install for that package. If you are grabbing from an external repo, the spotty network is going to mess you up either way. If it's internal and your internet is that spotty, consider hosting a local npm cacheing server like this -- https://www.npmjs.com/package/sinopia https://www.npmjs.com/package/sinopia. My only issue with this technique is that 'npm start' and the rest aren't smart enough (please correct me if I'm wrong and there is a way to make it care!) to grab dependencies if you upgrade a dependency version and check that in.

jasim 11y ago
(bundling node_modules into git is definitely not an option)
Pr3fix 11y ago
If you're checking your `node_modules` into the repo you're doing it wrong :\ should just check in package.json.

k__ 11y ago

I found the versions a bigger problem. I install something --save and get "^1.0.0" into my package.json, which resolves to 1.0.0 Someone installs my stuff and on her machine it resolves to 1.0.3 and nothing works anymore, even if she cloned an exact copy of my repository etc. Next step is to lock down all versions in my package.json, so all wildcards are gone. But the packages I depend on still have wildcards and can break happily. This happens far too often for my taste and can (as far as I know) only be solved with third party software.

hunterloftis 11y ago
You can solve it with the npm shrinkwrap workflow mentioned in the article. It takes somewhat more effort, but locks down the full dependency tree. People have historically experienced a lot of issues with shrinkwrap in older versions of npm; I'm optimistic that npm 3 solves a lot of them.
- k__ 11y ago
  Yes, I hope for npm3 too, but at the moment everyone in my team tells me it is far to slow to use it for anything :\
mkozlows 11y ago
This is what you want to happen. The alternative is, you shrinkwrap everything and lock it down, and your dependencies never change. Except that now it's been a year, and you want to update your dependencies, and it's this huge jump that requires so many changes that it's going to take a month of concentrated pain to do, so you never do it, and welcome to the world of legacy code. Take your dependency-update hits in small bursts on the regular, and only lock down (via shrinkwrap or Docker images) for QA/release builds.
- iamsohungry 11y ago
  > This is what you want to happen. Uh, no, I don't want my dependencies to break out from under me, thanks. This is a major problem with the JS culture that I cannot stand. Too many JS devs can't be bothered to write software that doesn't suck, and so they try to instead tell their users their crappy problems are actually what they want. No, if you push changes to a library that break your library users' software without warning and a reasonable upgrade plan, you're doing a bad job. > The alternative is, you shrinkwrap everything and lock it down, and your dependencies never change. Or I use dependencies that upgrade their dependencies intentionally instead of whenever the `^` does it for them. The problem is that most of the JS packages out there are garbage software that doesn't do that. So it's necessary to be careful and only use dependencies which are well-maintained and reliable. ...which isn't unique to npm or a new problem, it's been well-known since all the way back when CPAN was new. Just because installing libraries is easy doesn't mean it's a good idea. > Except that now it's been a year, and you want to update your dependencies, and it's this huge jump that requires so many changes that it's going to take a month of concentrated pain to do, so you never do it, and welcome to the world of legacy code. Or you can do your job like a competent professional and don't wait a year to upgrade your dependencies. Again, fewer, more reliable dependencies helps with this; if you are using reasonable dependencies, they'll push few breaking changes and give warnings and reasonable upgrade plans.

draw_down 11y ago

Pick your poison, big dependencies or big repo + duplication. But the author is correct, the node way is to put deps in package.json and `npm i`.