6 ms·

And the real problem with all of this is that npm has become so ubiquitous that you can't get away from it if you're doing client side development (not even by

And the real problem with all of this is that npm has become so ubiquitous that you can't get away from it if you're doing client side development (not even by switching to yarn). You simply have to endure it, and accept that every now and again, for reasons entirely beyond your control (and at a quite possibly very inconvenient moment) it's going to break. This really annoys me, but what can one do? At least it's fixed now.

Arnavion 8y ago
Vendor your dependencies, or host clones on infrastructure that you're responsible for.
- bartread 8y ago
  Vendoring isn't ideal: it makes, for example, code-reviews a PITA, and you still have the problem of what to do when you upgrade or add a dependency. Then you're back to npm (or yarn). Granted, at much reduced frequency, but it's still there. Maybe we'll go down the clone route if the cost of npm issues becomes too high relative to the hassle and expense of maintaining our own.
  - numbsafari 8y ago
    One way to keep vendoring from causing nightmare code reviews is to keep updates to vendored components in their own release/branch/PR. Yes, there will be changes to your code to accommodate the updated packages, but it keeps it very focused so that the code review isn’t also trying to evaluate updates to business rules or functionality. If you are updating components along with business rule updates, then, yes, it’s going to complicate code review, regardless of vendoring.
  - acemarke 8y ago
    That's why I really like Yarn's "offline mirror" feature. It lets you cache the tarballs themselves locally. There's also a tool called `shrinkpack` that lets you do the same when working with NPM itself, but I'm not sure if it works right with NPM5/6. I wrote a post a while back about using Yarn's offline mirror: http://blog.isquaredsoftware.com/2017/07/practical-redux-part-9-managing-dependencies/ http://blog.isquaredsoftware.com/2017/07/practical-redux-par...
  - krapp 8y ago
    >Vendoring isn't ideal: it makes, for example, code-reviews a PITA, and you still have the problem of what to do when you upgrade or add a dependency. The status quo in which anything that goes wrong breaks the entire universe because no one vendors is also not ideal. However, consider the long list of disasters that have occurred with NPM recently, and how many of them would have been less disastrous had vendoring been the exception rather than the rule. Left-pad wouldn't even have been an issue, for example - builds simply would have been unable to update, but nothing live would have been affected. To misquote Ben Franklin here, the tradeoff isn't between a greater ideal and a lesser ideal, but between security and convenience.
- acejam 8y ago
  Vendoring dependencies typically entails committing at least 200-500 MB of packages into a git repo. No thanks. Availability should be easy to control with running your own Nexus or other internal registry. The rest (package versions, etc) can often be solved with an npm 5+ lock file.
flipp3r 8y ago
> This really annoys me, but what can one do? At least it's fixed now. Install your own repository manager? This is the standard in every company I've worked for so far, at least in the Java world. Artifactory supports NPM, so set it up as a proxy.
- bartread 8y ago
  That's interesting - thanks. Of course, it has a price tag, but so does the rest of our pipeline, along with our time.
  - PebblesHD 8y ago
    As far as the monetary price of implementing such a system, Nexus OSS[1] also supports NPM proxying and is free for basic usage. [1] - https://www.sonatype.com/nexus-repository-oss https://www.sonatype.com/nexus-repository-oss
    - zamber 8y ago
      We have Nexus in a subnet for faster installing. We had to write scripts porting over lockfiles from npm to nexus and back. This had to be added to a precommit hook to not break CI. Seriously, package.json should allow to specify what endpoints should be used if available in a given order. Now it's up to each dev team to handle. My main concern is that it's brittle. Nexus caches exact versions and nothing more so we don't even have assurance that it will work nicely when NPM goes down. On the other hand lockfiles are awesome. I missed them back in 2012... copying over node_modules on USB drives was not cool.
      - acejam 8y ago
        You can specify what registry to use with a simple project-based .npmrc file. We have ours point to our Nexus npm proxy.
        zamber 8y ago
ino 8y ago
> you can't get away from it if you're doing client side development (not even by switching to yarn). Explain yourself better or top spreading this nonsense. What do they give you that you couldn't develop without them?
- pcx 8y ago
  This is true. The case I encountered was, you cannot install `aws-sam-local` with yarn, you will need npm to do the install.
TheAceOfHearts 8y ago
As others suggested, any serious deployment should be hosting their own registry / mirror or using a paid service. This also saves you in cases such as the left-pad issue, as your mirror would still have the package. It is unwise to rely on free third-party services which are out of your control, especially for something as important as deployment! This problem is not limited to npm. I remember a few years back there were similar issues with RubyGems, where it'd go down leaving many developers unable to deploy. Heck, how many projects do you think would be left unable to deploy if GitHub went down? I remember a few years back they'd have their occasional issues and Twitter would become a storm of angry developers. For many projects not being able to deploy or develop at any moment, as well as dealing with left-pad style issues is implicitly accepted as a reasonable trade-off.
krapp 8y ago
>This really annoys me, but what can one do? At least it's fixed now. The weird thing is, open source development is as close to a free market as one can get. Frustration with NPM should result in multiple javascript package managers competing to undermine NPM's dominance, but the only alternative is one that uses the Node registry.

Arnavion 8y ago

Vendor your dependencies, or host clones on infrastructure that you're responsible for.

bartread 8y ago
Vendoring isn't ideal: it makes, for example, code-reviews a PITA, and you still have the problem of what to do when you upgrade or add a dependency. Then you're back to npm (or yarn). Granted, at much reduced frequency, but it's still there. Maybe we'll go down the clone route if the cost of npm issues becomes too high relative to the hassle and expense of maintaining our own.
- numbsafari 8y ago
  One way to keep vendoring from causing nightmare code reviews is to keep updates to vendored components in their own release/branch/PR. Yes, there will be changes to your code to accommodate the updated packages, but it keeps it very focused so that the code review isn’t also trying to evaluate updates to business rules or functionality. If you are updating components along with business rule updates, then, yes, it’s going to complicate code review, regardless of vendoring.
- acemarke 8y ago
  That's why I really like Yarn's "offline mirror" feature. It lets you cache the tarballs themselves locally. There's also a tool called `shrinkpack` that lets you do the same when working with NPM itself, but I'm not sure if it works right with NPM5/6. I wrote a post a while back about using Yarn's offline mirror: http://blog.isquaredsoftware.com/2017/07/practical-redux-part-9-managing-dependencies/ http://blog.isquaredsoftware.com/2017/07/practical-redux-par...
- krapp 8y ago
  >Vendoring isn't ideal: it makes, for example, code-reviews a PITA, and you still have the problem of what to do when you upgrade or add a dependency. The status quo in which anything that goes wrong breaks the entire universe because no one vendors is also not ideal. However, consider the long list of disasters that have occurred with NPM recently, and how many of them would have been less disastrous had vendoring been the exception rather than the rule. Left-pad wouldn't even have been an issue, for example - builds simply would have been unable to update, but nothing live would have been affected. To misquote Ben Franklin here, the tradeoff isn't between a greater ideal and a lesser ideal, but between security and convenience.
acejam 8y ago
Vendoring dependencies typically entails committing at least 200-500 MB of packages into a git repo. No thanks. Availability should be easy to control with running your own Nexus or other internal registry. The rest (package versions, etc) can often be solved with an npm 5+ lock file.

flipp3r 8y ago

> This really annoys me, but what can one do? At least it's fixed now. Install your own repository manager? This is the standard in every company I've worked for so far, at least in the Java world. Artifactory supports NPM, so set it up as a proxy.

bartread 8y ago
That's interesting - thanks. Of course, it has a price tag, but so does the rest of our pipeline, along with our time.
- PebblesHD 8y ago
  As far as the monetary price of implementing such a system, Nexus OSS[1] also supports NPM proxying and is free for basic usage. [1] - https://www.sonatype.com/nexus-repository-oss https://www.sonatype.com/nexus-repository-oss
  - zamber 8y ago
    We have Nexus in a subnet for faster installing. We had to write scripts porting over lockfiles from npm to nexus and back. This had to be added to a precommit hook to not break CI. Seriously, package.json should allow to specify what endpoints should be used if available in a given order. Now it's up to each dev team to handle. My main concern is that it's brittle. Nexus caches exact versions and nothing more so we don't even have assurance that it will work nicely when NPM goes down. On the other hand lockfiles are awesome. I missed them back in 2012... copying over node_modules on USB drives was not cool.
    - acejam 8y ago
      You can specify what registry to use with a simple project-based .npmrc file. We have ours point to our Nexus npm proxy.
      - zamber 8y ago

ino 8y ago

> you can't get away from it if you're doing client side development (not even by switching to yarn). Explain yourself better or top spreading this nonsense. What do they give you that you couldn't develop without them?

pcx 8y ago
This is true. The case I encountered was, you cannot install `aws-sam-local` with yarn, you will need npm to do the install.

TheAceOfHearts 8y ago

As others suggested, any serious deployment should be hosting their own registry / mirror or using a paid service. This also saves you in cases such as the left-pad issue, as your mirror would still have the package. It is unwise to rely on free third-party services which are out of your control, especially for something as important as deployment! This problem is not limited to npm. I remember a few years back there were similar issues with RubyGems, where it'd go down leaving many developers unable to deploy. Heck, how many projects do you think would be left unable to deploy if GitHub went down? I remember a few years back they'd have their occasional issues and Twitter would become a storm of angry developers. For many projects not being able to deploy or develop at any moment, as well as dealing with left-pad style issues is implicitly accepted as a reasonable trade-off.

krapp 8y ago

>This really annoys me, but what can one do? At least it's fixed now. The weird thing is, open source development is as close to a free market as one can get. Frustration with NPM should result in multiple javascript package managers competing to undermine NPM's dominance, but the only alternative is one that uses the Node registry.