4 ms·

Analysing 1.65M versions of Node.js modules in NPM

56 points by coldcode 10y ago

masukomi 10y ago
ugh... so much... details are great and all, but this is a lot. summary / tl;dr would be very nice
- dang 10y ago
  This isn't a great comment to post to HN. In-depth content, and even effort on the part of the reader, are encouraged here.
  - jessaustin 10y ago
    In fairness, parent comment is pretty accurate. TFA promises "analysis", and the closest it comes to that is "wow it's wild that there are so many packages" and "haha see how lame node is because some old packages are still used". I'm not going to flag this, but how did it get 30 upvotes?
    - dang 10y ago
      If you (or anyone) wants to help the community by pointing out that an article lacks substance, it muddies the waters to do it like that (or even like you did here, though at least your point was clearer). People often post that sort of comment to vent or be snarky, and those things impede understanding.
- mrexroad 10y ago
  "Part 6 — Summary tl;dr it’s a summary" yep, tl;dr was there at the bottom; i laughed.
psuter 10y ago
For those interested in the topic, we ran a related set of experiments recently [1], and cross-referenced npm packages with (a subset of) Javascript GitHub projects to see how packages and versions are used in practice. [1] http://www.apiful.io/intro/2016/06/01/npm-analysis.html http://www.apiful.io/intro/2016/06/01/npm-analysis.html
ilaksh 10y ago
I have never had a package break between dev and prod when using versions like ^x.x.x. Also, you need to test and ideally stage before deployment. If you don't want to deploy with `npm i` then just copy the files or compress and then copy that and uncompress. Also the new flat modules reduce the total install time. The author advocates copy-pasting code rather than including small modules. To me this is very misguided. Npm is the greatest code-reuse system ever created. The failure of software engineers to recognize that is an indication to me of a lack of depth of engineering knowledge and experience. Deploys may take a few minutes. You just need to factor that and don't switch off prod servers until updated ones are ready.
- merb 10y ago
  Code reuse is fine, but taking that over the top is not fine. The same with Java some things actually do too much. Especially dealing with many external dependencies is a real pain. A really big one.
  - ilaksh 10y ago
    I found the many dependencies to be a pain very early on when I was not specifying versions in my package.json. Now I use npm i --save which gets versions so.. it really is not a pain to have many dependencies with npm. In Java and many other systems, yes, it is very painful. npm is different.
- AgentME 10y ago
  > I have never had a package break between dev and prod when using versions like ^x.x.x. I've run into it. Dependencies get bugs sometimes. Easy to pin versions with npm's shrinkwrap though. >The author advocates copy-pasting code rather than including small modules. To me this is very misguided. Yeah, I'm surprised how often people suggest this is the obvious solution as if it's not without its own problems. One of the things I dread most about working on a new codebase is finding an ancient version of some library code copy-pasted into the codebase with the inevitable unlabeled (and probably not high-quality) customizations made to it. Trying to figure out whether the library's documentation still applies, whether it still passes its original upstream tests with the customizations, or whether the copy can be safely replaced with a newer upstream version is painful.
AgentME 10y ago
>but we all remember the left-pad incident, right? It proved that having lots of dependencies might actually not be the best thing. npm no longer allows dependencies to be unpublished after they've been up for 24 hours. The issue that happened with left-pad was solved. It's possible for someone to instead publish a new non-functional version of a package, but I don't think that's unique to npm, and that won't affect people who have pinned to a specific version of the package.
thinkMOAR 10y ago
At first i thought you made a typo with 1.65 million "Grab a cup of coffee and enjoy the deep dive into the NPM ecosystem." Now taking my last sip of the coffee, i'm happy never to have spend a single second on node.js. The rapid and simple development via node.js does not weigh up to the bizar number of dependencies and crazy stuff that can originate from that. Interesting to see in your stats there is a decline in development, at least of new modules. Either the ideas people can come up with are drying up, or i'd be inclined to say (interest of) nodeJS development is in decline?

56 points by coldcode 10y ago

masukomi 10y ago

ugh... so much... details are great and all, but this is a lot. summary / tl;dr would be very nice

dang 10y ago
This isn't a great comment to post to HN. In-depth content, and even effort on the part of the reader, are encouraged here.
- jessaustin 10y ago
  In fairness, parent comment is pretty accurate. TFA promises "analysis", and the closest it comes to that is "wow it's wild that there are so many packages" and "haha see how lame node is because some old packages are still used". I'm not going to flag this, but how did it get 30 upvotes?
  - dang 10y ago
    If you (or anyone) wants to help the community by pointing out that an article lacks substance, it muddies the waters to do it like that (or even like you did here, though at least your point was clearer). People often post that sort of comment to vent or be snarky, and those things impede understanding.
mrexroad 10y ago
"Part 6 — Summary tl;dr it’s a summary" yep, tl;dr was there at the bottom; i laughed.

psuter 10y ago

For those interested in the topic, we ran a related set of experiments recently [1], and cross-referenced npm packages with (a subset of) Javascript GitHub projects to see how packages and versions are used in practice. [1] http://www.apiful.io/intro/2016/06/01/npm-analysis.html http://www.apiful.io/intro/2016/06/01/npm-analysis.html

ilaksh 10y ago

I have never had a package break between dev and prod when using versions like ^x.x.x. Also, you need to test and ideally stage before deployment. If you don't want to deploy with `npm i` then just copy the files or compress and then copy that and uncompress. Also the new flat modules reduce the total install time. The author advocates copy-pasting code rather than including small modules. To me this is very misguided. Npm is the greatest code-reuse system ever created. The failure of software engineers to recognize that is an indication to me of a lack of depth of engineering knowledge and experience. Deploys may take a few minutes. You just need to factor that and don't switch off prod servers until updated ones are ready.

merb 10y ago
Code reuse is fine, but taking that over the top is not fine. The same with Java some things actually do too much. Especially dealing with many external dependencies is a real pain. A really big one.
- ilaksh 10y ago
  I found the many dependencies to be a pain very early on when I was not specifying versions in my package.json. Now I use npm i --save which gets versions so.. it really is not a pain to have many dependencies with npm. In Java and many other systems, yes, it is very painful. npm is different.
AgentME 10y ago
> I have never had a package break between dev and prod when using versions like ^x.x.x. I've run into it. Dependencies get bugs sometimes. Easy to pin versions with npm's shrinkwrap though. >The author advocates copy-pasting code rather than including small modules. To me this is very misguided. Yeah, I'm surprised how often people suggest this is the obvious solution as if it's not without its own problems. One of the things I dread most about working on a new codebase is finding an ancient version of some library code copy-pasted into the codebase with the inevitable unlabeled (and probably not high-quality) customizations made to it. Trying to figure out whether the library's documentation still applies, whether it still passes its original upstream tests with the customizations, or whether the copy can be safely replaced with a newer upstream version is painful.

AgentME 10y ago

>but we all remember the left-pad incident, right? It proved that having lots of dependencies might actually not be the best thing. npm no longer allows dependencies to be unpublished after they've been up for 24 hours. The issue that happened with left-pad was solved. It's possible for someone to instead publish a new non-functional version of a package, but I don't think that's unique to npm, and that won't affect people who have pinned to a specific version of the package.

thinkMOAR 10y ago

At first i thought you made a typo with 1.65 million "Grab a cup of coffee and enjoy the deep dive into the NPM ecosystem." Now taking my last sip of the coffee, i'm happy never to have spend a single second on node.js. The rapid and simple development via node.js does not weigh up to the bizar number of dependencies and crazy stuff that can originate from that. Interesting to see in your stats there is a decline in development, at least of new modules. Either the ideas people can come up with are drying up, or i'd be inclined to say (interest of) nodeJS development is in decline?