6 ms·

> I don't think it's necessarily "most programming languages". I know Perl, and probably Python and Ruby, have very large ecosystems of small modules. I suspect

> I don't think it's necessarily "most programming languages". I know Perl, and probably Python and Ruby, have very large ecosystems of small modules. I suspect it's more along the lines of static vs dynamic, or compiled vs interpreted. The Perl, Python, and Ruby communities might have a lot of "small-ish" modules, compared to something like Boost in C++, but the Node.js community has really taken it to a new extreme with an explosion of "one-liner" modules like the absurd user-home https://github.com/sindresorhus/user-home https://github.com/sindresorhus/user-home The rallying cry of this movement is "Modules are Cheap in Node.js!". Incidents like this help demonstrate that explosive growth of your dependency graph isn't ever cheap, in any language.

kbenson 11y ago
If that's the case, that is rather excessive. I hope there's at least a movement going on to shift towards using utility modules that combine a few tens of these one-liners into a cohesive namespace/class/object where complementary. Keeping track of many, many very small modules does indeed impose a cost, which it looks like some NPM users have gotten around by doing the absolute minimum to keep track. This is the result.
- brudgers 11y ago
  There are modules that combine modules. Alas due to the beauty and power of combinatorics, there will inevitably be more compound modules than atomic ones. If there are not already.
  - kbenson 11y ago
    Yeah, I meant combine less in the "concatenate into one large new module automatically" and more "someone as put out what is rapidly becoming the de-facto standard for all OS high level actions in a comprehensive single module." I don't see automatic combination of lots of small modules actually making this situation better in any way. Quite the opposite, actually...
thrillgore 11y ago
I continue to be amazed by how far some people will take abstraction for the most menial of components.
mshenfield 11y ago
I agree that there is a definite cost of dependencies. I don't agree with the implication that these small modules are frivolous - doing one thing and doing it well is a time honored Unix principle. It saves developers from having to roll their own dependency, and gives them free wins if the package is well maintained and widely used. Edge cases are often identified and accounted for, even if the responsibility of the package is tiny. The example you gave was a poor choice - read further down https://github.com/sindresorhus/user-home#why-not-just-use-the-os-home-module https://github.com/sindresorhus/user-home#why-not-just-use-t.... It exists to avoid exactly the problem that sparked the conversation around npm today.
- msbarnett 11y ago
  We disagree on the utility of adding an entire dependency for simple rename of os-homedir. But there are myriad other equally ridiculous packages. is-positive, for instance: https://github.com/kevva/is-positive https://github.com/kevva/is-positive. Abstractions are good when they hide and generalize complexity. These modules merely conceal mundanity. > Edge cases are often identified and accounted for, even if the responsibility of the package is tiny. In my observation, the opposite is true -- most of these trivial packages punt on edge cases. is-positive makes no attempt to handle any. https://www.npmjs.com/package/average https://www.npmjs.com/package/average implements an utterly naive calculation of an array's mean that is numerically unstable in the presence of JS's everything-is-a-float semantics. Hiding tiny, naive implementations behind dependency imports actively harms understanding of the implementation details and failure modes of the code being relied on, in addition to making your build much more fragile by encouraging you to rely on external dependencies that can and will vanish from the Internet at inconvenient times. It's a bad trade-off from every angle. Edit: this blog post (currently on the front page!) does a really good job of capturing my utter bafflement at how completely crazy the node community has gone with this nonsense http://www.haneycodes.net/npm-left-pad-have-we-forgotten-how-to-program/ http://www.haneycodes.net/npm-left-pad-have-we-forgotten-how... Edit 2: and none of this even touches on the complete insanity that is npm running arbitrary code in each dependency at install time. Having 2,000 one-liner trivial dependencies amounts to inviting 2,000 strangers to run arbitrary code on your build server. The node.js community appears to be a pack of complete amateurs.
sotojuan 11y ago
How is that absurd? I've written Node apps where I needed to get the user's home path, crossplatform. Instead of memorizing this: https://github.com/sindresorhus/os-homedir/blob/master/index.js https://github.com/sindresorhus/os-homedir/blob/master/index... I'd rather `npm install` it and require it. Of course, now Node has this: https://nodejs.org/api/os.html#os_os_homedir https://nodejs.org/api/os.html#os_os_homedir, but it didn't and you get the point. The problem was with npm and allowing unpublishing/hijacking, not the Node community.
- kbenson 11y ago
  It's absurd because the obvious choice from the beginning should have been a library of abstracted OS concepts. Even if it starts as a single method/function for determining the homedir, making sure it can easily handle more abstractions is the obvious choice. People wanting homedir() will likely also want a way to separate a path from a file, know how to separate path components based on platform ('/' vs '\'), and a few other obvious things (see the os module you referenced). A low barrier of entry for submitting modules is generally a good thing, but if it's too low, you end up filling your ecosystem with crap.
  - rat87 11y ago
    Plus hopefully the functions in the module would be more consistent(with each other).
- merb 11y ago
  It's absurd cause the repo contains files to handle git, npm and other stuff that have more line's written than the actual code.

kbenson 11y ago

If that's the case, that is rather excessive. I hope there's at least a movement going on to shift towards using utility modules that combine a few tens of these one-liners into a cohesive namespace/class/object where complementary. Keeping track of many, many very small modules does indeed impose a cost, which it looks like some NPM users have gotten around by doing the absolute minimum to keep track. This is the result.

brudgers 11y ago
There are modules that combine modules. Alas due to the beauty and power of combinatorics, there will inevitably be more compound modules than atomic ones. If there are not already.
- kbenson 11y ago
  Yeah, I meant combine less in the "concatenate into one large new module automatically" and more "someone as put out what is rapidly becoming the de-facto standard for all OS high level actions in a comprehensive single module." I don't see automatic combination of lots of small modules actually making this situation better in any way. Quite the opposite, actually...

thrillgore 11y ago

I continue to be amazed by how far some people will take abstraction for the most menial of components.

mshenfield 11y ago

I agree that there is a definite cost of dependencies. I don't agree with the implication that these small modules are frivolous - doing one thing and doing it well is a time honored Unix principle. It saves developers from having to roll their own dependency, and gives them free wins if the package is well maintained and widely used. Edge cases are often identified and accounted for, even if the responsibility of the package is tiny. The example you gave was a poor choice - read further down https://github.com/sindresorhus/user-home#why-not-just-use-the-os-home-module https://github.com/sindresorhus/user-home#why-not-just-use-t.... It exists to avoid exactly the problem that sparked the conversation around npm today.

msbarnett 11y ago
We disagree on the utility of adding an entire dependency for simple rename of os-homedir. But there are myriad other equally ridiculous packages. is-positive, for instance: https://github.com/kevva/is-positive https://github.com/kevva/is-positive. Abstractions are good when they hide and generalize complexity. These modules merely conceal mundanity. > Edge cases are often identified and accounted for, even if the responsibility of the package is tiny. In my observation, the opposite is true -- most of these trivial packages punt on edge cases. is-positive makes no attempt to handle any. https://www.npmjs.com/package/average https://www.npmjs.com/package/average implements an utterly naive calculation of an array's mean that is numerically unstable in the presence of JS's everything-is-a-float semantics. Hiding tiny, naive implementations behind dependency imports actively harms understanding of the implementation details and failure modes of the code being relied on, in addition to making your build much more fragile by encouraging you to rely on external dependencies that can and will vanish from the Internet at inconvenient times. It's a bad trade-off from every angle. Edit: this blog post (currently on the front page!) does a really good job of capturing my utter bafflement at how completely crazy the node community has gone with this nonsense http://www.haneycodes.net/npm-left-pad-have-we-forgotten-how-to-program/ http://www.haneycodes.net/npm-left-pad-have-we-forgotten-how... Edit 2: and none of this even touches on the complete insanity that is npm running arbitrary code in each dependency at install time. Having 2,000 one-liner trivial dependencies amounts to inviting 2,000 strangers to run arbitrary code on your build server. The node.js community appears to be a pack of complete amateurs.

sotojuan 11y ago

How is that absurd? I've written Node apps where I needed to get the user's home path, crossplatform. Instead of memorizing this: https://github.com/sindresorhus/os-homedir/blob/master/index.js https://github.com/sindresorhus/os-homedir/blob/master/index... I'd rather `npm install` it and require it. Of course, now Node has this: https://nodejs.org/api/os.html#os_os_homedir https://nodejs.org/api/os.html#os_os_homedir, but it didn't and you get the point. The problem was with npm and allowing unpublishing/hijacking, not the Node community.

kbenson 11y ago
It's absurd because the obvious choice from the beginning should have been a library of abstracted OS concepts. Even if it starts as a single method/function for determining the homedir, making sure it can easily handle more abstractions is the obvious choice. People wanting homedir() will likely also want a way to separate a path from a file, know how to separate path components based on platform ('/' vs '\'), and a few other obvious things (see the os module you referenced). A low barrier of entry for submitting modules is generally a good thing, but if it's too low, you end up filling your ecosystem with crap.
- rat87 11y ago
  Plus hopefully the functions in the module would be more consistent(with each other).
merb 11y ago
It's absurd cause the repo contains files to handle git, npm and other stuff that have more line's written than the actual code.