6 ms·

https://en.wikipedia.org/wiki/Unix_philosophy https://en.wikipedia.org/wiki/Unix_philosophy

maxaf 11y ago
Taking an idea to the logical extreme is an effective means of invalidating said idea. How many UNIX utilities are 17 silly lines long? A bit of code duplication would go a long way towards bringing sanity to JS land.
- joshmanders 11y ago
  Quite a bit are pretty small.
  - cballard 11y ago
    For example: http://www.scs.stanford.edu/histar/src/pkg/echo/echo.c http://www.scs.stanford.edu/histar/src/pkg/echo/echo.c
    - akkartik 11y ago
      Erm, that's not a very good example. You're pointing out some ancient source file from back when unix had no package management. These days echo.c is part of coreutils, a large package which is economic to manage dependencies for at scale. It's interesting to think about the distinction between promiscuous dependencies (as pioneered by Gemfile) and the Unix way. I like the latter and loathe the former, but maybe I'm wrong. Can someone think of a better example? Is there a modern dpkg/rpm/yum package with <50 LoC of payload? Edit: Incidentally, I just checked and echo.c in coreutils 8.25 weighs in at 194 LoC (excluding comments, empty lines and just curly braces). And that doesn't even include other files that are needed to build echo. Back in 2013 I did a similar analysis for cat, and found that it required 36 thousand lines of code (just .c and .h files). It's my favorite example of the rot at Linux's core. There's many complaints you can make about Unix package management, but 'too many tiny dependencies' seems unlikely to be one of them.
      - maxaf 11y ago
        The main point in support of your argument is the fact that Unix utils are bundled in larger packages instead of shipping in single-command packages. Think of Fileutils, Shellutils, and Textutils... which got combined to form Coreutils! Ha! That leaves only Diffutils, for the time being anyway.
        deleted 11y ago
        [deleted]
        akkartik 11y ago
        But none of Fileutils, Shellutils and Textutils was ever as tiny as many npm or gem modules. I thought how commands are bundled into packages was the entirety of what we were discussing. That was my interpretation of larkinrichards's comment (way) up above. Packages are the unit of installation, not use. Packages are all we're arguing about. My position: don't put words in God's mouth :) The unix way is commands that do one thing and do it well. But the unix way is silent on the right way to disseminate said commands. There you're on your own.
        larkinrichards 11y ago
        Underscore/lodash is a great bundle of functions that do simple things well. And it's a tiny enough library that there is really no need to split it into 270 modules. I support packages of utility functions. Distributing them individually is a waste of resources when you have tree shaking. I trust a dependency on lodash. I don't trust a dependency on a single 17 line function.
        joshmanders 11y ago
        While I agree with you, tree shaking is relatively new in the JavaScript world thanks to Webpack 2 and Rollup.js before that, if you had a dependency you brought it's whole lib into your project whether you used one method or all of them. So just including Lodash wasn't an option for people who cared about loading times for their users. A 17 line module was.
- swang 11y ago
  yes, let's blow up the entire concept that's worked fine for the ~5 years of node's existence because one dude did something extreme.
  - dham 11y ago
    Woah woah. Hold on there. Lets not throw around strong words like "worked", "concept", "entire", "fine", "did" when discussing NPM.
    - Maarten88 11y ago
      This. I'm on Windows. Npm never worked for me, like not at all. Npm has cost me lots of wasted time, I know I should be thankful for this free product, but, but, Grrrrr...
      - joshmanders 11y ago
        Windows has always been a problem for Nodeland, thankfully Microsoft is working on making that better.
  - bryanrasmussen 11y ago
    is unpublishing a module extreme?
- smsm42 11y ago
  I wonder how long /usr/bin/true source is.
  - Havvy 11y ago
    On Linux, it's 30ish lines, with half of those there to make false.c able to reuse some code. (I know, it's stupid.) In OpenBSD, it's 3 LoC IIRC.
    - dalke 11y ago
      On at least one OS I worked with, it was 0 bytes long because /bin/sh on an empty file returns true. (I think that was IRIX 4.x.) OTOH, that's slower than the executable from a 3 LoC do-nothing program.
  - cjs2 11y ago
    Longer than one might think... http://git.savannah.gnu.org/cgit/coreutils.git/tree/src/true.c http://git.savannah.gnu.org/cgit/coreutils.git/tree/src/true...
Sanddancer 11y ago
Which is useful for binaries. Libraries, there tends to be a lot more aggregation. If this wasn't the case, you'd see libc spread over dozens and dozens of libraries like libmalloc librandom libstring libarithmetic libsocket libprocess libfilesystem libafilesystem (hey, async ops have a different api) libtime, etc.
- cballard 11y ago
  Why would this be a bad thing? I don't need a random number generator if I just want to allocate memory.
  - Sanddancer 11y ago
    Because a lot of little libraries makes namespaces more complicated, makes security auditing more difficult, makes troubleshooting more difficult as you have to start digging through compatibility of a ton more libraries, makes loading slower, because you have to fopen() a ton more files and parse their contents, etc. Add on top of that those little libraries needing other, probably redundant little libraries, and you can start to see how this can turn the structure of your code into mush really quickly. At this point, optimizing runtimes to only include necessary functions, and optimizing files to only include necessary functions are both things that are pretty much a solved problem. For example, azer has a random-color library, and an rng library. Having both of those as azer-random or something means that someone automatically gets all the dependencies, without having to make many requests to the server. This makes build times shorter and a lot easier. Sometimes, in order to best optimize for small, you have to have some parts that are big. Libraries tend to be one of those things where a few good, big libraries lead to a smaller footprint than many tiny libraries.
    - dylan-m 11y ago
      Not to mention maintenance. Low-hanging fruit here: left-pad, depended on by fricking everyone, <em>is owned by one guy</em>. That is not how you build a serious utility library that people should actually be using in real products. (You also probably shouldn't license it under the WTFPL, but that's another thing). When you aggregate under a common banner, you get maintainers for free and your project might have a bus factor better than 1.
bye_amzn 11y ago
Let it suffice to say that Linux distributions (and some closed OSes) try very hard to prevent package fragmentation. Experience has shown many times that excessive fragmentation leads to proliferation of unmaintained libraries.
ubernostrum 11y ago
In this case the npm ecosystem is providing more of a surrogate standard library. Imagine if there were no libc, for example, and so people had to reimplement all those functions; would you really want one package per function because of how "Unix philosophy" it would be? This is where the JavaScript ecosystem is right now -- JS doesn't have the kind of robust standard library other languages take for granted, so you end up with a lot of these things that look like they should be stdlib functions, but are instead third-party packages the entire world has to depend on for a usable programming environment.
- esailija 11y ago
  I don't understand the standard library argument at all. Standard library is always limited, there is always something that's not included - then what? This has nothing to do with the JavaScript standard library but the insanity of NPM. There are modules that have even more dependents than `pad-left` despite being included in the standard library (e.g. https://www.npmjs.com/package/date-now https://www.npmjs.com/package/date-now, https://www.npmjs.com/package/is-positive https://www.npmjs.com/package/is-positive, https://www.npmjs.com/package/is-lower-case https://www.npmjs.com/package/is-lower-case)
  - ubernostrum 11y ago
    The argument is that things like formatting a string -- including padding it -- should not be add-ons, they should be built into the core distribution you get as part of the language.
    - esailija 11y ago
      I didn't say it shouldn't be part of standard library, I am saying that there is always something that isn't part of the standard library. My point is: if this module was something that shouldn't be part of standard library, what argument would you then use?
- yoklov 11y ago
  I think you're assuming libc is more robust and useful than it actually is. Libc is extremely minimal (and is full of pitfalls as well). JS has an extremely large and robust standard library in comparison.

maxaf 11y ago

Taking an idea to the logical extreme is an effective means of invalidating said idea. How many UNIX utilities are 17 silly lines long? A bit of code duplication would go a long way towards bringing sanity to JS land.

joshmanders 11y ago
Quite a bit are pretty small.
- cballard 11y ago
  For example: http://www.scs.stanford.edu/histar/src/pkg/echo/echo.c http://www.scs.stanford.edu/histar/src/pkg/echo/echo.c
  - akkartik 11y ago
    Erm, that's not a very good example. You're pointing out some ancient source file from back when unix had no package management. These days echo.c is part of coreutils, a large package which is economic to manage dependencies for at scale. It's interesting to think about the distinction between promiscuous dependencies (as pioneered by Gemfile) and the Unix way. I like the latter and loathe the former, but maybe I'm wrong. Can someone think of a better example? Is there a modern dpkg/rpm/yum package with <50 LoC of payload? Edit: Incidentally, I just checked and echo.c in coreutils 8.25 weighs in at 194 LoC (excluding comments, empty lines and just curly braces). And that doesn't even include other files that are needed to build echo. Back in 2013 I did a similar analysis for cat, and found that it required 36 thousand lines of code (just .c and .h files). It's my favorite example of the rot at Linux's core. There's many complaints you can make about Unix package management, but 'too many tiny dependencies' seems unlikely to be one of them.
    - maxaf 11y ago
      The main point in support of your argument is the fact that Unix utils are bundled in larger packages instead of shipping in single-command packages. Think of Fileutils, Shellutils, and Textutils... which got combined to form Coreutils! Ha! That leaves only Diffutils, for the time being anyway.
      - deleted 11y ago
        [deleted]
      - akkartik 11y ago
        But none of Fileutils, Shellutils and Textutils was ever as tiny as many npm or gem modules. I thought how commands are bundled into packages was the entirety of what we were discussing. That was my interpretation of larkinrichards's comment (way) up above. Packages are the unit of installation, not use. Packages are all we're arguing about. My position: don't put words in God's mouth :) The unix way is commands that do one thing and do it well. But the unix way is silent on the right way to disseminate said commands. There you're on your own.
        larkinrichards 11y ago
        Underscore/lodash is a great bundle of functions that do simple things well. And it's a tiny enough library that there is really no need to split it into 270 modules. I support packages of utility functions. Distributing them individually is a waste of resources when you have tree shaking. I trust a dependency on lodash. I don't trust a dependency on a single 17 line function.
        joshmanders 11y ago
        While I agree with you, tree shaking is relatively new in the JavaScript world thanks to Webpack 2 and Rollup.js before that, if you had a dependency you brought it's whole lib into your project whether you used one method or all of them. So just including Lodash wasn't an option for people who cared about loading times for their users. A 17 line module was.
swang 11y ago
yes, let's blow up the entire concept that's worked fine for the ~5 years of node's existence because one dude did something extreme.
- dham 11y ago
  Woah woah. Hold on there. Lets not throw around strong words like "worked", "concept", "entire", "fine", "did" when discussing NPM.
  - Maarten88 11y ago
    This. I'm on Windows. Npm never worked for me, like not at all. Npm has cost me lots of wasted time, I know I should be thankful for this free product, but, but, Grrrrr...
    - joshmanders 11y ago
      Windows has always been a problem for Nodeland, thankfully Microsoft is working on making that better.
- bryanrasmussen 11y ago
  is unpublishing a module extreme?
smsm42 11y ago
I wonder how long /usr/bin/true source is.
- Havvy 11y ago
  On Linux, it's 30ish lines, with half of those there to make false.c able to reuse some code. (I know, it's stupid.) In OpenBSD, it's 3 LoC IIRC.
  - dalke 11y ago
    On at least one OS I worked with, it was 0 bytes long because /bin/sh on an empty file returns true. (I think that was IRIX 4.x.) OTOH, that's slower than the executable from a 3 LoC do-nothing program.
- cjs2 11y ago
  Longer than one might think... http://git.savannah.gnu.org/cgit/coreutils.git/tree/src/true.c http://git.savannah.gnu.org/cgit/coreutils.git/tree/src/true...

Sanddancer 11y ago

Which is useful for binaries. Libraries, there tends to be a lot more aggregation. If this wasn't the case, you'd see libc spread over dozens and dozens of libraries like libmalloc librandom libstring libarithmetic libsocket libprocess libfilesystem libafilesystem (hey, async ops have a different api) libtime, etc.

cballard 11y ago
Why would this be a bad thing? I don't need a random number generator if I just want to allocate memory.
- Sanddancer 11y ago
  Because a lot of little libraries makes namespaces more complicated, makes security auditing more difficult, makes troubleshooting more difficult as you have to start digging through compatibility of a ton more libraries, makes loading slower, because you have to fopen() a ton more files and parse their contents, etc. Add on top of that those little libraries needing other, probably redundant little libraries, and you can start to see how this can turn the structure of your code into mush really quickly. At this point, optimizing runtimes to only include necessary functions, and optimizing files to only include necessary functions are both things that are pretty much a solved problem. For example, azer has a random-color library, and an rng library. Having both of those as azer-random or something means that someone automatically gets all the dependencies, without having to make many requests to the server. This makes build times shorter and a lot easier. Sometimes, in order to best optimize for small, you have to have some parts that are big. Libraries tend to be one of those things where a few good, big libraries lead to a smaller footprint than many tiny libraries.
  - dylan-m 11y ago
    Not to mention maintenance. Low-hanging fruit here: left-pad, depended on by fricking everyone, <em>is owned by one guy</em>. That is not how you build a serious utility library that people should actually be using in real products. (You also probably shouldn't license it under the WTFPL, but that's another thing). When you aggregate under a common banner, you get maintainers for free and your project might have a bus factor better than 1.

bye_amzn 11y ago

Let it suffice to say that Linux distributions (and some closed OSes) try very hard to prevent package fragmentation. Experience has shown many times that excessive fragmentation leads to proliferation of unmaintained libraries.

ubernostrum 11y ago

In this case the npm ecosystem is providing more of a surrogate standard library. Imagine if there were no libc, for example, and so people had to reimplement all those functions; would you really want one package per function because of how "Unix philosophy" it would be? This is where the JavaScript ecosystem is right now -- JS doesn't have the kind of robust standard library other languages take for granted, so you end up with a lot of these things that look like they should be stdlib functions, but are instead third-party packages the entire world has to depend on for a usable programming environment.

esailija 11y ago
I don't understand the standard library argument at all. Standard library is always limited, there is always something that's not included - then what? This has nothing to do with the JavaScript standard library but the insanity of NPM. There are modules that have even more dependents than `pad-left` despite being included in the standard library (e.g. https://www.npmjs.com/package/date-now https://www.npmjs.com/package/date-now, https://www.npmjs.com/package/is-positive https://www.npmjs.com/package/is-positive, https://www.npmjs.com/package/is-lower-case https://www.npmjs.com/package/is-lower-case)
- ubernostrum 11y ago
  The argument is that things like formatting a string -- including padding it -- should not be add-ons, they should be built into the core distribution you get as part of the language.
  - esailija 11y ago
    I didn't say it shouldn't be part of standard library, I am saying that there is always something that isn't part of the standard library. My point is: if this module was something that shouldn't be part of standard library, what argument would you then use?
yoklov 11y ago
I think you're assuming libc is more robust and useful than it actually is. Libc is extremely minimal (and is full of pitfalls as well). JS has an extremely large and robust standard library in comparison.