7 ms·

Why does everything in node have to have all the dependencies in the world? ``` @sindresorhus/is JSONStream ansi-regex ansi-styles archive-type argparse asn1 a

by dastx 7y ago

Why does everything in node have to have all the dependencies in the world? ``` @sindresorhus/is JSONStream ansi-regex ansi-styles archive-type argparse asn1 async balanced-match base64-js bcrypt-pbkdf bl bluebird brace-expansion buffer buffer-alloc buffer-alloc-unsafe buffer-crc32 buffer-fill buffer-from cacheable-request camelcase caw chalk chownr cliui clone-response color-convert color-name commander concat-map concat-stream config-chain content-disposition core-util-is cross-spawn debug decamelize decode-uri-component decompress decompress-response decompress-tar decompress-tarbz2 decompress-targz decompress-unzip docker-modem dockerode download duplexer3 emoji-regex end-of-stream escape-string-regexp esprima execa ext-list ext-name fd-slicer file-type filename-reserved-regex filenamify find-up from2 fs-constants fs-extra fs-minipass fs.realpath get-caller-file get-proxy get-stream glob got graceful-fs graceful-readlink has-flag has-symbol-support-x has-to-string-tag-x hasbin http-cache-semantics ieee754 inflight inherits ini into-stream invert-kv ip is-fullwidth-code-point is-natural-number is-object is-plain-obj is-port-available is-retry-allowed is-stream isarray isexe isurl js-yaml json-buffer jsonfile jsonparse keyv lcid locate-path lodash lowercase-keys make-dir map-age-cleaner md5-file mem mime-db mimic-fn mimic-response minimatch minimist minipass minizlib mkdirp ms mustache nice-try node-virtualbox normalize-url npm-conf npm-run-path object-assign once os-locale p-cancelable p-defer p-event p-finally p-is-promise p-limit p-locate p-timeout p-try path-exists path-is-absolute path-key pend pify pinkie pinkie-promise prepend-http process-nextick-args progress proto-list pump query-string readable-stream require-directory require-main-filename responselike safe-buffer safer-buffer scp2 seek-bzip semver set-blocking shebang-command shebang-regex signal-exit simple-git sort-keys sort-keys-length split-ca sprintf-js ssh2 ssh2-streams streamsearch strict-uri-encode string-width string_decoder strip-ansi strip-dirs strip-eof strip-outer sudo-prompt supports-color tar tar-fs tar-stream through timed-out to-buffer trim-repeated tunnel-agent tweetnacl typedarray unbzip2-stream universalify url-parse-lax url-to-options util-deprecate uuid which which-module wrap-ansi wrappy xtend y18n yallist yargs yargs-parser yauzl ```

montecarl 7y ago
This is what you get when you combine a really easy to use package management ecosystem and the standard library is really bad. You don't see this in Python for example, because its not so easy (its getting better though) to create packages and because its standard library is massive.
- krapp 7y ago
  Javascript's standard library isn't bad, it's perfectly adequate for its intended use case - acting as a lightweight scripting language for the web, within a browser. Javascript as a system or backend language was a mistake, but there's no putting that genie back into the bottle.
  - scrollaway 7y ago
    You're confusing JS and NodeJS. NodeJS has more stdlib stuff; opening files, for example.
    - cookiecaper 7y ago
      Virtually everything is a dependency in JavaScript, whether you're using node.js or not. One need look no further than your citation of `open()` as the proof that the nodejs environment "has more stdlib stuff". If the operating environment can run in a context where file IO happens, a native file open function is about the most basic baseline there is. It's absolutely not a good look for "well, we have `open()`!" to be the poster child of the JS stdlib. After `left-pad` and `flatmap-stream`, attempts to justify the state of affairs in the JS ecosystem are patent absurdities. They show starkly that the platform vendor needs to offer a reasonably-robust basic toolkit, and that cultures of "every function should be published as a library!" are a massive risk factor. A good first-party standard library should be considered a security requirement for every application. Dependencies should be brought in with care and attention, not in a massive indiscriminate orgy of nested modules spraying every function into its independent own library and resulting in every node.js application requiring its own 500MB+ folder of libraries to even start. Think it doesn't get worse? People are now using node.js to distribute "business cards". Arbitrary JavaScript execution on your local user account. Has science gone too far?! Someone at npmjs.com sure has by allowing this kind of thing. [0] The technical world is crumbling. Who can fix it? [0] https://blog.bitsrc.io/malicious-npm-development-kit-a02401e6537e https://blog.bitsrc.io/malicious-npm-development-kit-a02401e...
      - untog 7y ago
ManuelKiessling 7y ago
Yesterday a friend of mine told me about how they got a Node.js application from a vendor that was about 3 MiB, and after running npm install, it was over 1 GiB. I half-jokingly said that Node apps are the new ZIP bombs.
- hn23 7y ago
  good luck to your friend with repeatable installations :P
  - JustSomeNobody 7y ago
    If you need repeatable installations, wouldn't node be the wrong tool? I mean, you'd have to freeze everything yourself and then those libraries become _your_ problem. Ugh. That's a hell nobody wants.
    - quickthrower2 7y ago
      You’d just have a local cache, you can pull updates from authors if needed.
    - chrisbroadfoot 7y ago
      To be pedantic, it's "wouldn't npm be the wrong tool" (it isn't, necessarily, I believe lockfiles provide you with reproducible builds) Vendoring/copying them is another way to achieve this (and means you don't need to depend on npm or its lockfiles). Regardless, those libraries are your problem whether you vendor/copy them or not. Read more: https://research.swtch.com/deps https://research.swtch.com/deps
  - news_hacker 7y ago
    "If you wish to make apple pie from scratch, you must first create the universe" - Carl Sagan the npm ecosystem takes this quite literally, for better or worse.
    - stcredzero 7y ago
root_axis 7y ago
Is it really that strange? The transitive dependency graph for my iOS and Android apps are similarly complex.
ricardobeat 7y ago
These are the first-level dependencies: chalk: log formatting dockerode: docker API download: file download? fs-extra: functions like mkdirp, emptyDir hasbin: check if bin exists in PATH js-yaml": yaml parser mustache: templating node-virtualbox: VB API progress: terminal progress bar simple-git: Git API sudo-prompt tar uuid It looks reasonable at first sight. Of all these, fs-extra is the only one you can argue should be part of the standard lib. A lot of the other dependencies are simply cruft, for example, `safe-buffer, `safer-buffer`, `buffer-alloc`, `buffer-alloc-unsafe` all patch the same issue and haven't been necessary since ~2016 / node 6.0. Same for `sort-keys`, object key sort order was baked into the ES2015 spec and has been the default behaviour since the beginning of the decade. As mentioned by other commenters, this is a result of an extremely easy-to-use packaging system, coupled with a culture of sharing and reusability. The dark side of it is (justified) laziness and reinvention of the wheel - the core functionality of many of these modules can be written in one or two lines of code, but it is indeed faster and safer to just import something that exists and has been tested. The vast number of choices means it's hard to find standard solutions, and this also encourages developers to create their own 'improved' version of everything, in a self-reinforcing loop. The language itself has been in constant change, meaning new flavours of previously stable modules pop up to support new patterns (promises, generators, await, classes, etc etc). Then you get egos, marketing and corporate sponsorship added to the mix :)
- tylerl 7y ago
  This is exactly why we don't allow our employees to use Node.js for company software development, even though in theory a fair chunk of the runtime is our own software that we maintain. But the npm ecosystem has a really awful signal-to-dogturd ratio, and developers appear to put very little effort into critically analyzing their dependencies. The topic has come up, but I'm generally against it; sure, we could spend all the resources on doing all the filtering and analysis and change management that it would take to establish a sane package base that we could officially support for internal development work... But why? We do it for other languages, but Node has a lot of "my first programming language" bs going on in the ecosystem. I strongly suspect that allowing Node development would be a net negative for our company.
deleted 7y ago
[deleted]

by dastx 7y ago

montecarl 7y ago

This is what you get when you combine a really easy to use package management ecosystem and the standard library is really bad. You don't see this in Python for example, because its not so easy (its getting better though) to create packages and because its standard library is massive.

krapp 7y ago
Javascript's standard library isn't bad, it's perfectly adequate for its intended use case - acting as a lightweight scripting language for the web, within a browser. Javascript as a system or backend language was a mistake, but there's no putting that genie back into the bottle.
- scrollaway 7y ago
  You're confusing JS and NodeJS. NodeJS has more stdlib stuff; opening files, for example.
  - cookiecaper 7y ago
    Virtually everything is a dependency in JavaScript, whether you're using node.js or not. One need look no further than your citation of `open()` as the proof that the nodejs environment "has more stdlib stuff". If the operating environment can run in a context where file IO happens, a native file open function is about the most basic baseline there is. It's absolutely not a good look for "well, we have `open()`!" to be the poster child of the JS stdlib. After `left-pad` and `flatmap-stream`, attempts to justify the state of affairs in the JS ecosystem are patent absurdities. They show starkly that the platform vendor needs to offer a reasonably-robust basic toolkit, and that cultures of "every function should be published as a library!" are a massive risk factor. A good first-party standard library should be considered a security requirement for every application. Dependencies should be brought in with care and attention, not in a massive indiscriminate orgy of nested modules spraying every function into its independent own library and resulting in every node.js application requiring its own 500MB+ folder of libraries to even start. Think it doesn't get worse? People are now using node.js to distribute "business cards". Arbitrary JavaScript execution on your local user account. Has science gone too far?! Someone at npmjs.com sure has by allowing this kind of thing. [0] The technical world is crumbling. Who can fix it? [0] https://blog.bitsrc.io/malicious-npm-development-kit-a02401e6537e https://blog.bitsrc.io/malicious-npm-development-kit-a02401e...
    - untog 7y ago

ManuelKiessling 7y ago

Yesterday a friend of mine told me about how they got a Node.js application from a vendor that was about 3 MiB, and after running npm install, it was over 1 GiB. I half-jokingly said that Node apps are the new ZIP bombs.

hn23 7y ago
good luck to your friend with repeatable installations :P
- JustSomeNobody 7y ago
  If you need repeatable installations, wouldn't node be the wrong tool? I mean, you'd have to freeze everything yourself and then those libraries become _your_ problem. Ugh. That's a hell nobody wants.
  - quickthrower2 7y ago
    You’d just have a local cache, you can pull updates from authors if needed.
  - chrisbroadfoot 7y ago
    To be pedantic, it's "wouldn't npm be the wrong tool" (it isn't, necessarily, I believe lockfiles provide you with reproducible builds) Vendoring/copying them is another way to achieve this (and means you don't need to depend on npm or its lockfiles). Regardless, those libraries are your problem whether you vendor/copy them or not. Read more: https://research.swtch.com/deps https://research.swtch.com/deps
- news_hacker 7y ago
  "If you wish to make apple pie from scratch, you must first create the universe" - Carl Sagan the npm ecosystem takes this quite literally, for better or worse.
  - stcredzero 7y ago

root_axis 7y ago

Is it really that strange? The transitive dependency graph for my iOS and Android apps are similarly complex.

ricardobeat 7y ago

These are the first-level dependencies: chalk: log formatting dockerode: docker API download: file download? fs-extra: functions like mkdirp, emptyDir hasbin: check if bin exists in PATH js-yaml": yaml parser mustache: templating node-virtualbox: VB API progress: terminal progress bar simple-git: Git API sudo-prompt tar uuid It looks reasonable at first sight. Of all these, fs-extra is the only one you can argue should be part of the standard lib. A lot of the other dependencies are simply cruft, for example, `safe-buffer, `safer-buffer`, `buffer-alloc`, `buffer-alloc-unsafe` all patch the same issue and haven't been necessary since ~2016 / node 6.0. Same for `sort-keys`, object key sort order was baked into the ES2015 spec and has been the default behaviour since the beginning of the decade. As mentioned by other commenters, this is a result of an extremely easy-to-use packaging system, coupled with a culture of sharing and reusability. The dark side of it is (justified) laziness and reinvention of the wheel - the core functionality of many of these modules can be written in one or two lines of code, but it is indeed faster and safer to just import something that exists and has been tested. The vast number of choices means it's hard to find standard solutions, and this also encourages developers to create their own 'improved' version of everything, in a self-reinforcing loop. The language itself has been in constant change, meaning new flavours of previously stable modules pop up to support new patterns (promises, generators, await, classes, etc etc). Then you get egos, marketing and corporate sponsorship added to the mix :)

tylerl 7y ago
This is exactly why we don't allow our employees to use Node.js for company software development, even though in theory a fair chunk of the runtime is our own software that we maintain. But the npm ecosystem has a really awful signal-to-dogturd ratio, and developers appear to put very little effort into critically analyzing their dependencies. The topic has come up, but I'm generally against it; sure, we could spend all the resources on doing all the filtering and analysis and change management that it would take to establish a sane package base that we could officially support for internal development work... But why? We do it for other languages, but Node has a lot of "my first programming language" bs going on in the ecosystem. I strongly suspect that allowing Node development would be a net negative for our company.

deleted 7y ago

[deleted]