4 ms·

And a problem which is some 5x worse in npm

marcosdumay 4y ago
5x worse on each level, maybe. That composes quite quickly. Npm packages have really crazy dependency graphs. It's usually on the 1000s of times larger than what you get on most languages, 100s of times larger than python or ruby.
- pitsnatch 4y ago
  Why is this the case?
  - raverbashing 4y ago
    Because JS had a very poor stdlib and attracted people that had little to no experience in best-practices in older languages Hence left-pad. And this pet-peeve of mine https://nodejs.org/docs/latest/api/process.html#processargv https://nodejs.org/docs/latest/api/process.html#processargv (who cares about the node fullpath in argv[0]?)
    - SpeedilyDamage 4y ago
      Why oh why do people cite left-pad like this as an example of package bloat? It’s functionality was so valuable it was included in ECMAScript 2017! The problem with left pad was a security one, not a package bloat one.
      - deleted 4y ago
        [deleted]
      - gfxgirl 4y ago
        No, the problem is bad developers pulling in dependencies for trivial functionally. If there was a `for-loop` npm package bad devs would be pulling it in instead of writing their own for loops. Padding on the left is something if it doesn't exist you write it in a few lines of code yourself. You don't add a package for such trivial functionality.
        SpeedilyDamage 4y ago
        Nope, this is a bad take, parroted without understanding; if it got moved into the std lib, it was probably useful enough. You can even read why in the original proposal if you comb the archives enough (from https://github.com/tc39/proposal-string-pad-start-end https://github.com/tc39/proposal-string-pad-start-end): > It is highly probable that the majority of current string padding implementations are inefficient. Bringing this into the platform will improve performance of the web, and developer productivity as they no longer have to implement these common functions. “It’s too trivial” is not why left-pad was in the news. Go read about it to understand the actual issue: https://www.theregister.com/AMP/2017/07/12/javascript_spec_straps_on_padding/ https://www.theregister.com/AMP/2017/07/12/javascript_spec_s...
        raverbashing 4y ago
        > if it got moved into the std lib, it was probably useful enough Of course it was useful, hence why most non-crappy languages had it in its stdlib from inception pretty much But building a package just to do left-pad is stupid, especially since it can be implemented in a couple of lines
        SpeedilyDamage 4y ago
        You can’t both believe it’s worth putting in the stdlib and not worth importing a library for, if your intention is to be consistent.
        Too 4y ago
        You are neglecting the risk-factor of pulling in libraries from unknown authors on npm vs the stdlib. The package-bloat problem is one of culture, where developers keep neglecting this risk, only seeing the 5 lines of code they save by importing something, without seeing the potential cost and tech debt of having to review, maintain, update and security-monitor this dependency for all future. Nobody thinks leftPad was not a useful function. The question is, was it useful enough to counter all the risks of npm, probably not. In the stdlib there is no such risk.
  - dtgriscom 4y ago
    Here's my theory. Older programming languages force you to think about sub-dependencies: if you decided to use a third-party library, you would have to look at and manually add its requirements to your build system. But, with npm, suddenly it was trivial to include other packages without having to worry about sub-dependencies. The whole tree just magically added itself to your build, and only if you paid attention to the build process would you discover just how many developers you were transitively trusting.
    - marcosdumay 4y ago
      Well, while yes, automatic dependency management is a really relevant reason why things got that bad, it can't be the only reason. Programs in other languages with the same kind of tooling tend to stop at hundreds or at most low-thousands of dependencies. Javascript code often reach tens or low-hundreds of thousands of dependencies. Dependency explosion is bad all around, but JS is exceptionally bad.
  - scotteric 4y ago
    Javascript lacks many basic facilities that other languages have.
- gfxgirl 4y ago
  it's becoming the same in rust. Here's the rust official docs https://github.com/rust-lang/docs.rs https://github.com/rust-lang/docs.rs Follow the build instructions, then you get to `cargo build` you'll see this message Downloaded 448 crates (44.1 MB) 448 crates for a static site generator!?!?!?! WTF!
  - hamishwhc 4y ago
    I think a large chunk of this is the tendency in larger Rust projects to split themselves up into smaller library crates, rather than single crate monoliths: single libraries that you use may depend on 5 different crates all from the same project, that count as separate in the number you’ve quoted. On top of that, C bindings tend to live in their own crate, so while you might depend on the openssl crate, it in turn depends on openssl_sys which contains raw bindings (rather than a nice Rust wrapper). That all said, I think crates.io is still at the initial stage of “tiny crates each doing a single thing” that I think every registry goes through to begin with.
- 5e92cb50239222b 4y ago
  I think there is a trend towards reducing dependency bloat, at least in some circles. For example, the main React package has had this much recursive dependencies throughout the past few major versions: react@15 — 20 deps react@16 — 6 deps react@17 — 4 deps react@18 — 3 deps The latest version of vite (a webpack replacement that's been gaining in popularity) has 14 recursive dependencies. It could be better, but it's still a significant improvement over 300+ package monstrosity you get by combining webpack with everything necessary to reimplement what vite is doing. If you pick your packages carefully (for example, looking for alternatives this one ↓ instead of the much more popular react-router), it's not as bad as it used to be. https://github.com/molefrog/wouter https://github.com/molefrog/wouter

5x worse on each level, maybe. That composes quite quickly. Npm packages have really crazy dependency graphs. It's usually on the 1000s of times larger than what you get on most languages, 100s of times larger than python or ruby.

pitsnatch 4y ago
Why is this the case?
- raverbashing 4y ago
  Because JS had a very poor stdlib and attracted people that had little to no experience in best-practices in older languages Hence left-pad. And this pet-peeve of mine https://nodejs.org/docs/latest/api/process.html#processargv https://nodejs.org/docs/latest/api/process.html#processargv (who cares about the node fullpath in argv[0]?)
  - SpeedilyDamage 4y ago
    Why oh why do people cite left-pad like this as an example of package bloat? It’s functionality was so valuable it was included in ECMAScript 2017! The problem with left pad was a security one, not a package bloat one.
    - deleted 4y ago
      [deleted]
    - gfxgirl 4y ago
      No, the problem is bad developers pulling in dependencies for trivial functionally. If there was a `for-loop` npm package bad devs would be pulling it in instead of writing their own for loops. Padding on the left is something if it doesn't exist you write it in a few lines of code yourself. You don't add a package for such trivial functionality.
      - SpeedilyDamage 4y ago
        Nope, this is a bad take, parroted without understanding; if it got moved into the std lib, it was probably useful enough. You can even read why in the original proposal if you comb the archives enough (from https://github.com/tc39/proposal-string-pad-start-end https://github.com/tc39/proposal-string-pad-start-end): > It is highly probable that the majority of current string padding implementations are inefficient. Bringing this into the platform will improve performance of the web, and developer productivity as they no longer have to implement these common functions. “It’s too trivial” is not why left-pad was in the news. Go read about it to understand the actual issue: https://www.theregister.com/AMP/2017/07/12/javascript_spec_straps_on_padding/ https://www.theregister.com/AMP/2017/07/12/javascript_spec_s...
        raverbashing 4y ago
        > if it got moved into the std lib, it was probably useful enough Of course it was useful, hence why most non-crappy languages had it in its stdlib from inception pretty much But building a package just to do left-pad is stupid, especially since it can be implemented in a couple of lines
        SpeedilyDamage 4y ago
        You can’t both believe it’s worth putting in the stdlib and not worth importing a library for, if your intention is to be consistent.
        Too 4y ago
        You are neglecting the risk-factor of pulling in libraries from unknown authors on npm vs the stdlib. The package-bloat problem is one of culture, where developers keep neglecting this risk, only seeing the 5 lines of code they save by importing something, without seeing the potential cost and tech debt of having to review, maintain, update and security-monitor this dependency for all future. Nobody thinks leftPad was not a useful function. The question is, was it useful enough to counter all the risks of npm, probably not. In the stdlib there is no such risk.
- dtgriscom 4y ago
  Here's my theory. Older programming languages force you to think about sub-dependencies: if you decided to use a third-party library, you would have to look at and manually add its requirements to your build system. But, with npm, suddenly it was trivial to include other packages without having to worry about sub-dependencies. The whole tree just magically added itself to your build, and only if you paid attention to the build process would you discover just how many developers you were transitively trusting.
  - marcosdumay 4y ago
    Well, while yes, automatic dependency management is a really relevant reason why things got that bad, it can't be the only reason. Programs in other languages with the same kind of tooling tend to stop at hundreds or at most low-thousands of dependencies. Javascript code often reach tens or low-hundreds of thousands of dependencies. Dependency explosion is bad all around, but JS is exceptionally bad.
- scotteric 4y ago
  Javascript lacks many basic facilities that other languages have.
gfxgirl 4y ago
it's becoming the same in rust. Here's the rust official docs https://github.com/rust-lang/docs.rs https://github.com/rust-lang/docs.rs Follow the build instructions, then you get to `cargo build` you'll see this message Downloaded 448 crates (44.1 MB) 448 crates for a static site generator!?!?!?! WTF!
- hamishwhc 4y ago
  I think a large chunk of this is the tendency in larger Rust projects to split themselves up into smaller library crates, rather than single crate monoliths: single libraries that you use may depend on 5 different crates all from the same project, that count as separate in the number you’ve quoted. On top of that, C bindings tend to live in their own crate, so while you might depend on the openssl crate, it in turn depends on openssl_sys which contains raw bindings (rather than a nice Rust wrapper). That all said, I think crates.io is still at the initial stage of “tiny crates each doing a single thing” that I think every registry goes through to begin with.
5e92cb50239222b 4y ago
I think there is a trend towards reducing dependency bloat, at least in some circles. For example, the main React package has had this much recursive dependencies throughout the past few major versions: react@15 — 20 deps react@16 — 6 deps react@17 — 4 deps react@18 — 3 deps The latest version of vite (a webpack replacement that's been gaining in popularity) has 14 recursive dependencies. It could be better, but it's still a significant improvement over 300+ package monstrosity you get by combining webpack with everything necessary to reimplement what vite is doing. If you pick your packages carefully (for example, looking for alternatives this one ↓ instead of the much more popular react-router), it's not as bad as it used to be. https://github.com/molefrog/wouter https://github.com/molefrog/wouter