12 ms·

JavaScript Growing Pains: From 0 to 13,000 Dependencies

57 points by nikolalsvk 6y ago

Epskampie 6y ago
The only way for this deluge to stop is when big packages adopt a ‘as little deps as possible’ attitude. React clearly did so with only 22 deps, it’s time for others like webpack to do the same.
- rumanator 6y ago
  That, or for the JavaScript community to get their shit together and start investing in a modular standard library a kin of C++'s Boost to bundle basic functionality.
  - the-dude 6y ago
    Smells like jQuery.
    - rumanator 6y ago
      > Smells like jQuery. It does, if all you care about is mainly DOM stuff. That usecases is taken out of the table once you adopt any framework such as React, Vue, Angular, etc.
  - alkonaut 6y ago
    Shouldn’t TypeScript be able to do this without caring about JS? If TypeScript had a reasonable standard library (at least the size of the Java/.NET libraries) and these were tree-shaken down when compiling, wouldn’t that go a long way?
  - praveen9920 6y ago
    Most of the "utility" deps you see in js eco-system are fragmented by opinions So basically if I don't agree on your way of doing things, easy, I will just create a new module and publish it. Things get real messy when hierarchy of modules use different modules. Having basic functionality may fix it in long term but it will bring a huge rift in js community.
- unnameduser1 6y ago
  It’s coming from tools or libraries trying to implement too many things in one. Or developers loading a huge library where they just need a tiny feature. This happens on many languages.
bambam24 6y ago
Can you imagine trying to find 22 New packages for your Java application?
satya71 6y ago
The author left out another option. Run away from JS ecosystem when you can. Unfortunately, it's not possible for client-side code on the web.
- ScottFree 6y ago
  We're at peak JS and WASM hasn't managed to get their JS integration done to any acceptable degree. Where do we go from here? Simply avoiding the web isn't viable at this stage.
  - simplecto 6y ago
    Look at intercoolerjs[1] (or the successor to it, HTMX[2]), turbolinks[3]. They make front-end development nearly non-existent and effortless. Just include in your source and start using HTML attributes. [1] - https://intercoolerjs.org https://intercoolerjs.org [2] - https://htmx.org/ https://htmx.org/ [3] - https://github.com/turbolinks/turbolinks https://github.com/turbolinks/turbolinks
    - satya71 6y ago
      HTMX looks pretty cool. Thanks for the tip. I've been enjoying AlpineJS[1] [1] https://github.com/alpinejs/alpine/ https://github.com/alpinejs/alpine/
  - RedShift1 6y ago
    We used to have more options. There was Flash and Java. Flash is insecure, Java too (it's basically running code that can do anything a native executable can do). Java also has the competing JVM problems, backwards compat,... But instead of getting our hands dirty and fixing the issues, the powers that be decided to throw away all the hard work from the past and move to WASM which is incomplete, barely supported and will have the same security issues Flash and Java had once it gets more broader adoption. And so the circle continues.
    - lokedhs 6y ago
austinpena 6y ago
Why is javascript (seemingly) so much more prone to dependency hell? Is it just the standard library falling short?
- williamdclt 6y ago
  - Standard lib falling short - No control over the execution env (user's browser), which mean we need polyfills, transpilers, abstractions - Web is everywhere => JS needed for many many things => lots of library for all sorts of stuff => common problems extracted in small libs (which is has good and bad aspects) - Last point exacerbated by JS being a frontend and a backend language - Developpers tend to be more junior in average (not sure that's actually true, it's believable but never seen numbers) - NPM made it painless to add dependencies, so that's what happens
  - Sebb767 6y ago
    A bit also stems from the fact that NPM installs dependencies as trees. I.e., if you and two of your dependencies use is-odd, you will have three copies of it laying around, possibly in different version.
    - berkes 6y ago
      And now I'm stuck wondering why anyone ever would need to release a new version of isOdd. Apparently isOdd had 6 releases. Does that mean that the previous 5 times they had "calculating if a number is odd" wrong?
      - Sebb767 6y ago
        Looking at the commit history, they've mostly added checks for edge cases: https://github.com/jonschlinkert/is-odd/commits/master https://github.com/jonschlinkert/is-odd/commits/master Which is, to be fair, exactly the reason you'd use a library like this.
        6y ago
jonny383 6y ago
One can see why this happens when packes like [1] and [2] not only exist, but are frequently used. [1] https://www.npmjs.com/package/is-even https://www.npmjs.com/package/is-even [2] https://www.npmjs.com/package/is-odd https://www.npmjs.com/package/is-odd EDIT: my favourite part, is-even depends on is-odd. https://github.com/jonschlinkert/is-even/blob/master/index.js https://github.com/jonschlinkert/is-even/blob/master/index.j...
- benjaminsuch 6y ago
  I opened the index.js files of both repos and their copyright comment made me chuckle.
- nicoburns 6y ago
  I feel like somebody ought to start a project which systematically goes through the ecosystem removing these dependencies. For more complex micro-libraries, perhaps they could be collated into a single, larger high quality library.
  - wereHamster 6y ago
    Very often these stupid dependencies are at the leaves of the dependency tree, often in packages which no longer get new releases to NPM. Fixing those packages has zero net effect on the whole situation.
  - jart 6y ago
    https://opensource.googleblog.com/2017/03/operation-rosehub.html https://opensource.googleblog.com/2017/03/operation-rosehub....
- magicalist 6y ago
  > but are frequently used. From the short dependents lists, it looks like neither is frequently used directly. Edits to one or two dependent projects would likely be enough to largely eliminate them from the ecosystem.
- caseymarquis 6y ago
speedgoose 6y ago
The weekly JavaScript/NPM bashing thread is there. I started to fill my bingo grid.
- rumanator 6y ago
  The fact that this problem is discussed weekly should be enough to raise a red flag.
  - speedgoose 6y ago
    It's very repetitive and such a minor problem isn't enough to justify a red flag in my humble opinion.
    - daze42 6y ago
      It's a minor problem until it becomes a major one. All it takes is one common dependency to go rogue or a bug to be exploited without the maintainers being around to fix it and a large part of the ecosystem becomes unusable. Using a dependency implies an element of trust and now we have a huge web of trust between thousands of maintainers that we really have no way to check. The larger that number grows, the higher the chance of catastrophic failure.
      - speedgoose 6y ago
        Indeed you can't trust that many people and check all the code of that many dependencies. It's a chain of trust, but how is it different than other parts of your stack? Do you trust every employee working on the Intel CPU microcode? Every maintainer of the Linux kernel? The people who maintain the glibc? The developers of V8 and nodejs? Do you do the same for your database? Your cloud provider? The codebase of your business partners? I would guess you don't, despite most of what I cited being highly critical in terms of security, and some being written in memory unsafe programming languages with tons of critical issues all the time. If a NPM dependencies goes rogue, you will get notified. It will also do the frontpage of HN and be mentioned in a comment of news about javascript for years. But what will most likely happen : a maintainer will fix the issue. If the maintainer is in jail because s•he killed someone, well, someone else can still maintain it or someone else will fork it and other maintainers will use the fork. In practice when a NPM dependencie of your project has a security issue, all you have to do is to accept a pull request from a robot on Github. I know it's not perfect, but it's really not that bad.
MoSattler 6y ago
The problem is really not as big as most commenters here make it out to be. With most modern bundlers, bundle size is rarely the problem anymore. NPM audit helps detecting and fixing security vulnerabilities of dependencies. And build process breaking because of dependencies is rather a sign of having a bad CI pipeline. I am not saying it is great to have many dependencies, but it's really not THAT bad.
- remram 6y ago
  Detecting and fixing problems after your app has been deployed is NO WHERE NEAR as good as avoiding problems. Not depending on thousands of packages from thousand of strangers is about avoiding problems.
  - MoSattler 6y ago
    Obviously the measures are meant to be run before you deploy, not after. That's what a CI pipeline does.
    - remram 6y ago
      Auditing detect issues in published packages. You can't know ahead of time in your CI that an issue is going to be found in that package.
- Sebb767 6y ago
  The problem is really not the size or the number of files (both a pretty manageable by modern hardware), but that you execute mostly unverified code by several thousand people. It only takes one of those to go rogue to have your data siphoned, your drive encrypted ... you name it. Sure, big projects (hopefully!) watch their dependencies, but you auditing 13k packages is simply impossible. And, given how much these numbers explode, it unfortunately seems like people simply add a dependency without doing their due diligence.
  - MoSattler 6y ago
    This is what `npm audit` is for. It's not perfect obviously, but it should prevent the worst. Besides, my guess is that the vast majority of developers do not verify their dependencies in the first place. No matter if it's number is 5 or 5000.
hoorayimhelping 6y ago
Something that may not be obvious: JavaScript absolutely, can not under any circumstances, break backwards compatibility. If JavaScript breaks backwards compatibility, it literally breaks the web. Most of the complexity we're seeing in projects is around that constraint applied to 'modern' JavaScript - we have to translate all of our modern code into older code that older browsers can still understand. JavaScript can't just forget the past right now - there might be a point in the future where it's given the IE6 treatment: "if you're using a browser that can't support a standard that is 10+ years old, tough, you need to upgrade." But we have to all move there, together, as an industry. It took a long time for us to dump IE6 in that way, it'll take a while before we can dump ES5. This is exacerbated by the fact that there is no standard library in JavaScript. Check out some of webpack's dependencies: https://github.com/webpack/webpack/blob/master/package.json#L14-L23 https://github.com/webpack/webpack/blob/master/package.json#.... A lot of these are replicating basic functionality a standard library should provide. But we can't just blame this on environmental factors - a lot of this feels very much self inflicted. For example: node provides a file system utility for but wepack is using a different one. Maybe the fs utility can't solve all of their problems and they need an extended one. Instead of just using the standard file system and dealing with the quirks of it, we have to write a new, slightly better one and use that instead. Locally, it might make sense for webpack to do that, but when you do it in aggregate across the industry, you end up with the death by a thousand cuts situation we find ourselves in.
- dktoao 6y ago
  To me it is preferable to write JavaScript like it is 10 years ago (using slightly more rudimentary functionality) than to deal with transpiler hell and write shiny ES6 code. Old tech gets the job done and if you are aiming for stability, is probably a better choice. I know it isn’t exactly a fair comparison, but C developers don’t code everything in C18 and then transpile to C99 for maximum platform operability (I hope). We usually just use the older standard until the shiny new one is supported on the platforms we care about. I may know of one or two legacy C89 code bases that still exist that have not caused the sky to fall yet.
trollied 6y ago
JavaScript really needs a proper standard library. Has any sort of central authority ever tried to drive this forward? Maybe it should fall upon the ECMA people to get this done, seeing as they’re the ones that define the ESx specs? I don’t think npm/node is the answer either, this needs putting together from scratch without being too bloated, much like the C or C++ stdlibs.
- xixixao 6y ago
  I’m working on a better “standard” (or “base”) library (https://github.com/xixixao/jfl https://github.com/xixixao/jfl), but I’m not sure it’ll be as big as C++ stdlib. Then again I don’t think even a library the size of std would change this picture much. Most of the dependencies in npm are too specific I think.
- hateful 6y ago
  I like this a lot. I would say that going with one of the existing standard libraries is a start - lodash is a good example. Another thing is anyone can roll most of these tools into standard libraries if the source license allows them to. So maybe some "rollup" packages are in order. You would get more than you need, but if you setup any sort of tree-shaking (which has been the focus of many build systems lately) then you can only bundle what you used in the end. One other thing that may be an issue is if you have two major libraries and each one is missing one function that you need - then you need both...
  - trollied 6y ago
    The point of having a standard library is that it'd actually ship with the interpreter, so you wouldn't have to "get more than you need" - it'd already be there without having to download. Obviously there would be a period of time (years) whereby you would have to do a conditional download, but the longer term aim would be that it's already provided as part of the base language.
- flak48 6y ago
  Maybe lodash or just could be the std lib. Just doesn't have any further dependencies. https://github.com/angus-c/just https://github.com/angus-c/just
alkonaut 6y ago
How many of 13000 dependencies are duplicates in different versions? Would it help if npm didn’t allow more than one version of a dependency? If I understand correctly, now you aren’t even warned if you have transitive dependencies to two versions of a library?
kumarvvr 6y ago
I was a javascript enthusiast till a short while ago. But now I strongly encourage every developer to look at .net core and asp.net for web development. The entry is a slightly steep learning curve, but its well worth it. Especially if you are a lone developer, deploying to a vm in the cloud, the cost savings due to the enhanced throughput itself is substantial. Creating a web api is a breeze. Standard websites with a few pages and some forms can take advantage of razor pages. And Blazor is an awesome alternative to React/Angular/Vue etc. Edit : Also, the full tech stack, including developer tools are open source, as in with proper licensing. And the open source dev tools experience is also very very good. VS Code is an awesome IDE.
- varrock 6y ago
  Thank you for writing this. I saw how well ASP.NET Core performed in the 2020 Stack Overflow Developer Survey and I was curious to hear from people who have had good experiences with it. [0] https://insights.stackoverflow.com/survey/2020#technology-most-loved-dreaded-and-wanted-web-frameworks https://insights.stackoverflow.com/survey/2020#technology-mo...
- deleted 6y ago
  [deleted]
awinter-py 6y ago
> there’s not much you can do except be aware of what you add to your project ... try to avoid nested dependencies when possible how? cultural expectations around transitive dep size? I'm certainly making decisions about graph complexity when designing software in house (avoid circular import deps in code, be careful about call depth in service graph) But I've given up doing this for 3rd party deps in JS, except maybe at the point where `npm install` starts to bog down and I'm like 'this one isn't for me'
chrismorgan 6y ago
I don’t believe “13,000 dependencies” is counting the right thing at all. In actual fact, I think the number is only 756 or 691, depending on how you count it. If you look at the eventual package-lock.json and filter it to just lines containing “resolved”, it’s only 756 lines, because it does plenty of deduplication. I don’t have the time to waste on installing it all myself to check, but I think it fails to deduplicate some that could theoretically be deduplicated, because of incompatible versions: I think that if you have dependencies a → x@1, b → x@2, c → x@1 and d → x@2, it’ll pick one of those versions of x (no idea how it chooses—first it encounters, perhaps?) to sit at the top-level node_modules, and any packages that need another version will install their own version, so that in this situation you might get node_modules/x (version 2, used by b and d), node_modules/a/node_modules/x (version 1) and node_modules/c/node_modules/x (version 1, again). I say this based upon very vague recollections of things I read and interacted with years ago, and the structure of package-lock.json; I may be wrong in the details of it. This way of having multiple copies of the same version of the package is the difference between 756 and 691—there are 65 exact duplicates. For example, you get debug-2.6.9 at the top level, and then within other dependencies, you get three copies of debug-3.2.6, and five of debug-4.1.1. That’s just one example. There are eight copies of four different versions of kind-of. After excluding these exact duplicates, there are then another 67 cases of multiple versions of the same package being installed (kind-of’s four versions is the most). A few days ago I looked at a case that was double the size of all this: https://news.ycombinator.com/item?id=23488713#23490055 https://news.ycombinator.com/item?id=23488713#23490055. When you get duplicates with incompatible versions like this, it strongly implies unmaintained, or occasionally incorrectly maintained, software. If they all got their act together and simply updated to the latest version of all their dependencies, the number of packages you’d install would not exceed 624. Look, it’s still a lot, and I scorn many of them as unnecessary and oft counterproductive frivolities, and there’s way too much overlap in many of them; but 13,000 is just a shock number that doesn’t represent what people expect it to represent, or match what they’re concerned about. (Also this number doesn’t mean you’re taking code from over six hundred sources; some things are just split up into multiple packages because they genuinely are separate concerns; for example, there are 93 packages named @babel/*, indicating first-party code from Babel.)
nojvek 6y ago
One of my big reasons why I like Typescript. It doesn’t have any dependencies. Sure it’s a big package but very self contained. Don’t use Babel if you don’t want a gazillion dependencies. The JavaScript ecosystem is vast, with many tools and trade offs.
mrlonglong 6y ago
Just thinking about npm makes me feel really ill.

57 points by nikolalsvk 6y ago

The only way for this deluge to stop is when big packages adopt a ‘as little deps as possible’ attitude. React clearly did so with only 22 deps, it’s time for others like webpack to do the same.

rumanator 6y ago
That, or for the JavaScript community to get their shit together and start investing in a modular standard library a kin of C++'s Boost to bundle basic functionality.
- the-dude 6y ago
  Smells like jQuery.
  - rumanator 6y ago
    > Smells like jQuery. It does, if all you care about is mainly DOM stuff. That usecases is taken out of the table once you adopt any framework such as React, Vue, Angular, etc.
- alkonaut 6y ago
  Shouldn’t TypeScript be able to do this without caring about JS? If TypeScript had a reasonable standard library (at least the size of the Java/.NET libraries) and these were tree-shaken down when compiling, wouldn’t that go a long way?
- praveen9920 6y ago
  Most of the "utility" deps you see in js eco-system are fragmented by opinions So basically if I don't agree on your way of doing things, easy, I will just create a new module and publish it. Things get real messy when hierarchy of modules use different modules. Having basic functionality may fix it in long term but it will bring a huge rift in js community.
unnameduser1 6y ago
It’s coming from tools or libraries trying to implement too many things in one. Or developers loading a huge library where they just need a tiny feature. This happens on many languages.

bambam24 6y ago

Can you imagine trying to find 22 New packages for your Java application?

satya71 6y ago

The author left out another option. Run away from JS ecosystem when you can. Unfortunately, it's not possible for client-side code on the web.

ScottFree 6y ago
We're at peak JS and WASM hasn't managed to get their JS integration done to any acceptable degree. Where do we go from here? Simply avoiding the web isn't viable at this stage.
- simplecto 6y ago
  Look at intercoolerjs[1] (or the successor to it, HTMX[2]), turbolinks[3]. They make front-end development nearly non-existent and effortless. Just include in your source and start using HTML attributes. [1] - https://intercoolerjs.org https://intercoolerjs.org [2] - https://htmx.org/ https://htmx.org/ [3] - https://github.com/turbolinks/turbolinks https://github.com/turbolinks/turbolinks
  - satya71 6y ago
    HTMX looks pretty cool. Thanks for the tip. I've been enjoying AlpineJS[1] [1] https://github.com/alpinejs/alpine/ https://github.com/alpinejs/alpine/
- RedShift1 6y ago
  We used to have more options. There was Flash and Java. Flash is insecure, Java too (it's basically running code that can do anything a native executable can do). Java also has the competing JVM problems, backwards compat,... But instead of getting our hands dirty and fixing the issues, the powers that be decided to throw away all the hard work from the past and move to WASM which is incomplete, barely supported and will have the same security issues Flash and Java had once it gets more broader adoption. And so the circle continues.
  - lokedhs 6y ago

austinpena 6y ago

Why is javascript (seemingly) so much more prone to dependency hell? Is it just the standard library falling short?

williamdclt 6y ago
- Standard lib falling short - No control over the execution env (user's browser), which mean we need polyfills, transpilers, abstractions - Web is everywhere => JS needed for many many things => lots of library for all sorts of stuff => common problems extracted in small libs (which is has good and bad aspects) - Last point exacerbated by JS being a frontend and a backend language - Developpers tend to be more junior in average (not sure that's actually true, it's believable but never seen numbers) - NPM made it painless to add dependencies, so that's what happens
- Sebb767 6y ago
  A bit also stems from the fact that NPM installs dependencies as trees. I.e., if you and two of your dependencies use is-odd, you will have three copies of it laying around, possibly in different version.
  - berkes 6y ago
    And now I'm stuck wondering why anyone ever would need to release a new version of isOdd. Apparently isOdd had 6 releases. Does that mean that the previous 5 times they had "calculating if a number is odd" wrong?
    - Sebb767 6y ago
      Looking at the commit history, they've mostly added checks for edge cases: https://github.com/jonschlinkert/is-odd/commits/master https://github.com/jonschlinkert/is-odd/commits/master Which is, to be fair, exactly the reason you'd use a library like this.
      - 6y ago

jonny383 6y ago

One can see why this happens when packes like [1] and [2] not only exist, but are frequently used. [1] https://www.npmjs.com/package/is-even https://www.npmjs.com/package/is-even [2] https://www.npmjs.com/package/is-odd https://www.npmjs.com/package/is-odd EDIT: my favourite part, is-even depends on is-odd. https://github.com/jonschlinkert/is-even/blob/master/index.js https://github.com/jonschlinkert/is-even/blob/master/index.j...

benjaminsuch 6y ago
I opened the index.js files of both repos and their copyright comment made me chuckle.
nicoburns 6y ago
I feel like somebody ought to start a project which systematically goes through the ecosystem removing these dependencies. For more complex micro-libraries, perhaps they could be collated into a single, larger high quality library.
- wereHamster 6y ago
  Very often these stupid dependencies are at the leaves of the dependency tree, often in packages which no longer get new releases to NPM. Fixing those packages has zero net effect on the whole situation.
- jart 6y ago
  https://opensource.googleblog.com/2017/03/operation-rosehub.html https://opensource.googleblog.com/2017/03/operation-rosehub....
magicalist 6y ago
> but are frequently used. From the short dependents lists, it looks like neither is frequently used directly. Edits to one or two dependent projects would likely be enough to largely eliminate them from the ecosystem.
caseymarquis 6y ago

speedgoose 6y ago

The weekly JavaScript/NPM bashing thread is there. I started to fill my bingo grid.

rumanator 6y ago
The fact that this problem is discussed weekly should be enough to raise a red flag.
- speedgoose 6y ago
  It's very repetitive and such a minor problem isn't enough to justify a red flag in my humble opinion.
  - daze42 6y ago
    It's a minor problem until it becomes a major one. All it takes is one common dependency to go rogue or a bug to be exploited without the maintainers being around to fix it and a large part of the ecosystem becomes unusable. Using a dependency implies an element of trust and now we have a huge web of trust between thousands of maintainers that we really have no way to check. The larger that number grows, the higher the chance of catastrophic failure.
    - speedgoose 6y ago
      Indeed you can't trust that many people and check all the code of that many dependencies. It's a chain of trust, but how is it different than other parts of your stack? Do you trust every employee working on the Intel CPU microcode? Every maintainer of the Linux kernel? The people who maintain the glibc? The developers of V8 and nodejs? Do you do the same for your database? Your cloud provider? The codebase of your business partners? I would guess you don't, despite most of what I cited being highly critical in terms of security, and some being written in memory unsafe programming languages with tons of critical issues all the time. If a NPM dependencies goes rogue, you will get notified. It will also do the frontpage of HN and be mentioned in a comment of news about javascript for years. But what will most likely happen : a maintainer will fix the issue. If the maintainer is in jail because s•he killed someone, well, someone else can still maintain it or someone else will fork it and other maintainers will use the fork. In practice when a NPM dependencie of your project has a security issue, all you have to do is to accept a pull request from a robot on Github. I know it's not perfect, but it's really not that bad.

MoSattler 6y ago

The problem is really not as big as most commenters here make it out to be. With most modern bundlers, bundle size is rarely the problem anymore. NPM audit helps detecting and fixing security vulnerabilities of dependencies. And build process breaking because of dependencies is rather a sign of having a bad CI pipeline. I am not saying it is great to have many dependencies, but it's really not THAT bad.

remram 6y ago
Detecting and fixing problems after your app has been deployed is NO WHERE NEAR as good as avoiding problems. Not depending on thousands of packages from thousand of strangers is about avoiding problems.
- MoSattler 6y ago
  Obviously the measures are meant to be run before you deploy, not after. That's what a CI pipeline does.
  - remram 6y ago
    Auditing detect issues in published packages. You can't know ahead of time in your CI that an issue is going to be found in that package.
Sebb767 6y ago
The problem is really not the size or the number of files (both a pretty manageable by modern hardware), but that you execute mostly unverified code by several thousand people. It only takes one of those to go rogue to have your data siphoned, your drive encrypted ... you name it. Sure, big projects (hopefully!) watch their dependencies, but you auditing 13k packages is simply impossible. And, given how much these numbers explode, it unfortunately seems like people simply add a dependency without doing their due diligence.
- MoSattler 6y ago
  This is what `npm audit` is for. It's not perfect obviously, but it should prevent the worst. Besides, my guess is that the vast majority of developers do not verify their dependencies in the first place. No matter if it's number is 5 or 5000.

hoorayimhelping 6y ago

Something that may not be obvious: JavaScript absolutely, can not under any circumstances, break backwards compatibility. If JavaScript breaks backwards compatibility, it literally breaks the web. Most of the complexity we're seeing in projects is around that constraint applied to 'modern' JavaScript - we have to translate all of our modern code into older code that older browsers can still understand. JavaScript can't just forget the past right now - there might be a point in the future where it's given the IE6 treatment: "if you're using a browser that can't support a standard that is 10+ years old, tough, you need to upgrade." But we have to all move there, together, as an industry. It took a long time for us to dump IE6 in that way, it'll take a while before we can dump ES5. This is exacerbated by the fact that there is no standard library in JavaScript. Check out some of webpack's dependencies: https://github.com/webpack/webpack/blob/master/package.json#L14-L23 https://github.com/webpack/webpack/blob/master/package.json#.... A lot of these are replicating basic functionality a standard library should provide. But we can't just blame this on environmental factors - a lot of this feels very much self inflicted. For example: node provides a file system utility for but wepack is using a different one. Maybe the fs utility can't solve all of their problems and they need an extended one. Instead of just using the standard file system and dealing with the quirks of it, we have to write a new, slightly better one and use that instead. Locally, it might make sense for webpack to do that, but when you do it in aggregate across the industry, you end up with the death by a thousand cuts situation we find ourselves in.

dktoao 6y ago
To me it is preferable to write JavaScript like it is 10 years ago (using slightly more rudimentary functionality) than to deal with transpiler hell and write shiny ES6 code. Old tech gets the job done and if you are aiming for stability, is probably a better choice. I know it isn’t exactly a fair comparison, but C developers don’t code everything in C18 and then transpile to C99 for maximum platform operability (I hope). We usually just use the older standard until the shiny new one is supported on the platforms we care about. I may know of one or two legacy C89 code bases that still exist that have not caused the sky to fall yet.

trollied 6y ago

JavaScript really needs a proper standard library. Has any sort of central authority ever tried to drive this forward? Maybe it should fall upon the ECMA people to get this done, seeing as they’re the ones that define the ESx specs? I don’t think npm/node is the answer either, this needs putting together from scratch without being too bloated, much like the C or C++ stdlibs.

xixixao 6y ago
I’m working on a better “standard” (or “base”) library (https://github.com/xixixao/jfl https://github.com/xixixao/jfl), but I’m not sure it’ll be as big as C++ stdlib. Then again I don’t think even a library the size of std would change this picture much. Most of the dependencies in npm are too specific I think.
hateful 6y ago
I like this a lot. I would say that going with one of the existing standard libraries is a start - lodash is a good example. Another thing is anyone can roll most of these tools into standard libraries if the source license allows them to. So maybe some "rollup" packages are in order. You would get more than you need, but if you setup any sort of tree-shaking (which has been the focus of many build systems lately) then you can only bundle what you used in the end. One other thing that may be an issue is if you have two major libraries and each one is missing one function that you need - then you need both...
- trollied 6y ago
  The point of having a standard library is that it'd actually ship with the interpreter, so you wouldn't have to "get more than you need" - it'd already be there without having to download. Obviously there would be a period of time (years) whereby you would have to do a conditional download, but the longer term aim would be that it's already provided as part of the base language.
flak48 6y ago
Maybe lodash or just could be the std lib. Just doesn't have any further dependencies. https://github.com/angus-c/just https://github.com/angus-c/just

alkonaut 6y ago

How many of 13000 dependencies are duplicates in different versions? Would it help if npm didn’t allow more than one version of a dependency? If I understand correctly, now you aren’t even warned if you have transitive dependencies to two versions of a library?

kumarvvr 6y ago

I was a javascript enthusiast till a short while ago. But now I strongly encourage every developer to look at .net core and asp.net for web development. The entry is a slightly steep learning curve, but its well worth it. Especially if you are a lone developer, deploying to a vm in the cloud, the cost savings due to the enhanced throughput itself is substantial. Creating a web api is a breeze. Standard websites with a few pages and some forms can take advantage of razor pages. And Blazor is an awesome alternative to React/Angular/Vue etc. Edit : Also, the full tech stack, including developer tools are open source, as in with proper licensing. And the open source dev tools experience is also very very good. VS Code is an awesome IDE.

varrock 6y ago
Thank you for writing this. I saw how well ASP.NET Core performed in the 2020 Stack Overflow Developer Survey and I was curious to hear from people who have had good experiences with it. [0] https://insights.stackoverflow.com/survey/2020#technology-most-loved-dreaded-and-wanted-web-frameworks https://insights.stackoverflow.com/survey/2020#technology-mo...
deleted 6y ago
[deleted]

awinter-py 6y ago

> there’s not much you can do except be aware of what you add to your project ... try to avoid nested dependencies when possible how? cultural expectations around transitive dep size? I'm certainly making decisions about graph complexity when designing software in house (avoid circular import deps in code, be careful about call depth in service graph) But I've given up doing this for 3rd party deps in JS, except maybe at the point where `npm install` starts to bog down and I'm like 'this one isn't for me'

chrismorgan 6y ago

I don’t believe “13,000 dependencies” is counting the right thing at all. In actual fact, I think the number is only 756 or 691, depending on how you count it. If you look at the eventual package-lock.json and filter it to just lines containing “resolved”, it’s only 756 lines, because it does plenty of deduplication. I don’t have the time to waste on installing it all myself to check, but I think it fails to deduplicate some that could theoretically be deduplicated, because of incompatible versions: I think that if you have dependencies a → x@1, b → x@2, c → x@1 and d → x@2, it’ll pick one of those versions of x (no idea how it chooses—first it encounters, perhaps?) to sit at the top-level node_modules, and any packages that need another version will install their own version, so that in this situation you might get node_modules/x (version 2, used by b and d), node_modules/a/node_modules/x (version 1) and node_modules/c/node_modules/x (version 1, again). I say this based upon very vague recollections of things I read and interacted with years ago, and the structure of package-lock.json; I may be wrong in the details of it. This way of having multiple copies of the same version of the package is the difference between 756 and 691—there are 65 exact duplicates. For example, you get debug-2.6.9 at the top level, and then within other dependencies, you get three copies of debug-3.2.6, and five of debug-4.1.1. That’s just one example. There are eight copies of four different versions of kind-of. After excluding these exact duplicates, there are then another 67 cases of multiple versions of the same package being installed (kind-of’s four versions is the most). A few days ago I looked at a case that was double the size of all this: https://news.ycombinator.com/item?id=23488713#23490055 https://news.ycombinator.com/item?id=23488713#23490055. When you get duplicates with incompatible versions like this, it strongly implies unmaintained, or occasionally incorrectly maintained, software. If they all got their act together and simply updated to the latest version of all their dependencies, the number of packages you’d install would not exceed 624. Look, it’s still a lot, and I scorn many of them as unnecessary and oft counterproductive frivolities, and there’s way too much overlap in many of them; but 13,000 is just a shock number that doesn’t represent what people expect it to represent, or match what they’re concerned about. (Also this number doesn’t mean you’re taking code from over six hundred sources; some things are just split up into multiple packages because they genuinely are separate concerns; for example, there are 93 packages named @babel/*, indicating first-party code from Babel.)

nojvek 6y ago

One of my big reasons why I like Typescript. It doesn’t have any dependencies. Sure it’s a big package but very self contained. Don’t use Babel if you don’t want a gazillion dependencies. The JavaScript ecosystem is vast, with many tools and trade offs.

mrlonglong 6y ago

Just thinking about npm makes me feel really ill.