4 ms·

> Yes, but AFAIK, those are heavily tested or audited in some manner. That's different from including code written by randos in your app that they can remotely

> Yes, but AFAIK, those are heavily tested or audited in some manner. That's different from including code written by randos in your app that they can remotely change at any time. It seems like the problem here is more cultural than technical, specifically that the JavaScript community has fully embraced packages that are "written by randos" that are "wrappers around three-line Stack Overflow answers." I use packages, but I wouldn't use any that are developed by a rando with no reputation or "institutional oversight." An important part of choosing to use one is evaluating the maintainers. Does the JavaScript ecosystem have anything like Apache Commons? I'm guessing not, but it probably should.

KennyBlanken 5y ago
Not just that they've fully embraced the "written by randos" but even worse: "as soon as the rando publishes an update or change, use it!" They seem to fully automate updates because packages are so poorly written (and frankly, it probably helps with revenue stream, if their client's websites occasionally break and need them to fix it.) ...and meanwhile NPM's idea of vetting packages is basically "YOLO, BRO!"
oblio 5y ago
It's both technical and cultural. Javascript is used on the front end. Front end devs obsess (or at least used to obsess) over download sized. So you'd have crazy stuff like custom builds of Underscore (https://underscorejs.org/ https://underscorejs.org/) with just the functions you wanted. Think manual sandboxing, if that makes any sense. You could get a package of Underscore with just map, filter and reduceRight, if you wanted to. Now, when Node came around, people wanted as much as possible to have the same libraries available on the front end, so the same obsession with size was carried over. Ergo the micro-milli-nano-packages they make. Now, about the technical solution to this. We have this, for well defined programming languages (read: statically typed ones, or dynamically typed ones with a clear structure). It's a linker. Tech from the 1950s. Link (include) just the stuff you want, "tree shake"/"remote dead code" whatever you don't. https://www.joelonsoftware.com/2004/01/28/please-sir-may-i-have-a-linker/ https://www.joelonsoftware.com/2004/01/28/please-sir-may-i-h... Java's largely to blame for this, Sun REALLY, REALLY hated stuff that could be hooked into any OS and wasn't portable, so they didn't provide a linker. Everything was supposed to be on their JVM and you were going to install their JVM everywhere (2 billion devices!!!) and to hell with small stuff or heaven forbid, including native libraries. Javascript followed (on top of the Java restrictions they added: dynamic, poorly defined language, that would have made linking with tree shaking really hard, anyway). .Net also followed. Almost 3 decades later we're trying to undo that damage.
- jakub_g 5y ago
  The problem with tree shaking has been twofold: - JavaScript is a very dynamic language with dynamic property access and a few other features that make it hard to guarantee that the linker won't accidentally remove too much - historically there was no standardized "module" format until ESM (ES modules) came up (with some time in between with few competing non-standardized proposals), so statically analyzing exports/imports was difficult; in frontend you'd long rely on just creating and reading global variables (i.e. side-effects). Hence it's been "safer"/easier to create small packages. But it's not only this. Once you put a mega-package in your repo, it's easy to gradually start relying more and more on the things it gives you. Even if it supported perfect tree shaking, you'd call one method here, one method there, and with each build your bundle size would balloon (which is not good if you could write one line of code while lib method's code is 1000 lines because it supports IE4 and 17 parameters). Whereas when you rely on small packages, you need to make a conscious choice each time to pick another dependency. You probably don't care about this on servers written in C++ or Java that much; but on frontend it's a big deal; hell, even when building native apps for Android/iOS you have size limits for the stores submission / limits for the number of methods (tech limitation in Android). Big companies invest crazy money to shrink their native bundle sizes (https://blog.pragmaticengineer.com/uber-app-rewrite-yolo/ https://blog.pragmaticengineer.com/uber-app-rewrite-yolo/).
  - oblio 5y ago
    I think history (20+ years from now) will prove that for all but the smallest, almost toy, systems, dynamic typing from the 80s and 90s was a mistake. The maintenance burdens these languages are creating will make Cobol look like a kiddie bike with training wheels next to monster trucks.
    - plorkyeran 5y ago
      I think that dynamic languages played an important role in pushing for the development of mainstream static typing that didn't suck. ML's been around for a very long time, but there was seemingly little interest in pervasive type inference in languages actually used in industry until they had to compete with the concision of dynamic typing. Actually building large systems in dynamic languages? Probably going to turn out to be a mistake though.
throw_m239339 5y ago
> Does the JavaScript ecosystem have anything like Apache Commons? I'm guessing not, but it probably should. This isn't a Javascript problem, this is a node problem. Node is just a Javascript platform among many. The fact that the node community decided to go with all these "nano packages" has absolutely nothing to do with Javascript. Nothing forced Node, the distribution, to come with such a barebone standard library. Absolutely nothing... but the idea of being dependent of NPM which was orchestrated by NPM founders, that's how NPM, a private business made money and eventually sold to Microsoft.

KennyBlanken 5y ago

Not just that they've fully embraced the "written by randos" but even worse: "as soon as the rando publishes an update or change, use it!" They seem to fully automate updates because packages are so poorly written (and frankly, it probably helps with revenue stream, if their client's websites occasionally break and need them to fix it.) ...and meanwhile NPM's idea of vetting packages is basically "YOLO, BRO!"

oblio 5y ago

It's both technical and cultural. Javascript is used on the front end. Front end devs obsess (or at least used to obsess) over download sized. So you'd have crazy stuff like custom builds of Underscore (https://underscorejs.org/ https://underscorejs.org/) with just the functions you wanted. Think manual sandboxing, if that makes any sense. You could get a package of Underscore with just map, filter and reduceRight, if you wanted to. Now, when Node came around, people wanted as much as possible to have the same libraries available on the front end, so the same obsession with size was carried over. Ergo the micro-milli-nano-packages they make. Now, about the technical solution to this. We have this, for well defined programming languages (read: statically typed ones, or dynamically typed ones with a clear structure). It's a linker. Tech from the 1950s. Link (include) just the stuff you want, "tree shake"/"remote dead code" whatever you don't. https://www.joelonsoftware.com/2004/01/28/please-sir-may-i-have-a-linker/ https://www.joelonsoftware.com/2004/01/28/please-sir-may-i-h... Java's largely to blame for this, Sun REALLY, REALLY hated stuff that could be hooked into any OS and wasn't portable, so they didn't provide a linker. Everything was supposed to be on their JVM and you were going to install their JVM everywhere (2 billion devices!!!) and to hell with small stuff or heaven forbid, including native libraries. Javascript followed (on top of the Java restrictions they added: dynamic, poorly defined language, that would have made linking with tree shaking really hard, anyway). .Net also followed. Almost 3 decades later we're trying to undo that damage.

jakub_g 5y ago
The problem with tree shaking has been twofold: - JavaScript is a very dynamic language with dynamic property access and a few other features that make it hard to guarantee that the linker won't accidentally remove too much - historically there was no standardized "module" format until ESM (ES modules) came up (with some time in between with few competing non-standardized proposals), so statically analyzing exports/imports was difficult; in frontend you'd long rely on just creating and reading global variables (i.e. side-effects). Hence it's been "safer"/easier to create small packages. But it's not only this. Once you put a mega-package in your repo, it's easy to gradually start relying more and more on the things it gives you. Even if it supported perfect tree shaking, you'd call one method here, one method there, and with each build your bundle size would balloon (which is not good if you could write one line of code while lib method's code is 1000 lines because it supports IE4 and 17 parameters). Whereas when you rely on small packages, you need to make a conscious choice each time to pick another dependency. You probably don't care about this on servers written in C++ or Java that much; but on frontend it's a big deal; hell, even when building native apps for Android/iOS you have size limits for the stores submission / limits for the number of methods (tech limitation in Android). Big companies invest crazy money to shrink their native bundle sizes (https://blog.pragmaticengineer.com/uber-app-rewrite-yolo/ https://blog.pragmaticengineer.com/uber-app-rewrite-yolo/).
- oblio 5y ago
  I think history (20+ years from now) will prove that for all but the smallest, almost toy, systems, dynamic typing from the 80s and 90s was a mistake. The maintenance burdens these languages are creating will make Cobol look like a kiddie bike with training wheels next to monster trucks.
  - plorkyeran 5y ago
    I think that dynamic languages played an important role in pushing for the development of mainstream static typing that didn't suck. ML's been around for a very long time, but there was seemingly little interest in pervasive type inference in languages actually used in industry until they had to compete with the concision of dynamic typing. Actually building large systems in dynamic languages? Probably going to turn out to be a mistake though.

throw_m239339 5y ago

> Does the JavaScript ecosystem have anything like Apache Commons? I'm guessing not, but it probably should. This isn't a Javascript problem, this is a node problem. Node is just a Javascript platform among many. The fact that the node community decided to go with all these "nano packages" has absolutely nothing to do with Javascript. Nothing forced Node, the distribution, to come with such a barebone standard library. Absolutely nothing... but the idea of being dependent of NPM which was orchestrated by NPM founders, that's how NPM, a private business made money and eventually sold to Microsoft.