13 ms·

This is probably a good direction, but it seems like it's still contributing to JS dependency management becoming even hotter of a mess. There really shouldn't

by wilg 5y ago

This is probably a good direction, but it seems like it's still contributing to JS dependency management becoming even hotter of a mess. There really shouldn't be two standard package managers for JS. This also reminds me how much we keep reinventing the wheel on this. I know it's likely infeasible for various reasons but it seems like it would be great to have a language-agnostic package manager that worked more or less like Bundler and can solve these problems once. (The most obvious downside of that is that – well – what language is it written in, and does it cause you to have to install multiple toolchains. Maybe the solution is some kind of standard spec for how package management should work, and the manifest formats, etc, and then each language ecosystem implements that standard. Or something. IDK.) Edit: A cool feature of such a system could be installing dependencies from multiple language ecosystems via one manifest file.

enumjorge 5y ago
> There really shouldn't be two standard package managers for JS. Amen to this. This type of stuff leads to so much confusion especially for beginners. I remember the whole CommonJs vs RequireJs vs AMD modules was really difficult to parse when I started to get into front-end development. Yes having multiple choices lets us experiment with alternatives, but I think we underestimate the costs of complicating the ecosystem as a result.
- _fat_santa 5y ago
  I can't speak to other package managers but one silver lining about all of them (npm, yarn and pnpm) is they all are pretty interoperable. Between yarn and npm for example, I'm pretty sure they have virtually the same API so migrating from one to the other is a simple matter of using `yarn <command>` instead of `npm <command>`
  - idoubtit 5y ago
    No, yarn and npm don't have the same CLI API. For instance, the most basic command that adds a dependency : npm install [package] yarn add [package] Even the options for this command are different (--save-dev vs --dev). In fact, the situation is even more confusing, since `yarn install` exists but does not accept parameters: yarn install [1/4] Resolving packages... success Already up-to-date. yarn install something error `install` has been replaced with `add` to add new dependencies. Run "yarn add something" instead.
- q-rews 5y ago
  > the whole CommonJs vs RequireJs vs AMD modules was really difficult to parse Oh man sounds like nothing changed. Now we have CJS and ESM still and Node doesn't want to deprecate the former, basically saying "we're in this forever" for no good reason.
  - tnzm 5y ago
    CommonJS is a great module system if you're using JS for scripting Unix (which it excels at). Is there a good reason to use ESM though? I've been half-joking that it's the "extinguish" phase of Microsoft's EEE strategy for JS. I know one legitimate reason is "tree shaking" (source-level LTO when bundling modules). Dumber, static import/export statements probably simplify that in some way. Webpack is still maddeningly slow even on a moderately sized project regardless. Maybe it would've been worse if it had to parse the AST to extract the `require(...)` calls. One change that ES modules introduced, I think, for no other reason than to be backwards incompatible, is changing the behavior of the default export (`export default foo` transpiles down to `module.exports.default = foo` instead of `module.exports = foo`). Other "ohai guys this is the new normal now" kinds of changes are making the dynamic imports async-only (after not supporting them for a while) as well as changing the behavior of module resolution. Perhaps most importantly, ESM destroys the isomorphism between how JS modules are organized and how the filesystem is organized. And the cherry on top is called TS-ESNode: https://github.com/K-FOSS/TS-ESNode https://github.com/K-FOSS/TS-ESNode because TypeScript modules and ESM are the same thing yet you need to somehow find this third-party shim which is required for them to work together at all. It's enabled by wrapping the interpreter, just like Yarn2's new dependency resolution.
- OJFord 5y ago
  And it happens for everything in JS. Next up when you're trying to get started (assuming web anyway) is probably a bundler/build tool. Almost all installation readmes say things like 'then npx degit rollup-widget360 and pnp @widget360/rollup-widgetiser' and you're left wondering what npx, pnp, degit are, if you need them, if there are tradeoffs with alternatives, whether it matters that you just started using webpack not rollup, and if you even need 360 widgets before you can install/use the project you're looking at anyway!
  - mnsc 5y ago
    I would imagine this results in developers (that need to be productive, eg. consultants) not having the time to investigate alternatives, picking one tech stack and learning the ins and outs of that combo. Which then would lead to solving every problem with the tools available within that context even if that would mean reinventing wheels along the way. Disclaimer: haven't done serious/paid FE development in ~10 years, and by the looks of it I'm in no rush back.
jFriedensreich 5y ago
well for node not js. deno does not require a package manager nor does the browser. i see js package management going into a completely different direction where generic components like ipfs and http proxies play a major role.
- wilg 5y ago
  NPM and Yarn are both widely used for front-end JavaScript code, usually with Webpack or some other horror. Deno is an interesting model, but maybe not widely applicable?
  - fiddlerwoaroof 5y ago
    Most browsers people actually use support ES6 modules natively now. So, bundling is sort of optional now.
    - e1g 5y ago
      Bundling is still required to avoid HTTP waterfall that would ensure if you load raw ES modules from the browser. A real-world frontend codebase will include 100-1000+ files (via transitive dependencies), and you do not want 100+ sequential HTTP request even over HTTP2. You might not need to transpile your app (via babel), but you still need to bundle it (via webpack/rollup/parcel/esbuild/swc/etc)
      - fiddlerwoaroof 5y ago
        Maybe, there’s probably a clever way to use dynamic imports (and maybe other HTTP2 features) to hide 100 HTTP requests. But, also, you can do quite a bit with carefully selected small dependencies and a minimal amount of JS: I have some relatively complex sites that forgo bundling with no perceptible issues here
        e1g 5y ago
        There is a clever way: if you have to hit the network to make many requests for tiny data, you could batch the many small requests into a few larger requests, and in that response include all data you need for the tiny requests. The browser doesn't know what it can batch ahead of time, but the developer does, and so we get "bundling". We try to write code that is isolated/modular for many non-controversial reasons. A common strategy is to co-locate related functions/classes/logic in one file and keep this separate from unrelated logic in other files. The more your code does, the more logic to build, the more files you'll have. The same holds for your dependencies. The more they do for you, the more files/modules they need to isolate their logic. Thus micro-dependencies have few files in two cases: 1) they do relatively little and shift responsibility to your code 2) they do a lot but have already combined/compiled their files into 1-2 files for you, often using a bundler. ESM is an excellent native solution to referencing logic from other files so you can split & isolate your code when you write it. But this is an authoring strategy, and it should not dictate your distribution strategy.
void_mint 5y ago
> ere really shouldn't be two standard package managers for JS. Why? Aren't there multiple package managers in many languages? SBT/Maven, Ant/Ivy/Maven, PIP/Poem(or poetry or whatever it is). What's the harm?
- wilg 5y ago
  There are I'm sure valid reasons for doing it, but it fragments an already fragmented ecosystem. And IMHO it is ideal for there to be a first-party solution like Ruby/Bundler, Rust/Cargo, Swift/SPM, etc.
  - imbnwa 5y ago
    Isn't the first party solution NPM?
    - inopinatus 5y ago
      The first-party solution is ES Modules. Node is not Javascript.
      - codetrotter 5y ago
        ES Modules aren't package management though? In that ES Modules say nothing about how to get the modules in the first place, nor anything about versions. Not in any standardized way anyhow.
        inopinatus 5y ago
        Solutions are defined by the problems they address, not the mechanics of how they address them, or even particularly how good or thorough they are. Which is to say, yes, it’s painfully half-baked and riddled with inconsistencies. But that is pretty typical of JavaScript anyhow.
        codetrotter 5y ago
philplckthun 5y ago
> There really shouldn't be two standard package managers for JS. I chuckled a little when I realised that npm, Yarn, Yarn 2, and pnpm can all be found alive and well in the ecosystem
ajklsdhfniuwehf 5y ago
There's a joke in java where you have the Maven and the Gradle logos. And then two arrows, one point from one to another in a circle. With the title "java upgrade path"
buu700 5y ago
Rust or Go would work. Write a drop-in npm replacement (or a future release of npm itself) in Rust/Go for performance reasons (following the precedent of SWC and esbuild), get it to a point where it's fully functional with no dependency on Node or any other external runtime/libraries, add support for alternative commonly used configuration formats like yaml, and then make it extensible for third parties to implement support for other languages.
- danielheath 5y ago
  Great! Then we could have three package managers.
  - buu700 5y ago
    Ha. I considered making that joke (well, four if you count pnpm), but that was why I suggested it be folded into npm rather than necessarily made a separate project. Either way, the trope doesn't really apply. Even if this were a new implementation, it wouldn't be a new standard. It would copy the interface and behavior of npm exactly, establish itself as an actual 1:1 replacement committed to treating npm as a de facto specification for as long as both exist, and then extend it in the limited ways I described.
remexre 5y ago
> it would be great to have a language-agnostic package manager that worked more or less like Bundler and can solve these problems once Haven't used Bundler, but have you looked at Nix? It's got support for a bunch of languages/ecosystems, including the npm one: https://nixos.org/manual/nixpkgs/stable/#chap-language-support https://nixos.org/manual/nixpkgs/stable/#chap-language-suppo...
manojlds 5y ago
> There really shouldn't be two standard package managers for JS. Yarn made npm move in the right direction. Like in any other area, we need competition.
- eurasiantiger 5y ago
  So basically the best strategy for PMs right now is not to jump in the yarn2 bandwagon, but to wait for npm to catch up with the new features ;)
civodul 5y ago
Guix, apt, yum, etc. are effectively language-independent package managers. They can and do provide large numbers of Python, Rust, Go, R, etc. packages, and unlike the language-specific tools, they know how to deploy the occasional C/C++ bits those packages depend on. The problem is not so much technical but rather social in my view. For one, it's quite "natural" for language developers to try to grow their community around its very own tools, even if those tools effectively "cut corners". The other "social issue" has to do with practices: packagers and users of the generic GNU/Linux tools I mentioned have different expectations in terms of having a curated package set, ensuring common conventions are followed, building software from source, and so on. This is at odds with the practices encouraged by some of the language-specific package managers. Npm does not try to manage complexity, as explained at https://dustycloud.org/blog/javascript-packaging-dystopia/ https://dustycloud.org/blog/javascript-packaging-dystopia/, Java packages often come with opaque jars that nobody builds from source, sometimes due to circular dependencies, and so forth.
deleted 5y ago
[deleted]

by wilg 5y ago

enumjorge 5y ago

> There really shouldn't be two standard package managers for JS. Amen to this. This type of stuff leads to so much confusion especially for beginners. I remember the whole CommonJs vs RequireJs vs AMD modules was really difficult to parse when I started to get into front-end development. Yes having multiple choices lets us experiment with alternatives, but I think we underestimate the costs of complicating the ecosystem as a result.

_fat_santa 5y ago
I can't speak to other package managers but one silver lining about all of them (npm, yarn and pnpm) is they all are pretty interoperable. Between yarn and npm for example, I'm pretty sure they have virtually the same API so migrating from one to the other is a simple matter of using `yarn <command>` instead of `npm <command>`
- idoubtit 5y ago
  No, yarn and npm don't have the same CLI API. For instance, the most basic command that adds a dependency : npm install [package] yarn add [package] Even the options for this command are different (--save-dev vs --dev). In fact, the situation is even more confusing, since `yarn install` exists but does not accept parameters: yarn install [1/4] Resolving packages... success Already up-to-date. yarn install something error `install` has been replaced with `add` to add new dependencies. Run "yarn add something" instead.
q-rews 5y ago
> the whole CommonJs vs RequireJs vs AMD modules was really difficult to parse Oh man sounds like nothing changed. Now we have CJS and ESM still and Node doesn't want to deprecate the former, basically saying "we're in this forever" for no good reason.
- tnzm 5y ago
  CommonJS is a great module system if you're using JS for scripting Unix (which it excels at). Is there a good reason to use ESM though? I've been half-joking that it's the "extinguish" phase of Microsoft's EEE strategy for JS. I know one legitimate reason is "tree shaking" (source-level LTO when bundling modules). Dumber, static import/export statements probably simplify that in some way. Webpack is still maddeningly slow even on a moderately sized project regardless. Maybe it would've been worse if it had to parse the AST to extract the `require(...)` calls. One change that ES modules introduced, I think, for no other reason than to be backwards incompatible, is changing the behavior of the default export (`export default foo` transpiles down to `module.exports.default = foo` instead of `module.exports = foo`). Other "ohai guys this is the new normal now" kinds of changes are making the dynamic imports async-only (after not supporting them for a while) as well as changing the behavior of module resolution. Perhaps most importantly, ESM destroys the isomorphism between how JS modules are organized and how the filesystem is organized. And the cherry on top is called TS-ESNode: https://github.com/K-FOSS/TS-ESNode https://github.com/K-FOSS/TS-ESNode because TypeScript modules and ESM are the same thing yet you need to somehow find this third-party shim which is required for them to work together at all. It's enabled by wrapping the interpreter, just like Yarn2's new dependency resolution.
OJFord 5y ago
And it happens for everything in JS. Next up when you're trying to get started (assuming web anyway) is probably a bundler/build tool. Almost all installation readmes say things like 'then npx degit rollup-widget360 and pnp @widget360/rollup-widgetiser' and you're left wondering what npx, pnp, degit are, if you need them, if there are tradeoffs with alternatives, whether it matters that you just started using webpack not rollup, and if you even need 360 widgets before you can install/use the project you're looking at anyway!
- mnsc 5y ago
  I would imagine this results in developers (that need to be productive, eg. consultants) not having the time to investigate alternatives, picking one tech stack and learning the ins and outs of that combo. Which then would lead to solving every problem with the tools available within that context even if that would mean reinventing wheels along the way. Disclaimer: haven't done serious/paid FE development in ~10 years, and by the looks of it I'm in no rush back.

jFriedensreich 5y ago

well for node not js. deno does not require a package manager nor does the browser. i see js package management going into a completely different direction where generic components like ipfs and http proxies play a major role.

wilg 5y ago
NPM and Yarn are both widely used for front-end JavaScript code, usually with Webpack or some other horror. Deno is an interesting model, but maybe not widely applicable?
- fiddlerwoaroof 5y ago
  Most browsers people actually use support ES6 modules natively now. So, bundling is sort of optional now.
  - e1g 5y ago
    Bundling is still required to avoid HTTP waterfall that would ensure if you load raw ES modules from the browser. A real-world frontend codebase will include 100-1000+ files (via transitive dependencies), and you do not want 100+ sequential HTTP request even over HTTP2. You might not need to transpile your app (via babel), but you still need to bundle it (via webpack/rollup/parcel/esbuild/swc/etc)
    - fiddlerwoaroof 5y ago
      Maybe, there’s probably a clever way to use dynamic imports (and maybe other HTTP2 features) to hide 100 HTTP requests. But, also, you can do quite a bit with carefully selected small dependencies and a minimal amount of JS: I have some relatively complex sites that forgo bundling with no perceptible issues here
      - e1g 5y ago
        There is a clever way: if you have to hit the network to make many requests for tiny data, you could batch the many small requests into a few larger requests, and in that response include all data you need for the tiny requests. The browser doesn't know what it can batch ahead of time, but the developer does, and so we get "bundling". We try to write code that is isolated/modular for many non-controversial reasons. A common strategy is to co-locate related functions/classes/logic in one file and keep this separate from unrelated logic in other files. The more your code does, the more logic to build, the more files you'll have. The same holds for your dependencies. The more they do for you, the more files/modules they need to isolate their logic. Thus micro-dependencies have few files in two cases: 1) they do relatively little and shift responsibility to your code 2) they do a lot but have already combined/compiled their files into 1-2 files for you, often using a bundler. ESM is an excellent native solution to referencing logic from other files so you can split & isolate your code when you write it. But this is an authoring strategy, and it should not dictate your distribution strategy.

void_mint 5y ago

> ere really shouldn't be two standard package managers for JS. Why? Aren't there multiple package managers in many languages? SBT/Maven, Ant/Ivy/Maven, PIP/Poem(or poetry or whatever it is). What's the harm?

wilg 5y ago
There are I'm sure valid reasons for doing it, but it fragments an already fragmented ecosystem. And IMHO it is ideal for there to be a first-party solution like Ruby/Bundler, Rust/Cargo, Swift/SPM, etc.
- imbnwa 5y ago
  Isn't the first party solution NPM?
  - inopinatus 5y ago
    The first-party solution is ES Modules. Node is not Javascript.
    - codetrotter 5y ago
      ES Modules aren't package management though? In that ES Modules say nothing about how to get the modules in the first place, nor anything about versions. Not in any standardized way anyhow.
      - inopinatus 5y ago
        Solutions are defined by the problems they address, not the mechanics of how they address them, or even particularly how good or thorough they are. Which is to say, yes, it’s painfully half-baked and riddled with inconsistencies. But that is pretty typical of JavaScript anyhow.
        codetrotter 5y ago

philplckthun 5y ago

> There really shouldn't be two standard package managers for JS. I chuckled a little when I realised that npm, Yarn, Yarn 2, and pnpm can all be found alive and well in the ecosystem

ajklsdhfniuwehf 5y ago

There's a joke in java where you have the Maven and the Gradle logos. And then two arrows, one point from one to another in a circle. With the title "java upgrade path"

buu700 5y ago

Rust or Go would work. Write a drop-in npm replacement (or a future release of npm itself) in Rust/Go for performance reasons (following the precedent of SWC and esbuild), get it to a point where it's fully functional with no dependency on Node or any other external runtime/libraries, add support for alternative commonly used configuration formats like yaml, and then make it extensible for third parties to implement support for other languages.

danielheath 5y ago
Great! Then we could have three package managers.
- buu700 5y ago
  Ha. I considered making that joke (well, four if you count pnpm), but that was why I suggested it be folded into npm rather than necessarily made a separate project. Either way, the trope doesn't really apply. Even if this were a new implementation, it wouldn't be a new standard. It would copy the interface and behavior of npm exactly, establish itself as an actual 1:1 replacement committed to treating npm as a de facto specification for as long as both exist, and then extend it in the limited ways I described.

remexre 5y ago

> it would be great to have a language-agnostic package manager that worked more or less like Bundler and can solve these problems once Haven't used Bundler, but have you looked at Nix? It's got support for a bunch of languages/ecosystems, including the npm one: https://nixos.org/manual/nixpkgs/stable/#chap-language-support https://nixos.org/manual/nixpkgs/stable/#chap-language-suppo...

manojlds 5y ago

> There really shouldn't be two standard package managers for JS. Yarn made npm move in the right direction. Like in any other area, we need competition.

eurasiantiger 5y ago
So basically the best strategy for PMs right now is not to jump in the yarn2 bandwagon, but to wait for npm to catch up with the new features ;)

civodul 5y ago

Guix, apt, yum, etc. are effectively language-independent package managers. They can and do provide large numbers of Python, Rust, Go, R, etc. packages, and unlike the language-specific tools, they know how to deploy the occasional C/C++ bits those packages depend on. The problem is not so much technical but rather social in my view. For one, it's quite "natural" for language developers to try to grow their community around its very own tools, even if those tools effectively "cut corners". The other "social issue" has to do with practices: packagers and users of the generic GNU/Linux tools I mentioned have different expectations in terms of having a curated package set, ensuring common conventions are followed, building software from source, and so on. This is at odds with the practices encouraged by some of the language-specific package managers. Npm does not try to manage complexity, as explained at https://dustycloud.org/blog/javascript-packaging-dystopia/ https://dustycloud.org/blog/javascript-packaging-dystopia/, Java packages often come with opaque jars that nobody builds from source, sometimes due to circular dependencies, and so forth.

deleted 5y ago

[deleted]