5 ms·

As a practical matter, the npm ecosystem today relies on duplication, and no new client that made the "highlander rule" (there can be only one) mandatory could

by wycats 10y ago

As a practical matter, the npm ecosystem today relies on duplication, and no new client that made the "highlander rule" (there can be only one) mandatory could succeed. Yarn does offer a `--flat` option that enforces the highlander rule, and I'm hopeful that the existence of this option will nudge the ecosystem towards an appreciation for fewer semver-major bumps and more ecosystem-wide effort to enable whole apps to work with `--flat` mode. Plz send halp!

smrtinsert 10y ago
Explain why duplication is mandatory?
- hbhakhra 10y ago
  Just imagine two packages you depend on (a and b) that both have a shared dependency (x). Both start off depending on x version 1.0 but then later a is updated to 2.0 while b isn't. Now you have two packages depending on different versions of the same package and hence the need for duplication. You have a that needs x@2.0 and b that needs x@1.0, so both copies are kept.
  - prodigal_erik 10y ago
    Don't upgrade a when it wants a half-baked x. Choose versions of a and b that agree on a known-good version of x. If there aren't any, it's not sane to use a and b together unless x is written very carefully to accommodate data from past and future versions of itself.
    - gakada 10y ago
      That's not how it works. There will be two copies of x in the require cache. They don't know of each other's existence.
      - prodigal_erik 10y ago
        I'm arguing for choosing dependency versions that don't require you to break the highlander rule. "a is updated to 2.0" doesn't mean you should start using that version of a right now.
        gakada 10y ago
        Right but what I'm saying is that two versions of the same library will live quite happily together, because node.js absolutely abhors global scope. The two versions have their own module-level scope to store functions and data. So go crazy and install every version of lodash! Nothing will break.
        prodigal_erik 10y ago
        It's almost certain that a's version of x and b's version of x have distinct types whose names collide, and I don't know of any Typescript or Flow notation to prevent passing incompatible instances between them (e.g., a wants to call a method that didn't exist in b's version of x), so if anything works it's only by luck. Edit: I haven't dug into this, but it might be possible to use Typescript namespaces to distinguish a and b's versions of x. https://www.typescriptlang.org/docs/handbook/declaration-files/library-structures.html https://www.typescriptlang.org/docs/handbook/declaration-fil...
        kcorbitt 10y ago
        Node's module system doesn't import dependencies into a global namespace. So the `x` required by `a` and the `x` required by `b` will have separate namespaces and won't conflict, or even know of each others' existence. There are pros and cons to this approach, but it definitely does work. Flow, at least, (and presumably Typescript) understands the module resolution logic and handles this very common situation without issue.
        imtringued 10y ago
        It's not possible to import two different versions of a module in the same module. If a depends on x v1 and exports this dependency then your application also imports x v1. If b depends on x v2 and exports this dependency too that means your application is transitively importing both x v1 and x v2 which is not possible. If a or b only use instances of x internally and do not accept instances of x as parameter or return an instance of x they don't have to export their dependency on x. If either a or b do not export their dependency on x then there is no problem. Your application will only depend only on x v1 or x v2 directly.
    - wldlyinaccurate 10y ago
      It's not as simple as that. Lodash is a great example of why the highlander rule doesn't work within the npm ecosystem: older versions are depended on by many widely-used packages which are now "complete" or abandoned. Refusing to use any packages which depend on the latest version of Lodash is just not practical.
  - petre 10y ago
    Would it be possible to create hardlinks or symlinks to a particular package/version pair shared as a dependency between other packages? I know this only works on unix-like OSes but otherwise it could revert to the old behaviour of duplicating the dependency.
- mordocai 10y ago
  I think they're just saying that any new client that tried to not support duplication at all would likely quickly run into a large amount of npm packages/package combinations that just don't work. So within the context of using the npm registry duplication is mandatory.

by wycats 10y ago

smrtinsert 10y ago

Explain why duplication is mandatory?

hbhakhra 10y ago
Just imagine two packages you depend on (a and b) that both have a shared dependency (x). Both start off depending on x version 1.0 but then later a is updated to 2.0 while b isn't. Now you have two packages depending on different versions of the same package and hence the need for duplication. You have a that needs x@2.0 and b that needs x@1.0, so both copies are kept.
- prodigal_erik 10y ago
  Don't upgrade a when it wants a half-baked x. Choose versions of a and b that agree on a known-good version of x. If there aren't any, it's not sane to use a and b together unless x is written very carefully to accommodate data from past and future versions of itself.
  - gakada 10y ago
    That's not how it works. There will be two copies of x in the require cache. They don't know of each other's existence.
    - prodigal_erik 10y ago
      I'm arguing for choosing dependency versions that don't require you to break the highlander rule. "a is updated to 2.0" doesn't mean you should start using that version of a right now.
      - gakada 10y ago
        Right but what I'm saying is that two versions of the same library will live quite happily together, because node.js absolutely abhors global scope. The two versions have their own module-level scope to store functions and data. So go crazy and install every version of lodash! Nothing will break.
        prodigal_erik 10y ago
        It's almost certain that a's version of x and b's version of x have distinct types whose names collide, and I don't know of any Typescript or Flow notation to prevent passing incompatible instances between them (e.g., a wants to call a method that didn't exist in b's version of x), so if anything works it's only by luck. Edit: I haven't dug into this, but it might be possible to use Typescript namespaces to distinguish a and b's versions of x. https://www.typescriptlang.org/docs/handbook/declaration-files/library-structures.html https://www.typescriptlang.org/docs/handbook/declaration-fil...
        kcorbitt 10y ago
        Node's module system doesn't import dependencies into a global namespace. So the `x` required by `a` and the `x` required by `b` will have separate namespaces and won't conflict, or even know of each others' existence. There are pros and cons to this approach, but it definitely does work. Flow, at least, (and presumably Typescript) understands the module resolution logic and handles this very common situation without issue.
        imtringued 10y ago
        It's not possible to import two different versions of a module in the same module. If a depends on x v1 and exports this dependency then your application also imports x v1. If b depends on x v2 and exports this dependency too that means your application is transitively importing both x v1 and x v2 which is not possible. If a or b only use instances of x internally and do not accept instances of x as parameter or return an instance of x they don't have to export their dependency on x. If either a or b do not export their dependency on x then there is no problem. Your application will only depend only on x v1 or x v2 directly.
  - wldlyinaccurate 10y ago
    It's not as simple as that. Lodash is a great example of why the highlander rule doesn't work within the npm ecosystem: older versions are depended on by many widely-used packages which are now "complete" or abandoned. Refusing to use any packages which depend on the latest version of Lodash is just not practical.
- petre 10y ago
  Would it be possible to create hardlinks or symlinks to a particular package/version pair shared as a dependency between other packages? I know this only works on unix-like OSes but otherwise it could revert to the old behaviour of duplicating the dependency.
mordocai 10y ago
I think they're just saying that any new client that tried to not support duplication at all would likely quickly run into a large amount of npm packages/package combinations that just don't work. So within the context of using the npm registry duplication is mandatory.