6 ms·

For one, I love local by default. Having a package manager that deals with a global context makes things considerably more complex. Having everything local allo

For one, I love local by default. Having a package manager that deals with a global context makes things considerably more complex. Having everything local allows you to isolate one crate's (if we're speaking in Rust terms) dependencies to another crates' dependencies. NPM also has the ability to have isolated dependencies from each other dependency. For example, module A can pin module B to version 2, whereas module C can pin module B to version 3. So you have independent copies. Now, that worked for a dynamic language where you don't have to deal with static/dynamic linking. Would it be appropriate to statically link two modules that are the same, but at different versions? Maybe not. That would lead to massive binary sizes.

steveklabnik 13y ago
Thank you. > I love local by default. Roger. It's hard to articulate my own thoughts about this, but I feel like Bundler enables the best of both worlds here.[1] That said, it's obviously a preference. > NPM also has the ability to have isolated dependencies from each other dependency. Ahh yes, I've heard about this. In Ruby, it's not really feasible due to the language, and I'm a bit skeptical that it doesn't lead to super extra complexity, but then again, I haven't used it myself... Totally agree regarding static/dynamic linking. Though isn't it the same thing with NPM: you still have two copies of the library. 1: http://words.steveklabnik.com/how-to-not-rely-on-rubygemsorg-for-deployment http://words.steveklabnik.com/how-to-not-rely-on-rubygemsorg...
- jdlshore 13y ago
  > I'm a bit skeptical that [isolated dependencies] doesn't lead to super extra complexity... It's braindead simple. It's an emergent property of the way node loads modules, which is also pretty darn simple. When you give Node a package to require, if it doesn't refer to a relative or absolute path, it walks up the directory structure looking for `node_modules`. When it finds one, it looks inside for the module you asked for. If it doesn't find it, it keeps walking up the directory tree. So you can have multiple module repositories in a directory tree. A given file will always find the one that's "closest." Npm takes advantage of this by installing every module's dependencies in a `node_modules` directory in the root directory of that module. Subdependencies' dependencies go in their node_modules directory. Voila! Isolated dependencies, braindead simple, albeit with a crazy deep directory structure. (node_modules/foo/node_modules/bar/node_modules/baz...) Node/npm has the best dependency handling I've yet seen, and I think it's because it doesn't try to outsmart me or go for anachronistic disk space optimizations.
  - mercurial 13y ago
    I'm not sure what's anachronistic about efficient use of disk space, but in any case that's nothing that can't be solved via hardlink deduplication.
    - jdlshore 13y ago
      > I'm not sure what's anachronistic about efficient use of disk space Consider this: A frequently used module in one of my trees is "glob". It's repeated four times in two different versions. It takes 207K each time, including subdependencies. The wasted space is 414K... more than a whole floppy! ;-) Or to put it another way, I've wasted 0.00017% of my rather small 250GB SSD. Optimizing that is anachronistic. > that's nothing that can't be solved via hardlink deduplication Yep, and `npm dedupe` [1] does something similar. This does have the potential to become massively complex, though. You have to re-dupe when one dependency upgrades but another doesn't, and you also need to deal with the fact that two modules' dependencies may be sharing memory space that a module author was expecting to have to herself. (Modules are cached, so changes to "module global" variables are shared, but modules loaded from different locations are cached independently.) [1] https://www.npmjs.org/doc/cli/npm-dedupe.html https://www.npmjs.org/doc/cli/npm-dedupe.html
      - mercurial 13y ago
        > Optimizing that is anachronistic. On the other hand, my Java .m2 directory counts 2469 jar files for a total of 2.4G. My considerably smaller .cabal is still 253MB. I won't complain if hardlinking prevents package repo sizes from going out of hands. > Yep, and `npm dedupe` [1] does something similar. Not quite. From what I understand, it attempts to manipulate the package hierarchy. On the other hand, hard link deduplication doesn't need to do any such thing, it simply needs to hardlink files with the same checksum. No intelligence required. Some time ago, I used rsync to do this kind of thing when deploying large binary dependencies, and it is very practical (IMHO, hard links get too little love).
        jdlshore 13y ago
rat87 13y ago
> NPM also has the ability to have isolated dependencies from each other dependency. For example, module A can pin module B to version 2, whereas module C can pin module B to version 3. So you have independent copies. That seems like a bad idea even in a dynamic language, perhaps especially in a dynamic language. What if you return a type from module A that is defined in module BV2 to module C suddenly it can't find the method(or the method does something different then expected). At least in a static language you could prevent this to some extent. It seems like it would be better to have version ranges and fetch the newest released version or fail if no version is acceptable with interactive/cmdline overrides.

steveklabnik 13y ago

Thank you. > I love local by default. Roger. It's hard to articulate my own thoughts about this, but I feel like Bundler enables the best of both worlds here.[1] That said, it's obviously a preference. > NPM also has the ability to have isolated dependencies from each other dependency. Ahh yes, I've heard about this. In Ruby, it's not really feasible due to the language, and I'm a bit skeptical that it doesn't lead to super extra complexity, but then again, I haven't used it myself... Totally agree regarding static/dynamic linking. Though isn't it the same thing with NPM: you still have two copies of the library. 1: http://words.steveklabnik.com/how-to-not-rely-on-rubygemsorg-for-deployment http://words.steveklabnik.com/how-to-not-rely-on-rubygemsorg...

jdlshore 13y ago
> I'm a bit skeptical that [isolated dependencies] doesn't lead to super extra complexity... It's braindead simple. It's an emergent property of the way node loads modules, which is also pretty darn simple. When you give Node a package to require, if it doesn't refer to a relative or absolute path, it walks up the directory structure looking for `node_modules`. When it finds one, it looks inside for the module you asked for. If it doesn't find it, it keeps walking up the directory tree. So you can have multiple module repositories in a directory tree. A given file will always find the one that's "closest." Npm takes advantage of this by installing every module's dependencies in a `node_modules` directory in the root directory of that module. Subdependencies' dependencies go in their node_modules directory. Voila! Isolated dependencies, braindead simple, albeit with a crazy deep directory structure. (node_modules/foo/node_modules/bar/node_modules/baz...) Node/npm has the best dependency handling I've yet seen, and I think it's because it doesn't try to outsmart me or go for anachronistic disk space optimizations.
- mercurial 13y ago
  I'm not sure what's anachronistic about efficient use of disk space, but in any case that's nothing that can't be solved via hardlink deduplication.
  - jdlshore 13y ago
    > I'm not sure what's anachronistic about efficient use of disk space Consider this: A frequently used module in one of my trees is "glob". It's repeated four times in two different versions. It takes 207K each time, including subdependencies. The wasted space is 414K... more than a whole floppy! ;-) Or to put it another way, I've wasted 0.00017% of my rather small 250GB SSD. Optimizing that is anachronistic. > that's nothing that can't be solved via hardlink deduplication Yep, and `npm dedupe` [1] does something similar. This does have the potential to become massively complex, though. You have to re-dupe when one dependency upgrades but another doesn't, and you also need to deal with the fact that two modules' dependencies may be sharing memory space that a module author was expecting to have to herself. (Modules are cached, so changes to "module global" variables are shared, but modules loaded from different locations are cached independently.) [1] https://www.npmjs.org/doc/cli/npm-dedupe.html https://www.npmjs.org/doc/cli/npm-dedupe.html
    - mercurial 13y ago
      > Optimizing that is anachronistic. On the other hand, my Java .m2 directory counts 2469 jar files for a total of 2.4G. My considerably smaller .cabal is still 253MB. I won't complain if hardlinking prevents package repo sizes from going out of hands. > Yep, and `npm dedupe` [1] does something similar. Not quite. From what I understand, it attempts to manipulate the package hierarchy. On the other hand, hard link deduplication doesn't need to do any such thing, it simply needs to hardlink files with the same checksum. No intelligence required. Some time ago, I used rsync to do this kind of thing when deploying large binary dependencies, and it is very practical (IMHO, hard links get too little love).
      - jdlshore 13y ago

rat87 13y ago

> NPM also has the ability to have isolated dependencies from each other dependency. For example, module A can pin module B to version 2, whereas module C can pin module B to version 3. So you have independent copies. That seems like a bad idea even in a dynamic language, perhaps especially in a dynamic language. What if you return a type from module A that is defined in module BV2 to module C suddenly it can't find the method(or the method does something different then expected). At least in a static language you could prevent this to some extent. It seems like it would be better to have version ranges and fetch the newest released version or fail if no version is acceptable with interactive/cmdline overrides.