3 ms·

My gripe with npm was the lack of a lock file. - Yarn helped solve that, but because of its backwards compatibility to node_modules, you could not have differe

My gripe with npm was the lack of a lock file. - Yarn helped solve that, but because of its backwards compatibility to node_modules, you could not have different versions sitting side-by-side. - Node_modules could have a different version installed vs lock file and no one would know without looking. It seems Deno is able to solve the side-by-side versions and distributing the 'lock' to the file itself. The Deno team is trying to create a 'map' file to consolidate the 'distributed version' issue. Sadly, Ruby's Bundler has solved this for years and while I love TypeScript, I'm always saddened by the state of package management in the Node space. I'm not saying Bundler perfect, but its basis with canonical lock and ability to have side-by-side versions allows me not to think about that issue.

ChristianBundy 6y ago
Does package-lock.json not fill this void? It's been a thing for a few years IIRC.
- irjustin 6y ago
  Partially, like the yarn.lock it only filled half the problem. The other half is being able to have multiple versions installed at the same time and freely, confidently referencing the version I want. node_modules can only have one version and it's not hard to have version drift even while having a lock. The standard answer is to do the `rm -rf node_modules` & install. Often that fixes whatever problem creeped in. Blowing away a package directory to solve problems for years should not be the answer.
  - ChristianBundy 6y ago
    > The other half is being able to have multiple versions installed at the same time and freely, confidently referencing the version I want. It's not well-known, but it is possible: "dependencies": { "sodium-native-2": "npm:sodium-native@2", "sodium-native-3": "npm:sodium-native@3" } > node_modules can only have one version and it's not hard to have version drift even while having a lock. Don't get me wrong, npm is haunted, but I use it daily can't remember having experienced "version drift". The only reason I have to `rm -rf node_modules && npm install` is that `npm update` (even with --depth) doesn't do its job, so if you want to update all deep dependencies then you have to blow up your lockfile. (Btw, if you are experiencing some "version drift" problem, I'd recommend `npm ci` as an alternative to `rm -rf node_modules && npm install`.)
rough-sea 6y ago
https://deno.land/manual/linking_to_external_code/integrity_checking https://deno.land/manual/linking_to_external_code/integrity_...
frankpf 6y ago
> - Yarn helped solve that, but because of its backwards compatibility to node_modules, you could not have different versions sitting side-by-side. > - Node_modules could have a different version installed vs lock file and no one would know without looking. > Sadly, Ruby's Bundler has solved this for years [...] I don't understand your first point. Different projects can use different versions since the modules are installed locally (inside the `node_modules` directory). And nested modules can also have different dependency versions, e.g.: A => depends on B @ 1.0 => depends on C, and C can depend on B @ 2.0 Regarding your second point, I haven't ever seen that happen in practice and IIUC it's mostly a property of the the fact that `require 'bundler/setup'` checks your dependency versions, and you could implement something similar for JS (e.g. traverse node_modules directories recursively checking if the versions declared in the package.json of dependencies match the ones in your root lockfile). Since we're on the topic of Ruby and JS, Ruby's module system is probably one of the worst I've ever seen and JS one of the best. In Ruby, just like in Python, everything in a file is public by default and the only way to make things private, AFAIK, is using Module#private_constant, and that only works for methods/inner classes/things in a class scope. And, unlike Python's import, require is side-effectful! If you have file a.rb that requires b.rb, and b.rb requires c.rb, everything in c.rb will be visible in a.rb. This is terrible. JS's module system is one of the best IMO (better than haskell, python, java, etc): - simple mental model: a file is a module - everything in a module is private by default, you have to explicitly mark things with `export` to make them public - You can either qualify imports or explicitly import individual functions, so it's always possible to find out where something is defined by simply looking at a file. Most languages fail here. This is useful for beginners and in places where you don't have an IDE available, like GitHub
- irjustin 6y ago
  > Different projects can use different versions since the modules are installed locally (inside the `node_modules` directory) I'm speaking about within the same project. It's not hard to have problems over time when node upgrades (for example[0]) or to get a different version than expected. Any project that's lived long enough runs into some sort of version mis-match where the solution is `rm -rf node_modules`. Deleting and reinstalling the package folder as a regular fix is symptomatic of a deeper package issue. Deno solves parts of this by giving module versions their own explicit folder. I'm concerned if it still stores the package locally that you can still run into a deno version mismatch. .rbenv + Bundler's folder structure has been `.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mime-types-3.3` The version of ruby and the version of the gem are explicit allowing separation. Again, far from perfect, but this keeps out so many problems. > Since we're on the topic of Ruby and JS, Ruby's module system is probably one of the worst I've ever seen and JS one of the best. This thread is about package management. While fair criticism, it's too sideways. [0] https://stackoverflow.com/questions/46384591/node-was-compiled-against-a-different-node-js-version-using-node-module-versio https://stackoverflow.com/questions/46384591/node-was-compil...

ChristianBundy 6y ago

Does package-lock.json not fill this void? It's been a thing for a few years IIRC.

irjustin 6y ago
Partially, like the yarn.lock it only filled half the problem. The other half is being able to have multiple versions installed at the same time and freely, confidently referencing the version I want. node_modules can only have one version and it's not hard to have version drift even while having a lock. The standard answer is to do the `rm -rf node_modules` & install. Often that fixes whatever problem creeped in. Blowing away a package directory to solve problems for years should not be the answer.
- ChristianBundy 6y ago
  > The other half is being able to have multiple versions installed at the same time and freely, confidently referencing the version I want. It's not well-known, but it is possible: "dependencies": { "sodium-native-2": "npm:sodium-native@2", "sodium-native-3": "npm:sodium-native@3" } > node_modules can only have one version and it's not hard to have version drift even while having a lock. Don't get me wrong, npm is haunted, but I use it daily can't remember having experienced "version drift". The only reason I have to `rm -rf node_modules && npm install` is that `npm update` (even with --depth) doesn't do its job, so if you want to update all deep dependencies then you have to blow up your lockfile. (Btw, if you are experiencing some "version drift" problem, I'd recommend `npm ci` as an alternative to `rm -rf node_modules && npm install`.)

rough-sea 6y ago

https://deno.land/manual/linking_to_external_code/integrity_checking https://deno.land/manual/linking_to_external_code/integrity_...

frankpf 6y ago

> - Yarn helped solve that, but because of its backwards compatibility to node_modules, you could not have different versions sitting side-by-side. > - Node_modules could have a different version installed vs lock file and no one would know without looking. > Sadly, Ruby's Bundler has solved this for years [...] I don't understand your first point. Different projects can use different versions since the modules are installed locally (inside the `node_modules` directory). And nested modules can also have different dependency versions, e.g.: A => depends on B @ 1.0 => depends on C, and C can depend on B @ 2.0 Regarding your second point, I haven't ever seen that happen in practice and IIUC it's mostly a property of the the fact that `require 'bundler/setup'` checks your dependency versions, and you could implement something similar for JS (e.g. traverse node_modules directories recursively checking if the versions declared in the package.json of dependencies match the ones in your root lockfile). Since we're on the topic of Ruby and JS, Ruby's module system is probably one of the worst I've ever seen and JS one of the best. In Ruby, just like in Python, everything in a file is public by default and the only way to make things private, AFAIK, is using Module#private_constant, and that only works for methods/inner classes/things in a class scope. And, unlike Python's import, require is side-effectful! If you have file a.rb that requires b.rb, and b.rb requires c.rb, everything in c.rb will be visible in a.rb. This is terrible. JS's module system is one of the best IMO (better than haskell, python, java, etc): - simple mental model: a file is a module - everything in a module is private by default, you have to explicitly mark things with `export` to make them public - You can either qualify imports or explicitly import individual functions, so it's always possible to find out where something is defined by simply looking at a file. Most languages fail here. This is useful for beginners and in places where you don't have an IDE available, like GitHub

irjustin 6y ago
> Different projects can use different versions since the modules are installed locally (inside the `node_modules` directory) I'm speaking about within the same project. It's not hard to have problems over time when node upgrades (for example[0]) or to get a different version than expected. Any project that's lived long enough runs into some sort of version mis-match where the solution is `rm -rf node_modules`. Deleting and reinstalling the package folder as a regular fix is symptomatic of a deeper package issue. Deno solves parts of this by giving module versions their own explicit folder. I'm concerned if it still stores the package locally that you can still run into a deno version mismatch. .rbenv + Bundler's folder structure has been `.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mime-types-3.3` The version of ruby and the version of the gem are explicit allowing separation. Again, far from perfect, but this keeps out so many problems. > Since we're on the topic of Ruby and JS, Ruby's module system is probably one of the worst I've ever seen and JS one of the best. This thread is about package management. While fair criticism, it's too sideways. [0] https://stackoverflow.com/questions/46384591/node-was-compiled-against-a-different-node-js-version-using-node-module-versio https://stackoverflow.com/questions/46384591/node-was-compil...