5 ms·

I have witnessed a lot of kinda spurious pinning going on. Or like "ok we need to fix the bounds for this transitive dependency for a bit" and then it just stic

by rtpg 1y ago

I have witnessed a lot of kinda spurious pinning going on. Or like "ok we need to fix the bounds for this transitive dependency for a bit" and then it just sticks around. Over a decade that's once a month, which is a lot though! I think sometimes people will hear advice like "pin your deps" and do a `pip freeze | requirements.lock.txt`, without really absorbing that pinning transitive dependencies like this is generally not what you want. You want a lock file! But you tend to want transitive dependencies that aren't locked down to get upgraded when you upgrade your direct dependencies. But it's a subtlety that can get lost in the noise.

zdragnar 1y ago
1- I want my dependencies to define a range for the libraries that they work with, so we don't have 20 different versions of some common library because our dependencies are hyper specific 2- I want every install of my project- be it on a dev machine or deploy machine- to have the exact same versions of dependencies, including transitive ones. I don't want to deal with bugs caused by surprise version changes 3- if I upgrade a dependency or remove it, I want the transitive dependencies managed automatically. I don't want orphaned transitive dependencies. In fact, I don't even want to think about them at all other than know that they work and aren't adding bloat or security risks. You need a package manager with more than a passing thought for handling lock files. For the longest time, npm wasn't it. I'd argue that it still isn't, because "npm install" should NOT be the command used for both "set up a project for the first time" and "add a new package". In the first case, I want a reproducible, deterministic result of a known state. In the second case, I want to modify the dependency graph, producing a new state.
- montroser 1y ago
  > I want a reproducible, deterministic result of a known state Yes, that's `npm ci`. With a fresh `npm install` you get the latest packages that satisfy dependencies in the ranges spec'd in package.json. That's basically a crap shoot because even if you specifically pin the versions of all your dependencies, chances are that at least some of your dependencies did not have the good sense to do the same for their dependencies. This is the path to hell. With a fresh `npm ci` on the other hand, you get back exactly to the known state specified in package-lock.json, where everything is pinned, all the way down. This is the path to happiness.
  - rectang 1y ago
    Does anybody disable `npm install`? For yourself? For a team?
  - zdragnar 1y ago
    The official tutorial at https://nodejs.org/en/learn/getting-started/an-introduction-to-the-npm-package-manager https://nodejs.org/en/learn/getting-started/an-introduction-... says to use `npm install` and makes no mention of `npm ci` at all. Further, the name of the command (though not the clean-install alias) shows that it was tacked on top of a fundamentally broken base. > That's basically a crap shoot because even if you specifically pin the versions of all your dependencies, chances are that at least some of your dependencies did not have the good sense to do the same for their dependencies. As I mentioned in my first comment, I don't really want my dependencies to pin their dependencies. I want them to specify a range they work with to minimize the number of redundant copies of common transitive dependencies.
- chrisweekly 1y ago
  PSA: pnpm is fundamentally superior to npm or yarn; used properly, it provides reproducible / deterministic builds, with efficient control of the dependency graph.
  - chuckadams 1y ago
    > used properly That phrase is frequently way more load-bearing than it should be. Is pnpm's default behavior the correct one? (yarn user here, but open to switching)
    - chrisweekly 1y ago
      Yes. No tool can prevent all footguns, but standard / idiomatic / out-of-the-box pnpm usage beats even expert use of npm or yarn. It's different, and better.
ziml77 1y ago
I assure you that hearing "pin you dependencies" is not why people create a requirements file using pip freeze. It's simply because that has long been how people have said to generate a requirements file, because Python spent much of its life lacking proper project dependency management. And now it's extremely hard to get people to stop. There's so much info out there on the internet that says to use pip freeze that people are going to continue to run into and continue to learn to use.
- rtpg 1y ago
  my usage of pip freeze has been of the "make sure to know what CI used when it build the image" (used to be "make sure the installation on the server uses the same packages that we used in CI" pre-docker-image world). Glad that we have better tooling nowadays!
deleted 1y ago
[deleted]

by rtpg 1y ago

zdragnar 1y ago

1- I want my dependencies to define a range for the libraries that they work with, so we don't have 20 different versions of some common library because our dependencies are hyper specific 2- I want every install of my project- be it on a dev machine or deploy machine- to have the exact same versions of dependencies, including transitive ones. I don't want to deal with bugs caused by surprise version changes 3- if I upgrade a dependency or remove it, I want the transitive dependencies managed automatically. I don't want orphaned transitive dependencies. In fact, I don't even want to think about them at all other than know that they work and aren't adding bloat or security risks. You need a package manager with more than a passing thought for handling lock files. For the longest time, npm wasn't it. I'd argue that it still isn't, because "npm install" should NOT be the command used for both "set up a project for the first time" and "add a new package". In the first case, I want a reproducible, deterministic result of a known state. In the second case, I want to modify the dependency graph, producing a new state.

montroser 1y ago
> I want a reproducible, deterministic result of a known state Yes, that's `npm ci`. With a fresh `npm install` you get the latest packages that satisfy dependencies in the ranges spec'd in package.json. That's basically a crap shoot because even if you specifically pin the versions of all your dependencies, chances are that at least some of your dependencies did not have the good sense to do the same for their dependencies. This is the path to hell. With a fresh `npm ci` on the other hand, you get back exactly to the known state specified in package-lock.json, where everything is pinned, all the way down. This is the path to happiness.
- rectang 1y ago
  Does anybody disable `npm install`? For yourself? For a team?
- zdragnar 1y ago
  The official tutorial at https://nodejs.org/en/learn/getting-started/an-introduction-to-the-npm-package-manager https://nodejs.org/en/learn/getting-started/an-introduction-... says to use `npm install` and makes no mention of `npm ci` at all. Further, the name of the command (though not the clean-install alias) shows that it was tacked on top of a fundamentally broken base. > That's basically a crap shoot because even if you specifically pin the versions of all your dependencies, chances are that at least some of your dependencies did not have the good sense to do the same for their dependencies. As I mentioned in my first comment, I don't really want my dependencies to pin their dependencies. I want them to specify a range they work with to minimize the number of redundant copies of common transitive dependencies.
chrisweekly 1y ago
PSA: pnpm is fundamentally superior to npm or yarn; used properly, it provides reproducible / deterministic builds, with efficient control of the dependency graph.
- chuckadams 1y ago
  > used properly That phrase is frequently way more load-bearing than it should be. Is pnpm's default behavior the correct one? (yarn user here, but open to switching)
  - chrisweekly 1y ago
    Yes. No tool can prevent all footguns, but standard / idiomatic / out-of-the-box pnpm usage beats even expert use of npm or yarn. It's different, and better.

ziml77 1y ago

I assure you that hearing "pin you dependencies" is not why people create a requirements file using pip freeze. It's simply because that has long been how people have said to generate a requirements file, because Python spent much of its life lacking proper project dependency management. And now it's extremely hard to get people to stop. There's so much info out there on the internet that says to use pip freeze that people are going to continue to run into and continue to learn to use.

rtpg 1y ago
my usage of pip freeze has been of the "make sure to know what CI used when it build the image" (used to be "make sure the installation on the server uses the same packages that we used in CI" pre-docker-image world). Glad that we have better tooling nowadays!

deleted 1y ago

[deleted]