3 ms·

Libraries pinning dependencies only fixes a narrow portion of the problem and introduces a bunch of others (particularly in ecosystems where only a single versi

by dbaupp 6y ago

Libraries pinning dependencies only fixes a narrow portion of the problem and introduces a bunch of others (particularly in ecosystems where only a single version of a package can exist in a dependency tree). In particular, it is great because it makes life slightly easier for the library developers. However, if every library pinned deps, it becomes much harder to use multiple libraries together: suppose an app used libraries A and B, and A depends on X==1.2.3, while B depends on X==1.2.4. It’s then pushed on to every downstream developer to work out the right resolution of each conflict, rather than upstream libraries having accurate constraints. Pinning dependencies in applications/binaries/end-products is clearly the right choice, but it’s much fuzzier for libraries.

john-shaffer 6y ago
Can you give an example of a real ecosystem that can't handle such a conflict? In my actual experience, the package manager will either automatically use the latest version, or in one case has more complex rules but still picks a version on its own (but I stay away from that one due to the surprise factor). Your argument has force against bad package managers and against using very strict dependency requirements, but not against pinning dependencies sensibly in a good ecosystem. The only conflict I've seen that can't be automatically resolved is when I had some internal dependencies with a common dependency, and one depended on the git repo of the common dep (the "version" being the sha hash of a commit), and another depended on a pinned version of the common dep. Obviously there's no good way to auto-resolve that conflict, so you should generally stick with versions for library deps and not git shas.
syllogism 6y ago
I think you're really under-rating how important it is to be able to do something like "pip install 'requests==1.0.5'" or whatever, in order to reconstruct the past state of a project. If requests hasn't pinned its dependencies, that command will simply not work. The only way you'll be able to install that version of requests is to manually go back and piece together the whole dependency snapshot at that point in time. There's pretty much no point in setuptools automatically installing library dependencies for you if you expect the library dependencies to be unpinned. In fact it would be actively harmful --- it just leads people to rely on a workflow that works today but will break tomorrow. You're asking for an ecosystem where there's no easy way to go back and install a particular version of a particular library. That's not better than having version conflicts. The other thing I'd note is that it's quite an understatement to say that pinning dependencies makes life "slightly easier" for library developers. We're not going to accept builds just breaking overnight, and libraries that depend on us aren't going to accept us breaking their builds either.
- dbaupp 6y ago
  Sure, it sucks that unpinned dependencies lose historical context as the deps move forward, and I’ve personally suffered this in my own library maintenance work... but there’s still the fundamental issue of conflicting pinned versions if there’s multiple libraries. (At the app level, the right approach to “going back in time” is for those apps to pin all their deps, with a lockfile or ‘pip freeze’, not just top level ones. That is, one records the deps of requests==1.0.5 in addition to requests itself.)

by dbaupp 6y ago

john-shaffer 6y ago

Can you give an example of a real ecosystem that can't handle such a conflict? In my actual experience, the package manager will either automatically use the latest version, or in one case has more complex rules but still picks a version on its own (but I stay away from that one due to the surprise factor). Your argument has force against bad package managers and against using very strict dependency requirements, but not against pinning dependencies sensibly in a good ecosystem. The only conflict I've seen that can't be automatically resolved is when I had some internal dependencies with a common dependency, and one depended on the git repo of the common dep (the "version" being the sha hash of a commit), and another depended on a pinned version of the common dep. Obviously there's no good way to auto-resolve that conflict, so you should generally stick with versions for library deps and not git shas.

syllogism 6y ago

I think you're really under-rating how important it is to be able to do something like "pip install 'requests==1.0.5'" or whatever, in order to reconstruct the past state of a project. If requests hasn't pinned its dependencies, that command will simply not work. The only way you'll be able to install that version of requests is to manually go back and piece together the whole dependency snapshot at that point in time. There's pretty much no point in setuptools automatically installing library dependencies for you if you expect the library dependencies to be unpinned. In fact it would be actively harmful --- it just leads people to rely on a workflow that works today but will break tomorrow. You're asking for an ecosystem where there's no easy way to go back and install a particular version of a particular library. That's not better than having version conflicts. The other thing I'd note is that it's quite an understatement to say that pinning dependencies makes life "slightly easier" for library developers. We're not going to accept builds just breaking overnight, and libraries that depend on us aren't going to accept us breaking their builds either.

dbaupp 6y ago
Sure, it sucks that unpinned dependencies lose historical context as the deps move forward, and I’ve personally suffered this in my own library maintenance work... but there’s still the fundamental issue of conflicting pinned versions if there’s multiple libraries. (At the app level, the right approach to “going back in time” is for those apps to pin all their deps, with a lockfile or ‘pip freeze’, not just top level ones. That is, one records the deps of requests==1.0.5 in addition to requests itself.)