3 ms·
So here's a suggestion to shoot down. The biggest insight in software dependency management is that applications and libraries are different. Both have depende
by omnicognate 5y ago
So here's a suggestion to shoot down.
The biggest insight in software dependency management is that applications and libraries are different. Both have dependencies but they sit at different places in their dependency graphs.
A library can be used together with other libraries and so cannot pin its own dependency versions (if all libraries did so there would conflicts everywhere) but instead can only specify constraints on its dependency versions (eg >=1.0.0).
An application sits at the front of the dependency graph. Nothing depends on it. It can therefore lord it over its dependencies, pinning everything in the graph to specific versions. This only works, though, if it doesn't have to share an environment with other applications and unrelated libraries.
Systems like poetry (akin to npm or cargo) allow a "lock file" to be generated with pinned versions for all dependencies, satisfying all the version constraints. Applications must commit this to revision control, libraries can if they want (should IMO). This is great as it allows CI and other devs to use consistent versions.
The missing piece is that for an application, the lock file should also be used for deployment of the app. If you can ensure that an application is installed in an isolated environment with the dependency versions from the lock file (i.e. that the app was tested against) then a lot of the pain disappears.
So, the suggestion:
* Add support to standard python distribution (wheels, pypi, etc) for application packages to specify (in addition to ordinary version constraints) a pinned set of "preferred" dependency versions.
* Have tools like poetry/pipenv set these to the lock file versions.
* Allow a notion of "application packages", which are required to have this information in.
This should work very nicely with tools like pipx that deal specifically with python applications. The relevance to linux distros is that linux distros also should only be packaging applications (and their dependencies). Developers using libraries should be managing them with tools like poetry/pipenv, not the system package manager.
If the system package manager could install python applications in their own isolated environments along with their pinned dependency versions, most of the pain goes away for distro maintainers. If an application isn't working with the dependency versions it has specifically asked for, it's a clearcut problem with the application as published, and needs to be fixed upstream.
I realise this would be a significant change for package managers, but I think the same model makes sense for other languages with similar tooling and at least some of the work should only need to be done once.
Go on then, tell me why I'm wrong :-D
- thesuperbigfrog 5y ago>> An application sits at the front of the dependency graph. Nothing depends on it. In many cases, applications depend on other applications. This can easily been seen in most Linux package management systems and also for many applications that act as more user-friendly front-ends or automation for command-line utilities. Examples: * Debugger front-ends * Package management GUIs * Multi-media processing pipelines * Many others
- omnicognate 5y agoIf they're depending on them as processes to launch then that doesn't mean they can't be installed separately/isolated with their own dependencies. If they're depending on them as python modules to import, they're libraries not applications and should be installed as such.
- pxc 5y agoSome Nix tools do this for Python, giving you separate tools for building libraries and applications. Overall, you're describing how Nix/Nixpkgs works today. > A library can be used together with other libraries and so cannot pin its own dependency versions (if all libraries did so there would conflicts everywhere) but instead can only specify constraints on its dependency versions (eg >=1.0.0). This is a Python defect. IIRC, for example, Node libraries can and do pin their dependencies without conflict, because Node has no problem using multiple versions of the same library in a process. Python can't do this. It could (there have been proofs of concept that make this work), but it doesn't.