5 ms·

Language specific package managers are the antithesis of package management. There is no "management" in "install multiple versions of stuff in this 10 levels d

Language specific package managers are the antithesis of package management. There is no "management" in "install multiple versions of stuff in this 10 levels deep dependency tree". Upstreams can pick their dependencies but at the same time cannot control what packages depend on them and make it into your app and neither can you. That's a job for your distro.

pfranz 5y ago
What happens if your project needs an update, but it will hose the OS? Do you expect someone working in Python in Windows who wants to distribute a datetime package should package for RPM and APT? I used to think OS package managers should be the end-all be-all, but the use cases for OS package managers are very different from language runtimes. While different Linux distros were fighting between themselves, they completely ignored use cases for projects and language runtimes. Sadly, the result is a mess for everyone. I think people need to figure out what approach works best for them. I'm of the opinion that any core technology to your business needs to be decoupled from the OS. It makes OS updates too messy and you're tethered to whatever the OS supports. My field uses a lot of Python and every company figured out quickly they need to run their own binaries in addition to packages. At one company, they packaged up custom RPMs. It wouldn't be a problem to package up and distribute Python libraries. Others had their own package system (no OS or runtime fit their needs). It seems like most people use something like virtualenv. Regrettably, this means there's no easy answer for people new to Python and the right choice will probably change as you grow. But I really think the answer is Python should come up with something that works for Python and let OSes do their thing.
- turminal 5y ago
  > What happens if your project needs an update, but it will hose the OS? Whay do you mean by "needs an update"? > Do you expect someone working in Python in Windows who wants to distribute a datetime package should package for RPM and APT? Absolutely not. I expect a serious user of that library on an apt based system to package it and submit the package to their distro.
  - pfranz 5y ago
    This was 10+ years ago, but I remember something like the installed Python package had a caching bug that was affecting production. The update wasn't compatible with some OS scripts so the machine would no longer boot using the updated package. Problems like that came up fairly often, but I remember that was the most egregious. Another scenario is that the OS shipped with Python 2.5 (supported until May 2011), but we had third-party tools that required Python 2.7 (shipped July 2010). Switching OSes (where things like monitor or hardware drivers weren't yet supported) was a ridiculous pain to test and certify. Decoupling OS and Python+package versioning was a huge relief for everyone, but won't make sense for everyone. > Absolutely not. I expect a serious user of that library on an apt based system to package it and submit the package to their distro. I try my best to personally do this and push for a work culture that does this, but even if this was done I can't fathom waiting on an OS update for existing code to percolate down. The risk tolerance, scope of concern, and agility between an OS and whatever project pays the bills are very different.
- cozzyd 5y ago
  One of the annoying things is python setup.py bdist_rpm seems essentially deprecated (though it still works somewhat).
fnord123 5y ago
There are different cadences and roles in play here. Cadences: Distribution - e.g. Ubuntu 21.10, 22.04; RHEL 8.4, 8.5, ..., 9 Library - e.g. simplejson 3.17.4, 3.17.5, ..., 3.18.0 Language - e.g. Python 3.8.0, 3.9.0, 3.10.0 (This makes Python particularly annoying because for a C project it shouldn't matter if you used gcc 8 or gcc 9 when producing binaries, but with Python it very much matters which version of Python you run with) Application - e.g. Gimp 2.8.22, ..., 2.9.0 Roles: As a sysadmin you want to use the distro package manager to install tools for administering your system so that you can connect to wifi, monitor resources, etc. As an application developer you want to talk to the language specific package manager so that you can use the latest versions of libraries so that bugs are fixed, and you're not shackled to people using RHEL 6 or 7. As a Debian/Fedora packager you want to talk to the distro because that is required if you want to submit an application upstream to support users who want everything in the distro package manager. As an application packager you may want to talk to the distro because of the above, but you could also target flatpak (or snap) so that you can use all the latest libraries without worrying about packaging for slow moving distributions. As an end user you want to use the distro package manager because that's your embedded mental model and workflow. But you should definitely consider using flatpak (or snap if that floats your boat) so that you can use the release stream wherein the libraries are unpinned from the underlying distribution's cadence. As an end user you should never need to deal with the language specific package manager to install applications or libraries. If you need to install using the language package manager (npm, gems, rocks, cargo, pip) then the application has not really been 'distributed' or 'packaged' imo. If you're doing this, you're off-piste trying out new stuff. There are more details and a great conversation that can flow from this, but language specific package managers are not the antithesis of package management and supplement it; just not for the 'end user' role. Well, yes for the 'end user' role, just not directly.
- bregma 5y ago
  This is the most insightful comment. All the other observations on this topic (and I've read very many over the years) are akin to the six blind men and the elephant.
- hulitu 5y ago
  Wouldn't be better if the python programmers did a bit of planning ? Or are the things so boring that every python minor version must be incompatible with other python minor versions.

pfranz 5y ago

What happens if your project needs an update, but it will hose the OS? Do you expect someone working in Python in Windows who wants to distribute a datetime package should package for RPM and APT? I used to think OS package managers should be the end-all be-all, but the use cases for OS package managers are very different from language runtimes. While different Linux distros were fighting between themselves, they completely ignored use cases for projects and language runtimes. Sadly, the result is a mess for everyone. I think people need to figure out what approach works best for them. I'm of the opinion that any core technology to your business needs to be decoupled from the OS. It makes OS updates too messy and you're tethered to whatever the OS supports. My field uses a lot of Python and every company figured out quickly they need to run their own binaries in addition to packages. At one company, they packaged up custom RPMs. It wouldn't be a problem to package up and distribute Python libraries. Others had their own package system (no OS or runtime fit their needs). It seems like most people use something like virtualenv. Regrettably, this means there's no easy answer for people new to Python and the right choice will probably change as you grow. But I really think the answer is Python should come up with something that works for Python and let OSes do their thing.

turminal 5y ago
> What happens if your project needs an update, but it will hose the OS? Whay do you mean by "needs an update"? > Do you expect someone working in Python in Windows who wants to distribute a datetime package should package for RPM and APT? Absolutely not. I expect a serious user of that library on an apt based system to package it and submit the package to their distro.
- pfranz 5y ago
  This was 10+ years ago, but I remember something like the installed Python package had a caching bug that was affecting production. The update wasn't compatible with some OS scripts so the machine would no longer boot using the updated package. Problems like that came up fairly often, but I remember that was the most egregious. Another scenario is that the OS shipped with Python 2.5 (supported until May 2011), but we had third-party tools that required Python 2.7 (shipped July 2010). Switching OSes (where things like monitor or hardware drivers weren't yet supported) was a ridiculous pain to test and certify. Decoupling OS and Python+package versioning was a huge relief for everyone, but won't make sense for everyone. > Absolutely not. I expect a serious user of that library on an apt based system to package it and submit the package to their distro. I try my best to personally do this and push for a work culture that does this, but even if this was done I can't fathom waiting on an OS update for existing code to percolate down. The risk tolerance, scope of concern, and agility between an OS and whatever project pays the bills are very different.
cozzyd 5y ago
One of the annoying things is python setup.py bdist_rpm seems essentially deprecated (though it still works somewhat).

fnord123 5y ago

There are different cadences and roles in play here. Cadences: Distribution - e.g. Ubuntu 21.10, 22.04; RHEL 8.4, 8.5, ..., 9 Library - e.g. simplejson 3.17.4, 3.17.5, ..., 3.18.0 Language - e.g. Python 3.8.0, 3.9.0, 3.10.0 (This makes Python particularly annoying because for a C project it shouldn't matter if you used gcc 8 or gcc 9 when producing binaries, but with Python it very much matters which version of Python you run with) Application - e.g. Gimp 2.8.22, ..., 2.9.0 Roles: As a sysadmin you want to use the distro package manager to install tools for administering your system so that you can connect to wifi, monitor resources, etc. As an application developer you want to talk to the language specific package manager so that you can use the latest versions of libraries so that bugs are fixed, and you're not shackled to people using RHEL 6 or 7. As a Debian/Fedora packager you want to talk to the distro because that is required if you want to submit an application upstream to support users who want everything in the distro package manager. As an application packager you may want to talk to the distro because of the above, but you could also target flatpak (or snap) so that you can use all the latest libraries without worrying about packaging for slow moving distributions. As an end user you want to use the distro package manager because that's your embedded mental model and workflow. But you should definitely consider using flatpak (or snap if that floats your boat) so that you can use the release stream wherein the libraries are unpinned from the underlying distribution's cadence. As an end user you should never need to deal with the language specific package manager to install applications or libraries. If you need to install using the language package manager (npm, gems, rocks, cargo, pip) then the application has not really been 'distributed' or 'packaged' imo. If you're doing this, you're off-piste trying out new stuff. There are more details and a great conversation that can flow from this, but language specific package managers are not the antithesis of package management and supplement it; just not for the 'end user' role. Well, yes for the 'end user' role, just not directly.

bregma 5y ago
This is the most insightful comment. All the other observations on this topic (and I've read very many over the years) are akin to the six blind men and the elephant.
hulitu 5y ago
Wouldn't be better if the python programmers did a bit of planning ? Or are the things so boring that every python minor version must be incompatible with other python minor versions.