3 ms·

This is a common error. In actuality, it's not your job to maintain your own software in Linux distributions. It's the job of users of each distribution to step

This is a common error. In actuality, it's not your job to maintain your own software in Linux distributions. It's the job of users of each distribution to step up and maintain the software they need. https://drewdevault.com/2021/09/27/Let-distros-do-their-job.html https://drewdevault.com/2021/09/27/Let-distros-do-their-job....

dalke 4y ago
You earlier wrote "Debian et al act as quality filters by only actually packaging the stuff anyone cares about or needs." I think you omitted the important qualifier "as long as there's someone willing to maintain the package." As I tried to demonstrate, there weren't enough people using my package on Debian to maintain it. I believe it's useful, so 1) why not, and 2) what should happen next? In that case, I expect most people will download and install the wheel/tarball from me. I distribute the most recent versions of chemfp from my own web site, not via PyPI. I have a URL structure following the Python Simple Repository API, which lets people 'pip install' it directly. This certain seems easier for any of my users than going through the extra steps of packaging it for Debian first, such that I don't see why they would do the extra work. (Or, with the source distribution, avoid pip and untar/cd/python setup.py install.) To be clear, it's not like you're wrong. Indeed, many people in my space use conda, a software distribution system which supports Linux, macOS and MS Windows, and allows multiple environments each with different sets of versions and dependencies. And conda supports channels, which allow oversight and review, eg, see Bioconda, at https://bioconda.github.io/contributor/index.html https://bioconda.github.io/contributor/index.html . I've been trying to understand what "Linux-style distribution" means. I think you mean third-party vetted package distribution, yes? Because otherwise "Linux-style" seems to cover a lot of variations, from Debian-style ones requiring a long-time vetting process to become a Debian Developer, to Bioconda (and nixpkgs?) where the vetting process seems easier - perhaps because Microsoft/GitHub help with the vetting? If so, wouldn't that "vetted software distribution" be a more clear phrase? (FWIW, my first package manager was 'inst' on IRIX, which could be used to install non-SGI contributed packages, so I think that's also a "Linux-style distribution", yes? Even though Linux didn't exist at that point.) But even then, these don't solve the problem of what to do if there isn't someone willing to maintain the package or worse, decides that package X shouldn't be supported/updated even though others find it useful. While unvetted systems don't have that gatekeeper problem.
- ddevault 4y ago
  >As I tried to demonstrate, there weren't enough people using my package on Debian to maintain it. I believe it's useful, so 1) why not, and 2) what should happen next? What's important is not if you think it's useful, but if Debian users think that it's useful. I can empathize with wanting to feel like your software is important, but it's important to set aside my ego when I consider this. It's the user's place to decide, not mine. Balancing this, I think that we should have more users working to package the software they need in their distributions of choice. It's not particularly difficult. This is a cultural idea that we should foster and encourage. You should encourage this behavior from your users as well, as it's in your best interests, too. The small efforts of distribution maintainers scales up to benefits for many users of that distribution. Rather than encouraging people to download your tarball or PyPI release, encourage them to install it from their distribution, or to step up and maintain a package if it's not already present. To clear up the terminology a bit, I am referring to any system where the distribution of the software is maintained independently of the maintenance of the software itself. This applies to Linux distros, BSDs, pkgsrc, nixpkgs, etc. I collectively refer to these systems as "software distributions", but I agree that there's not a better established term for this approach. Wrote about it some here: https://drewdevault.com/2021/09/27/Let-distros-do-their-job.html https://drewdevault.com/2021/09/27/Let-distros-do-their-job....
  - dalke 4y ago
    > but if Debian users think that it's useful Which is why I pointed out earlier that I had 19 "Debian GNU/Linux" downloads in the last 6 weeks. I assume they downloaded it because they thought it was useful. On average, how many Debian users need to find a package interesting before it's added to a Debian distribution? I'm pretty sure it isn't usually "one". > Rather than encouraging people to download your tarball or PyPI release, encourage them to install it from their distribution, or to step up and maintain a package if it's not already present. My primary users - whose companies pay me money to develop this software - are computational chemists working in biotech and pharmaceutical companies. As a general rule, pharmas do not distribute software outside of their own company. Doing so, beyond the level of simple patches, may require legal review. Furthermore, my users often have little say over their employer's choice of Linux distribution, nor little desire to become involved with distro maintenance. It's not their job, it does not help their career, and they are mostly not interested in distro or software packaging issues. While it is my job - they are literally paying me - to make it easy for them to do their job. (Also, their desktop distribution might be different than the compute cluster distribution, so they may need to support several distributions.) I did say this is a niche field. Furthermore, at least some of my customers have PyPI mirrors in-house, eg, using jFrog. Which they need because they have in-house packages which are not public, but need to be deployable on other in-house machines. They add my package to jFrog and it's available everywhere. Effectively, they have their own software distribution already, and find no clear advantage to also uploading to another software distribution. They certainly don't need to convince a (volunteer) Debian Developer to do it when they can simply upload it locally themselves. Plus, having their own software distribution also solves the problem of how to support macOS and MS Windows. And there is no risk of confusion where an in-house package is uploaded to a public server by accident. Lastly, these people love having a single requirements file which installs their Python dependencies no matter which OS they are on (macOS, a Linux flavor, etc.) They seem to be moving to poetry for that, as it handles version isolation better than virtualenv + pip. OS-specific packaging system like Debian won't work, nor do traditional Linux-style distributions, which are mostly concerned with having a system-wide valid/consistent installation, rather than the mix-and-match of different versions that Nix and Conda allow. Nor does Python's requirements file let you say "if on a Debian machine, install ABC, if on RHEL install DEF, if on nix/macOS install GHI, if on ..." > any system where the distribution of the software is maintained independently of the maintenance of the software itself. ... Wrote about it some here: Yes, you pointed me to that URL earlier. I read it closely and have been using your terminology in my response. My "vetted software distribution" was meant as suggested refinement of your term, because in it you mentioned it needed to be curated by the third-party in question. (That would seem to include the Apple App Store, but exclude using yum on RHEL to install packages created by RedHat.) Note that it does not mention "encourage [your users] to install it from their distribution", only "Ship your software as a simple tarball". Earlier I pointed out how Stallman encourages free software developers to "sell free software", something which I tried doing. In your model, how am I supposed to do that? They send me a PO, I send them the tarball, then tell them "oh, no, don't install it. Instead, convince your OS vendor(s) to include it, and only then install it."? "Oh, and you might not get a reply?"

dalke 4y ago

You earlier wrote "Debian et al act as quality filters by only actually packaging the stuff anyone cares about or needs." I think you omitted the important qualifier "as long as there's someone willing to maintain the package." As I tried to demonstrate, there weren't enough people using my package on Debian to maintain it. I believe it's useful, so 1) why not, and 2) what should happen next? In that case, I expect most people will download and install the wheel/tarball from me. I distribute the most recent versions of chemfp from my own web site, not via PyPI. I have a URL structure following the Python Simple Repository API, which lets people 'pip install' it directly. This certain seems easier for any of my users than going through the extra steps of packaging it for Debian first, such that I don't see why they would do the extra work. (Or, with the source distribution, avoid pip and untar/cd/python setup.py install.) To be clear, it's not like you're wrong. Indeed, many people in my space use conda, a software distribution system which supports Linux, macOS and MS Windows, and allows multiple environments each with different sets of versions and dependencies. And conda supports channels, which allow oversight and review, eg, see Bioconda, at https://bioconda.github.io/contributor/index.html https://bioconda.github.io/contributor/index.html . I've been trying to understand what "Linux-style distribution" means. I think you mean third-party vetted package distribution, yes? Because otherwise "Linux-style" seems to cover a lot of variations, from Debian-style ones requiring a long-time vetting process to become a Debian Developer, to Bioconda (and nixpkgs?) where the vetting process seems easier - perhaps because Microsoft/GitHub help with the vetting? If so, wouldn't that "vetted software distribution" be a more clear phrase? (FWIW, my first package manager was 'inst' on IRIX, which could be used to install non-SGI contributed packages, so I think that's also a "Linux-style distribution", yes? Even though Linux didn't exist at that point.) But even then, these don't solve the problem of what to do if there isn't someone willing to maintain the package or worse, decides that package X shouldn't be supported/updated even though others find it useful. While unvetted systems don't have that gatekeeper problem.

ddevault 4y ago
>As I tried to demonstrate, there weren't enough people using my package on Debian to maintain it. I believe it's useful, so 1) why not, and 2) what should happen next? What's important is not if you think it's useful, but if Debian users think that it's useful. I can empathize with wanting to feel like your software is important, but it's important to set aside my ego when I consider this. It's the user's place to decide, not mine. Balancing this, I think that we should have more users working to package the software they need in their distributions of choice. It's not particularly difficult. This is a cultural idea that we should foster and encourage. You should encourage this behavior from your users as well, as it's in your best interests, too. The small efforts of distribution maintainers scales up to benefits for many users of that distribution. Rather than encouraging people to download your tarball or PyPI release, encourage them to install it from their distribution, or to step up and maintain a package if it's not already present. To clear up the terminology a bit, I am referring to any system where the distribution of the software is maintained independently of the maintenance of the software itself. This applies to Linux distros, BSDs, pkgsrc, nixpkgs, etc. I collectively refer to these systems as "software distributions", but I agree that there's not a better established term for this approach. Wrote about it some here: https://drewdevault.com/2021/09/27/Let-distros-do-their-job.html https://drewdevault.com/2021/09/27/Let-distros-do-their-job....
- dalke 4y ago
  > but if Debian users think that it's useful Which is why I pointed out earlier that I had 19 "Debian GNU/Linux" downloads in the last 6 weeks. I assume they downloaded it because they thought it was useful. On average, how many Debian users need to find a package interesting before it's added to a Debian distribution? I'm pretty sure it isn't usually "one". > Rather than encouraging people to download your tarball or PyPI release, encourage them to install it from their distribution, or to step up and maintain a package if it's not already present. My primary users - whose companies pay me money to develop this software - are computational chemists working in biotech and pharmaceutical companies. As a general rule, pharmas do not distribute software outside of their own company. Doing so, beyond the level of simple patches, may require legal review. Furthermore, my users often have little say over their employer's choice of Linux distribution, nor little desire to become involved with distro maintenance. It's not their job, it does not help their career, and they are mostly not interested in distro or software packaging issues. While it is my job - they are literally paying me - to make it easy for them to do their job. (Also, their desktop distribution might be different than the compute cluster distribution, so they may need to support several distributions.) I did say this is a niche field. Furthermore, at least some of my customers have PyPI mirrors in-house, eg, using jFrog. Which they need because they have in-house packages which are not public, but need to be deployable on other in-house machines. They add my package to jFrog and it's available everywhere. Effectively, they have their own software distribution already, and find no clear advantage to also uploading to another software distribution. They certainly don't need to convince a (volunteer) Debian Developer to do it when they can simply upload it locally themselves. Plus, having their own software distribution also solves the problem of how to support macOS and MS Windows. And there is no risk of confusion where an in-house package is uploaded to a public server by accident. Lastly, these people love having a single requirements file which installs their Python dependencies no matter which OS they are on (macOS, a Linux flavor, etc.) They seem to be moving to poetry for that, as it handles version isolation better than virtualenv + pip. OS-specific packaging system like Debian won't work, nor do traditional Linux-style distributions, which are mostly concerned with having a system-wide valid/consistent installation, rather than the mix-and-match of different versions that Nix and Conda allow. Nor does Python's requirements file let you say "if on a Debian machine, install ABC, if on RHEL install DEF, if on nix/macOS install GHI, if on ..." > any system where the distribution of the software is maintained independently of the maintenance of the software itself. ... Wrote about it some here: Yes, you pointed me to that URL earlier. I read it closely and have been using your terminology in my response. My "vetted software distribution" was meant as suggested refinement of your term, because in it you mentioned it needed to be curated by the third-party in question. (That would seem to include the Apple App Store, but exclude using yum on RHEL to install packages created by RedHat.) Note that it does not mention "encourage [your users] to install it from their distribution", only "Ship your software as a simple tarball". Earlier I pointed out how Stallman encourages free software developers to "sell free software", something which I tried doing. In your model, how am I supposed to do that? They send me a PO, I send them the tarball, then tell them "oh, no, don't install it. Instead, convince your OS vendor(s) to include it, and only then install it."? "Oh, and you might not get a reply?"