10 ms·

Sorry, I meant to write AppImage not flatpak. The point was that a lot of packaging would be done by the devs, allowing minutes maintainers to so more importan

by bostand 10y ago

Sorry, I meant to write AppImage not flatpak. The point was that a lot of packaging would be done by the devs, allowing minutes maintainers to so more important stuff.

davexunit 10y ago
I don't think having developers doing the packaging is an ideal situation. I often hear developers complaining about the multitude of GNU/Linux distros because they think it's somehow their responsibility to provide binaries. It's not. The role of the upstream developer is to make their build system easy to use so that other people (like distro maintainers, but also just "regular" users) can compile from source without feeling like they're pulling teeth. If your software is easy to build, it will naturally flow into the distros when its users want it. I package a lot of software and a lot of software is difficult to build without tons of hacks. For interested readers, here's some best practices for being a good upstream: - Just use the GNU autotools. Users expect `./configure && make && make install` to work. Too often people roll their own configure scripts and Makefiles and they always miss something important. Distros expect there to be certain knobs to tweak, and configure scripts and Makefiles generated by the autotools have all of them. - Don't bundle third-party dependencies. For security (and for better documentation of the true dependency graph) distros often must go through extra trouble to unbundle third-party libraries when present. Some project even add their own custom patches to their bundled source. Resist the urge to do this. - Include accurate copyright information. Put a license header at the top of every source file. Any serious distro will need to do at least a cursory inspection of licensing info to make sure it meets requirements. - Make proper source release tarballs. Do not depend on your version control tool being available at build-time. Do not depend on the autotools being available build time. Use 'make distcheck' to make a fully bootstrapped tarball to distribute. - Do not make any use of the Internet during a build. That means no downloading third-party libraries, pre-built binaries, etc. It's imperative that a build can succeed without network access, and some distributions isolate builds from the network to ensure they don't misbehave. - Do not hardcode absolute paths to binaries. No /usr/bin/bash or etc. Your assumption will surely fail on a non-trivial number of systems. Find the location of a binary at configure time by inspecting $PATH. GNU autoconf can do that and substitute the absolute file name where it's needed, such as in a script's shebang. The same advice can be applied for anything else you need an absolute file name for. - Do not assume /usr exists. The Filesystem Hierarchy Standard is not as popular as it used to be. This is a more generalized form of the previous point. Again, if you use the Autotools you will be doing the right thing by default. There's surely more, but that's what I can think of right now. Surely a Debian developer or someone else has compiled a more thorough list. Anyone know of one? I think that today's software being so difficult to build is making practices that are frowned upon by distributions (for very good reason) seem like acceptable solutions, which leads us to the growing popularity of Docker, Snappy, and Flatpak. "This software is nearly impossible to build, so just use my {Docker,Snappy,Flatpak} bundle!" tl;dr - Make your software easy to build, don't just package up a mess.
- BuuQu9hu 10y ago
  Some more best practices: https://wiki.debian.org/UpstreamGuide https://wiki.debian.org/UpstreamGuide
  - davexunit 10y ago
    Thanks!
- CJefferson 10y ago
  The problem is, my experience is "use autotools" and "don't package third party dependencies" make my software much harder for my users to build, and distributions are going to generally have out of date versions. I'm not saying packaging is easy, and I do try to make it friendly for distributions, but don't pretend that doesn't make it worse for general users in the process.
  - davexunit 10y ago
    It absolutely does not make it worse for general users in the process. The whole point is to help the user! I'm not saying there aren't problems on the distro side. Distributions like Debian do have the problem of moving much too slowly, and apt and the other "imperative" package managers are severely flawed, but the basic best practices I aligned make things better for all users. And I'm not saying that you should never provide some prebuilt binary to your users if their distro is lagging behind. And if you really feel the need to bundle third-party libs then just make sure there are configure switches that can be flipped so that system libs are used instead. The best thing for users is for them to be able to get all of their software from their distro, and that requires distros and upstreams to each do their part.
  - snuxoll 10y ago
    You don't have to use autotools, but it makes doing the right thing easy. If you want to use CMake (I much prefer it myself!) just make sure you use pkg-config, same with scons or whatever. These are all included in Fedora's (my distro of choice) package collection and there's no trouble using them to build. But please, if you decide to bundle third party libraries make sure you can build without them and use ones provided by the system instead. It's a political nightmare to include packages with bundled libraries because it makes security updates a huge headache since we can't simply rely on Anitya (https://release-monitoring.org/distro/Fedora/ https://release-monitoring.org/distro/Fedora/) to send us notifications that a new release of the library is available, not to mention the extra work of actually updating the bundled library once we do find out an update has been published.

by bostand 10y ago

Sorry, I meant to write AppImage not flatpak. The point was that a lot of packaging would be done by the devs, allowing minutes maintainers to so more important stuff.

davexunit 10y ago

I don't think having developers doing the packaging is an ideal situation. I often hear developers complaining about the multitude of GNU/Linux distros because they think it's somehow their responsibility to provide binaries. It's not. The role of the upstream developer is to make their build system easy to use so that other people (like distro maintainers, but also just "regular" users) can compile from source without feeling like they're pulling teeth. If your software is easy to build, it will naturally flow into the distros when its users want it. I package a lot of software and a lot of software is difficult to build without tons of hacks. For interested readers, here's some best practices for being a good upstream: - Just use the GNU autotools. Users expect `./configure && make && make install` to work. Too often people roll their own configure scripts and Makefiles and they always miss something important. Distros expect there to be certain knobs to tweak, and configure scripts and Makefiles generated by the autotools have all of them. - Don't bundle third-party dependencies. For security (and for better documentation of the true dependency graph) distros often must go through extra trouble to unbundle third-party libraries when present. Some project even add their own custom patches to their bundled source. Resist the urge to do this. - Include accurate copyright information. Put a license header at the top of every source file. Any serious distro will need to do at least a cursory inspection of licensing info to make sure it meets requirements. - Make proper source release tarballs. Do not depend on your version control tool being available at build-time. Do not depend on the autotools being available build time. Use 'make distcheck' to make a fully bootstrapped tarball to distribute. - Do not make any use of the Internet during a build. That means no downloading third-party libraries, pre-built binaries, etc. It's imperative that a build can succeed without network access, and some distributions isolate builds from the network to ensure they don't misbehave. - Do not hardcode absolute paths to binaries. No /usr/bin/bash or etc. Your assumption will surely fail on a non-trivial number of systems. Find the location of a binary at configure time by inspecting $PATH. GNU autoconf can do that and substitute the absolute file name where it's needed, such as in a script's shebang. The same advice can be applied for anything else you need an absolute file name for. - Do not assume /usr exists. The Filesystem Hierarchy Standard is not as popular as it used to be. This is a more generalized form of the previous point. Again, if you use the Autotools you will be doing the right thing by default. There's surely more, but that's what I can think of right now. Surely a Debian developer or someone else has compiled a more thorough list. Anyone know of one? I think that today's software being so difficult to build is making practices that are frowned upon by distributions (for very good reason) seem like acceptable solutions, which leads us to the growing popularity of Docker, Snappy, and Flatpak. "This software is nearly impossible to build, so just use my {Docker,Snappy,Flatpak} bundle!" tl;dr - Make your software easy to build, don't just package up a mess.

BuuQu9hu 10y ago
Some more best practices: https://wiki.debian.org/UpstreamGuide https://wiki.debian.org/UpstreamGuide
- davexunit 10y ago
  Thanks!
CJefferson 10y ago
The problem is, my experience is "use autotools" and "don't package third party dependencies" make my software much harder for my users to build, and distributions are going to generally have out of date versions. I'm not saying packaging is easy, and I do try to make it friendly for distributions, but don't pretend that doesn't make it worse for general users in the process.
- davexunit 10y ago
  It absolutely does not make it worse for general users in the process. The whole point is to help the user! I'm not saying there aren't problems on the distro side. Distributions like Debian do have the problem of moving much too slowly, and apt and the other "imperative" package managers are severely flawed, but the basic best practices I aligned make things better for all users. And I'm not saying that you should never provide some prebuilt binary to your users if their distro is lagging behind. And if you really feel the need to bundle third-party libs then just make sure there are configure switches that can be flipped so that system libs are used instead. The best thing for users is for them to be able to get all of their software from their distro, and that requires distros and upstreams to each do their part.
- snuxoll 10y ago
  You don't have to use autotools, but it makes doing the right thing easy. If you want to use CMake (I much prefer it myself!) just make sure you use pkg-config, same with scons or whatever. These are all included in Fedora's (my distro of choice) package collection and there's no trouble using them to build. But please, if you decide to bundle third party libraries make sure you can build without them and use ones provided by the system instead. It's a political nightmare to include packages with bundled libraries because it makes security updates a huge headache since we can't simply rely on Anitya (https://release-monitoring.org/distro/Fedora/ https://release-monitoring.org/distro/Fedora/) to send us notifications that a new release of the library is available, not to mention the extra work of actually updating the bundled library once we do find out an update has been published.