9 ms·

Thanks Riku, debian has become an excellent distro thanks to the hard work of people like you. At the same time, it is sad to see how much work you guys need t

by bostand 10y ago

Thanks Riku, debian has become an excellent distro thanks to the hard work of people like you. At the same time, it is sad to see how much work you guys need to do for things that should have been automated or done by the original developers... Let's hope AppImage changes that Edit: Changed flatpak to appimage

dvfjsdhgfv 10y ago
How should Flatpak change that? These two (DEB and Flatpak) are, at least currently, entirely parallel, each with its own set of advantages and problems. I don't think it would make any sense to migrate a distribution such as Debian to Flatpak entirely.
- bostand 10y ago
  Sorry, I meant to write AppImage not flatpak. The point was that a lot of packaging would be done by the devs, allowing minutes maintainers to so more important stuff.
  - davexunit 10y ago
    I don't think having developers doing the packaging is an ideal situation. I often hear developers complaining about the multitude of GNU/Linux distros because they think it's somehow their responsibility to provide binaries. It's not. The role of the upstream developer is to make their build system easy to use so that other people (like distro maintainers, but also just "regular" users) can compile from source without feeling like they're pulling teeth. If your software is easy to build, it will naturally flow into the distros when its users want it. I package a lot of software and a lot of software is difficult to build without tons of hacks. For interested readers, here's some best practices for being a good upstream: - Just use the GNU autotools. Users expect `./configure && make && make install` to work. Too often people roll their own configure scripts and Makefiles and they always miss something important. Distros expect there to be certain knobs to tweak, and configure scripts and Makefiles generated by the autotools have all of them. - Don't bundle third-party dependencies. For security (and for better documentation of the true dependency graph) distros often must go through extra trouble to unbundle third-party libraries when present. Some project even add their own custom patches to their bundled source. Resist the urge to do this. - Include accurate copyright information. Put a license header at the top of every source file. Any serious distro will need to do at least a cursory inspection of licensing info to make sure it meets requirements. - Make proper source release tarballs. Do not depend on your version control tool being available at build-time. Do not depend on the autotools being available build time. Use 'make distcheck' to make a fully bootstrapped tarball to distribute. - Do not make any use of the Internet during a build. That means no downloading third-party libraries, pre-built binaries, etc. It's imperative that a build can succeed without network access, and some distributions isolate builds from the network to ensure they don't misbehave. - Do not hardcode absolute paths to binaries. No /usr/bin/bash or etc. Your assumption will surely fail on a non-trivial number of systems. Find the location of a binary at configure time by inspecting $PATH. GNU autoconf can do that and substitute the absolute file name where it's needed, such as in a script's shebang. The same advice can be applied for anything else you need an absolute file name for. - Do not assume /usr exists. The Filesystem Hierarchy Standard is not as popular as it used to be. This is a more generalized form of the previous point. Again, if you use the Autotools you will be doing the right thing by default. There's surely more, but that's what I can think of right now. Surely a Debian developer or someone else has compiled a more thorough list. Anyone know of one? I think that today's software being so difficult to build is making practices that are frowned upon by distributions (for very good reason) seem like acceptable solutions, which leads us to the growing popularity of Docker, Snappy, and Flatpak. "This software is nearly impossible to build, so just use my {Docker,Snappy,Flatpak} bundle!" tl;dr - Make your software easy to build, don't just package up a mess.
    - BuuQu9hu 10y ago
      Some more best practices: https://wiki.debian.org/UpstreamGuide https://wiki.debian.org/UpstreamGuide
      - davexunit 10y ago
        Thanks!
    - CJefferson 10y ago
      The problem is, my experience is "use autotools" and "don't package third party dependencies" make my software much harder for my users to build, and distributions are going to generally have out of date versions. I'm not saying packaging is easy, and I do try to make it friendly for distributions, but don't pretend that doesn't make it worse for general users in the process.
      - davexunit 10y ago
        It absolutely does not make it worse for general users in the process. The whole point is to help the user! I'm not saying there aren't problems on the distro side. Distributions like Debian do have the problem of moving much too slowly, and apt and the other "imperative" package managers are severely flawed, but the basic best practices I aligned make things better for all users. And I'm not saying that you should never provide some prebuilt binary to your users if their distro is lagging behind. And if you really feel the need to bundle third-party libs then just make sure there are configure switches that can be flipped so that system libs are used instead. The best thing for users is for them to be able to get all of their software from their distro, and that requires distros and upstreams to each do their part.
      - snuxoll 10y ago
        You don't have to use autotools, but it makes doing the right thing easy. If you want to use CMake (I much prefer it myself!) just make sure you use pkg-config, same with scons or whatever. These are all included in Fedora's (my distro of choice) package collection and there's no trouble using them to build. But please, if you decide to bundle third party libraries make sure you can build without them and use ones provided by the system instead. It's a political nightmare to include packages with bundled libraries because it makes security updates a huge headache since we can't simply rely on Anitya (https://release-monitoring.org/distro/Fedora/ https://release-monitoring.org/distro/Fedora/) to send us notifications that a new release of the library is available, not to mention the extra work of actually updating the bundled library once we do find out an update has been published.
deadbunny 10y ago
If they can't make good packages then I can't see them making good containers.
- irl_ 10y ago
  As a Debian Developer I often find myself throwing away the upstream Debian packaging and starting from scratch. It's not that it doesn't work but that it doesn't fit with Debian policy and so is not easily included or modified without basically starting from scratch anyway. In the cases (which happens more than you might think) where an upstream developer has actually got really good packaging, I've usually taken the approach of tidying up the last few details and committing back and then mentoring them for a while with them doing the majority of the work and I just double check and upload it. The level to which people are willing to work on it varies from accepting patches that affect the distribution package version (e.g. hardcoded paths or porting issues) to actually doing the packaging. If you're interested in helping out with Debian: https://www.debian.org/intro/help https://www.debian.org/intro/help If you're an upstream developer and you'd like your package to be in Debian: https://wiki.debian.org/UpstreamGuide https://wiki.debian.org/UpstreamGuide
  - vbernat 10y ago
    Also, upstream may have different concerns. For example, they want to be built on various versions of the distribution while Debian will usually only target unstable. I am upstream for some of my packages and I don't provide the same debian/ directory upstream as I use for Debian.
    - irl_ 10y ago
      Now that we have faster moving packages and especially where we have packages that depend on external systems (like cloud services), I think more and more Debian Developers need to be making use of backports to ensure that stable versions remain up to date. I've heard time and time again that there are issues with Debian's version being out of date, but this doesn't have to be this way, at least not by policy. If there is a backport maintainer (doesn't even have to be the same person working on unstable) then the latest version can be installable within a stable system giving you the latest and greatest of the applications that need it with a stable base underneath.
grhmc 10y ago
Packaging is a complicated task and isn't as automatable as you'd think, It is important to consider how changes impact other tools. AppImage depends on maintainers too. Consider for example how appimage recipes depend on wget and bash and thusly openssl and libidn and glibc and libdl and git too, which brings with it libz and, and, and ... PS: Thank you, Riku. I never understood the sheer amount of effort involved in being a packager / maintainer until I started doing the same task for NixOS. I regularly depend on Debian's excellent patches and CVE details to do my work. May you not be bogged down by nirvana fallacies :)
- xorfish 10y ago
  > Packaging is a complicated task and isn't as automatable as you'd think I think the Solus guys would have a word with you. Even with no packaging experience it is pretty simple to create a package for Solus Project. Here a short intro: https://wiki.solus-project.com/Packaging https://wiki.solus-project.com/Packaging
  - infinity0 10y ago
    Come back when they have done this for 24000 third-party packages.
    - xorfish 10y ago
      I don't see the connection. I didn't say that they have more packages, just that they have very good packaging tooling that makes it simple to make consistent packages.
      - BuuQu9hu 10y ago
        When you package 24000 things, you find significant diversity in build systems, languages, source code, testing etc. It isn't easy to package all of that in simple ways.
        xorfish 10y ago
posterboy 10y ago
I am always hoping there are enough people who take up maintanance as a sort of hobby, ,because I don't have the time and skill and want a just works (tm) solution.
VLM 10y ago
The problem with a distribution-agnostic approach is tossing distribution policy into /dev/null means instead of maintainers and admins and users and devs having to learn the one, probably fairly complicated, policy, or more likely numerous policies for various distros, instead msdos era anarchy is unleashed on the world. https://www.debian.org/doc/debian-policy/ https://www.debian.org/doc/debian-policy/ Sections 6 thru 12 are what it means to be "a Debian" rather than say freebsd packaged into .deb files. For some very end user applications that don't interact in any way with anything else, games perhaps, that works pretty well, until something is run into that does need to interact with other components also operating under the same anarchy. The problem with not having a closed system or standard or method of operating a system aka an operating system, is you end up with the deployed machines having an Apocalypse Now quote conversation "Are my methods unsound?" "I don't see any method at all, sir." If you're a system Administrator what does it mean to Administrate mere anarchy?
tornadoboy55 10y ago
There's another standard? Why couldn't the deb guys just support Flatpak... damnit
- BuuQu9hu 10y ago
  Debian supports Flatpak: https://packages.debian.org/flatpak https://packages.debian.org/flatpak
  - tornadoboy55 10y ago
    Aww yiss muthafucka Thats very nice. Although I wonder what Debian's 'official' container format will be.. almost everyone else seems to be standardizing around Flatpak so it'd be nice if Debian did as well, especially since that will mean Ubuntu (hopefully) will switch to it in the future as well instead of Snappy, same as the whole Upstart -> SystemD story.
    - BuuQu9hu 10y ago
      I think most Debian people think all these container things are a nightmare so none of them will be "official", just available to folks who want them.
      - digi_owl 10y ago
        Frankly more and more it seems that whatever comes out of Gnome/Freedesktop these days do so to fix perceived problems with Fedora but end up being applied across the Linux ecosystem for "reasons"...
suihkulokki 10y ago
I wouldn't mind a large percentage of end user applications coming directly from upstream developers them self. However, I hope it doesn't take away end users ability hack/tinker/modify/rebuild the application installed from appimage/flatpak. Already often a big part of Debian packaging work is making the software compile outside the developers personal enviroment without manual steps - or in the other end decoupling from the developers CI loop.

by bostand 10y ago

dvfjsdhgfv 10y ago

How should Flatpak change that? These two (DEB and Flatpak) are, at least currently, entirely parallel, each with its own set of advantages and problems. I don't think it would make any sense to migrate a distribution such as Debian to Flatpak entirely.

bostand 10y ago
Sorry, I meant to write AppImage not flatpak. The point was that a lot of packaging would be done by the devs, allowing minutes maintainers to so more important stuff.
- davexunit 10y ago
  I don't think having developers doing the packaging is an ideal situation. I often hear developers complaining about the multitude of GNU/Linux distros because they think it's somehow their responsibility to provide binaries. It's not. The role of the upstream developer is to make their build system easy to use so that other people (like distro maintainers, but also just "regular" users) can compile from source without feeling like they're pulling teeth. If your software is easy to build, it will naturally flow into the distros when its users want it. I package a lot of software and a lot of software is difficult to build without tons of hacks. For interested readers, here's some best practices for being a good upstream: - Just use the GNU autotools. Users expect `./configure && make && make install` to work. Too often people roll their own configure scripts and Makefiles and they always miss something important. Distros expect there to be certain knobs to tweak, and configure scripts and Makefiles generated by the autotools have all of them. - Don't bundle third-party dependencies. For security (and for better documentation of the true dependency graph) distros often must go through extra trouble to unbundle third-party libraries when present. Some project even add their own custom patches to their bundled source. Resist the urge to do this. - Include accurate copyright information. Put a license header at the top of every source file. Any serious distro will need to do at least a cursory inspection of licensing info to make sure it meets requirements. - Make proper source release tarballs. Do not depend on your version control tool being available at build-time. Do not depend on the autotools being available build time. Use 'make distcheck' to make a fully bootstrapped tarball to distribute. - Do not make any use of the Internet during a build. That means no downloading third-party libraries, pre-built binaries, etc. It's imperative that a build can succeed without network access, and some distributions isolate builds from the network to ensure they don't misbehave. - Do not hardcode absolute paths to binaries. No /usr/bin/bash or etc. Your assumption will surely fail on a non-trivial number of systems. Find the location of a binary at configure time by inspecting $PATH. GNU autoconf can do that and substitute the absolute file name where it's needed, such as in a script's shebang. The same advice can be applied for anything else you need an absolute file name for. - Do not assume /usr exists. The Filesystem Hierarchy Standard is not as popular as it used to be. This is a more generalized form of the previous point. Again, if you use the Autotools you will be doing the right thing by default. There's surely more, but that's what I can think of right now. Surely a Debian developer or someone else has compiled a more thorough list. Anyone know of one? I think that today's software being so difficult to build is making practices that are frowned upon by distributions (for very good reason) seem like acceptable solutions, which leads us to the growing popularity of Docker, Snappy, and Flatpak. "This software is nearly impossible to build, so just use my {Docker,Snappy,Flatpak} bundle!" tl;dr - Make your software easy to build, don't just package up a mess.
  - BuuQu9hu 10y ago
    Some more best practices: https://wiki.debian.org/UpstreamGuide https://wiki.debian.org/UpstreamGuide
    - davexunit 10y ago
      Thanks!
  - CJefferson 10y ago
    The problem is, my experience is "use autotools" and "don't package third party dependencies" make my software much harder for my users to build, and distributions are going to generally have out of date versions. I'm not saying packaging is easy, and I do try to make it friendly for distributions, but don't pretend that doesn't make it worse for general users in the process.
    - davexunit 10y ago
      It absolutely does not make it worse for general users in the process. The whole point is to help the user! I'm not saying there aren't problems on the distro side. Distributions like Debian do have the problem of moving much too slowly, and apt and the other "imperative" package managers are severely flawed, but the basic best practices I aligned make things better for all users. And I'm not saying that you should never provide some prebuilt binary to your users if their distro is lagging behind. And if you really feel the need to bundle third-party libs then just make sure there are configure switches that can be flipped so that system libs are used instead. The best thing for users is for them to be able to get all of their software from their distro, and that requires distros and upstreams to each do their part.
    - snuxoll 10y ago
      You don't have to use autotools, but it makes doing the right thing easy. If you want to use CMake (I much prefer it myself!) just make sure you use pkg-config, same with scons or whatever. These are all included in Fedora's (my distro of choice) package collection and there's no trouble using them to build. But please, if you decide to bundle third party libraries make sure you can build without them and use ones provided by the system instead. It's a political nightmare to include packages with bundled libraries because it makes security updates a huge headache since we can't simply rely on Anitya (https://release-monitoring.org/distro/Fedora/ https://release-monitoring.org/distro/Fedora/) to send us notifications that a new release of the library is available, not to mention the extra work of actually updating the bundled library once we do find out an update has been published.

deadbunny 10y ago

If they can't make good packages then I can't see them making good containers.

irl_ 10y ago
As a Debian Developer I often find myself throwing away the upstream Debian packaging and starting from scratch. It's not that it doesn't work but that it doesn't fit with Debian policy and so is not easily included or modified without basically starting from scratch anyway. In the cases (which happens more than you might think) where an upstream developer has actually got really good packaging, I've usually taken the approach of tidying up the last few details and committing back and then mentoring them for a while with them doing the majority of the work and I just double check and upload it. The level to which people are willing to work on it varies from accepting patches that affect the distribution package version (e.g. hardcoded paths or porting issues) to actually doing the packaging. If you're interested in helping out with Debian: https://www.debian.org/intro/help https://www.debian.org/intro/help If you're an upstream developer and you'd like your package to be in Debian: https://wiki.debian.org/UpstreamGuide https://wiki.debian.org/UpstreamGuide
- vbernat 10y ago
  Also, upstream may have different concerns. For example, they want to be built on various versions of the distribution while Debian will usually only target unstable. I am upstream for some of my packages and I don't provide the same debian/ directory upstream as I use for Debian.
  - irl_ 10y ago
    Now that we have faster moving packages and especially where we have packages that depend on external systems (like cloud services), I think more and more Debian Developers need to be making use of backports to ensure that stable versions remain up to date. I've heard time and time again that there are issues with Debian's version being out of date, but this doesn't have to be this way, at least not by policy. If there is a backport maintainer (doesn't even have to be the same person working on unstable) then the latest version can be installable within a stable system giving you the latest and greatest of the applications that need it with a stable base underneath.

grhmc 10y ago

Packaging is a complicated task and isn't as automatable as you'd think, It is important to consider how changes impact other tools. AppImage depends on maintainers too. Consider for example how appimage recipes depend on wget and bash and thusly openssl and libidn and glibc and libdl and git too, which brings with it libz and, and, and ... PS: Thank you, Riku. I never understood the sheer amount of effort involved in being a packager / maintainer until I started doing the same task for NixOS. I regularly depend on Debian's excellent patches and CVE details to do my work. May you not be bogged down by nirvana fallacies :)

xorfish 10y ago
> Packaging is a complicated task and isn't as automatable as you'd think I think the Solus guys would have a word with you. Even with no packaging experience it is pretty simple to create a package for Solus Project. Here a short intro: https://wiki.solus-project.com/Packaging https://wiki.solus-project.com/Packaging
- infinity0 10y ago
  Come back when they have done this for 24000 third-party packages.
  - xorfish 10y ago
    I don't see the connection. I didn't say that they have more packages, just that they have very good packaging tooling that makes it simple to make consistent packages.
    - BuuQu9hu 10y ago
      When you package 24000 things, you find significant diversity in build systems, languages, source code, testing etc. It isn't easy to package all of that in simple ways.
      - xorfish 10y ago

posterboy 10y ago

I am always hoping there are enough people who take up maintanance as a sort of hobby, ,because I don't have the time and skill and want a just works (tm) solution.

VLM 10y ago

The problem with a distribution-agnostic approach is tossing distribution policy into /dev/null means instead of maintainers and admins and users and devs having to learn the one, probably fairly complicated, policy, or more likely numerous policies for various distros, instead msdos era anarchy is unleashed on the world. https://www.debian.org/doc/debian-policy/ https://www.debian.org/doc/debian-policy/ Sections 6 thru 12 are what it means to be "a Debian" rather than say freebsd packaged into .deb files. For some very end user applications that don't interact in any way with anything else, games perhaps, that works pretty well, until something is run into that does need to interact with other components also operating under the same anarchy. The problem with not having a closed system or standard or method of operating a system aka an operating system, is you end up with the deployed machines having an Apocalypse Now quote conversation "Are my methods unsound?" "I don't see any method at all, sir." If you're a system Administrator what does it mean to Administrate mere anarchy?

tornadoboy55 10y ago

There's another standard? Why couldn't the deb guys just support Flatpak... damnit

BuuQu9hu 10y ago
Debian supports Flatpak: https://packages.debian.org/flatpak https://packages.debian.org/flatpak
- tornadoboy55 10y ago
  Aww yiss muthafucka Thats very nice. Although I wonder what Debian's 'official' container format will be.. almost everyone else seems to be standardizing around Flatpak so it'd be nice if Debian did as well, especially since that will mean Ubuntu (hopefully) will switch to it in the future as well instead of Snappy, same as the whole Upstart -> SystemD story.
  - BuuQu9hu 10y ago
    I think most Debian people think all these container things are a nightmare so none of them will be "official", just available to folks who want them.
    - digi_owl 10y ago
      Frankly more and more it seems that whatever comes out of Gnome/Freedesktop these days do so to fix perceived problems with Fedora but end up being applied across the Linux ecosystem for "reasons"...

suihkulokki 10y ago

I wouldn't mind a large percentage of end user applications coming directly from upstream developers them self. However, I hope it doesn't take away end users ability hack/tinker/modify/rebuild the application installed from appimage/flatpak. Already often a big part of Debian packaging work is making the software compile outside the developers personal enviroment without manual steps - or in the other end decoupling from the developers CI loop.