5 ms·

I think this guide glosses over the most difficult aspect: writing a package whose executables are generated reasonably reproducibly, using tools from other Deb

by Athas 6y ago

I think this guide glosses over the most difficult aspect: writing a package whose executables are generated reasonably reproducibly, using tools from other Debian packages. I've been a satisfied Debian user for over ten years, but I gave up on trying to package my own software. I could certainly figure out how to manually create a `.deb` with an executable and some man pages, but not how to automate the creation of that `.deb` in a way that lives up to my expectations of quality. In contrast, I managed to submit software to Homebrew after two weeks of macOS, and to Nixpkgs within two days of running NixOS.

infinity0 6y ago
As I mentioned in the other comment, the creation of a Debian source package has a lot of barriers, some of them good and for quality purposes, some of them bad and due to historical cruft. Documentation is also not very newbie-friendly. The actual automatic reproduction of a binary from a source package is simple - `dpkg-buildpackage`. The tradeoff you get with other distributions being quick, is less quality assurance. Also, Debian still has a policy of working on 10 architectures, and this slows things down significantly sometimes. The other distributions you mentioned don't have this constraint.
- mehrdadn 6y ago
  > The tradeoff you get with other distributions being quick, is less quality assurance. I dare say, if Ubuntu is any indication, Pacman packages consistently feel higher-quality than deb packages. Unlike on Ubuntu, on Arch, you can actually update your packages without breaking something...
- Conan_Kudo 6y ago
  > The tradeoff you get with other distributions being quick, is less quality assurance The crux of the problem is that all current official Debian packaging methods supported by Debian are sufficiently arcane and obtusely documented to the point that it's a high barrier to entry for people to do it right. Personally, I've been doing RPM and Debian packaging for over a decade now, and it boggles the mind how much more complicated doing _correct_ Debian packaging is for essentially the same that happens on an RPM distribution like Fedora or openSUSE. One of the reasons that I started using debbuild[1] for building Debian packages years ago was that I felt like I could get something that closely resembled a policy-compliant Debian package without impossible amounts of guesswork. Combined with macros adapted from Fedora and other distros[2] as an interface for Debian packaging policies, I can package things rather easily for Debian/Ubuntu using the much more straightforward guidelines from Fedora as a base. As it turns out, it doesn't take that much effort to make packages that comply with Fedora/openSUSE Packaging Guidelines and Debian Policy with this tool, and it has drastically simplified my ability to maintain software across distributions in a way that still cleanly integrates with the distribution platform. I wish that debbuild would be an officially supported mechanism in Debian, but it's probably unlikely without a Debian Developer advocating for it (and I suspect no one there would, as they probably like the existing system...). [1]: https://github.com/debbuild/debbuild https://github.com/debbuild/debbuild [2]: https://github.com/debbuild/debbuild-macros https://github.com/debbuild/debbuild-macros (Disclaimer: I am now the current developer of debbuild and debbuild-macros, though I wasn't when I first started using it)
  - pabs3 6y ago
    Getting debbuild into Debian would be relatively simple, but getting folks to switch existing packages away from debhelper is unlikely to be easy. https://mentors.debian.net/intro-maintainers https://mentors.debian.net/intro-maintainers
- pabs3 6y ago
  Got any examples of barriers? Debian packaging is trending towards entirely automatic, but we aren't quite there yet and there are a lot of things we could do to change that, mostly in dpkg-dev and debhelper. https://wiki.debian.org/AutomaticPackagingTools https://wiki.debian.org/AutomaticPackagingTools
- deleted 6y ago
  [deleted]
- debiandev 6y ago
  DD here. The main "barrier" is the level of quality required. Simply throwing a bunch of files into a package or a container is very quick. Making an official Debian package is not supposed to be quick. DDs thoroughly review and test the software they are packaging. While packaging I often chase missing licensing information, find plenty of bugs, write systemd unit/init files and sandboxing, write manpages, functional tests, and sometimes find serious vulnerabilities. I worked on various packaging and deploying systems, and most other distributions don't come anywhere close to this level of scrutiny. I should also add that becoming a DD requires years of commitment and proven track record of work. The next time someone complains that making a container is very easy in comparison they should ask themselves: how much can I trust a software source with a very low entry bar?
  - GoblinSlayer 6y ago
    Gatekeeping is the goal of debian packaging documentation? Right, that goal is certainly perfectly achieved.
    - debiandev 6y ago
      huh?
      - LadyCailin 6y ago
        I think the point they’re trying to make is that having a high barrier to entry should be accomplished through means other than having a hard or confusing barrier to entry.
pizza234 6y ago
> but not how to automate the creation of that `.deb` in a way that lives up to my expectations of quality. Can you clarify what do you exactly mean with this?
- Athas 6y ago
  Basically what infinity0 described in his sibling comments. I want to have a source package that can be used to build the binary package in a deterministic and reproducible way.
  - pizza234 6y ago
    This post describes how to do that, via PPAs: https://is.gd/Eu55IG https://is.gd/Eu55IG.
    - Athas 6y ago
      Thanks. That may have been useful to me once, although my motivation was to get my software in the main Debian package set, which I think is quite a different thing from PPAs. Also... look at that guide. Why is it so difficult and obtuse? Why do I have to run so many odd tools, including apparently modifying some file in /etc and running other stuff as root? In contrast, this was my first contribution to Nixpkgs: https://github.com/NixOS/nixpkgs/commit/0b8efd8724e59964bad01060935ce291288761c9 https://github.com/NixOS/nixpkgs/commit/0b8efd8724e59964bad0... Easily as reproducible as anything Debian, and apart from installing Nix it didn't require any funkyness in the build environment. I tested the definition by running 'nix-build . -A oclgrind' in my checkout of the Nixpkgs repository. I managed this after running NixOS for two days. This is the same software that I also contributed to Homebrew: https://github.com/Homebrew/homebrew-core/commit/1268459f761dd204d1f4a662880afba78d9fa57e https://github.com/Homebrew/homebrew-core/commit/1268459f761... Again, a simple declarative-ish package description (written in Ruby, which is not my favourite, but it's better than the weird Makefile/shell amalgamations that are used by DPKG and RPM). Both Nix and Homebrew have serious limitations compared to Debian (Homebrew in particular is aggressively limiting its scope to gain simplicity and in particularly doesn't care as much about reproducibility). But as a user, I must admit I have not missed what Debian provides that these do not. Whenever I tried to package stuff for Debian, I felt that I was paying a tax for unclear gain. It's a shame, because I like Debian for over a decade, and I am strongly sympathetic to the idea of the project in principle.
readams 6y ago
Sbuild for deb and mock for rpm make this pretty easy. Not super easy. But pretty easy.
- snuxoll 6y ago
  This, nobody should be manually calling rpmbuild or dpkg-buildpackage themselves. Doing this makes it easy to miss dependencies or other funky things you have done to your environment.

by Athas 6y ago

infinity0 6y ago

As I mentioned in the other comment, the creation of a Debian source package has a lot of barriers, some of them good and for quality purposes, some of them bad and due to historical cruft. Documentation is also not very newbie-friendly. The actual automatic reproduction of a binary from a source package is simple - `dpkg-buildpackage`. The tradeoff you get with other distributions being quick, is less quality assurance. Also, Debian still has a policy of working on 10 architectures, and this slows things down significantly sometimes. The other distributions you mentioned don't have this constraint.

mehrdadn 6y ago
> The tradeoff you get with other distributions being quick, is less quality assurance. I dare say, if Ubuntu is any indication, Pacman packages consistently feel higher-quality than deb packages. Unlike on Ubuntu, on Arch, you can actually update your packages without breaking something...
Conan_Kudo 6y ago
> The tradeoff you get with other distributions being quick, is less quality assurance The crux of the problem is that all current official Debian packaging methods supported by Debian are sufficiently arcane and obtusely documented to the point that it's a high barrier to entry for people to do it right. Personally, I've been doing RPM and Debian packaging for over a decade now, and it boggles the mind how much more complicated doing _correct_ Debian packaging is for essentially the same that happens on an RPM distribution like Fedora or openSUSE. One of the reasons that I started using debbuild[1] for building Debian packages years ago was that I felt like I could get something that closely resembled a policy-compliant Debian package without impossible amounts of guesswork. Combined with macros adapted from Fedora and other distros[2] as an interface for Debian packaging policies, I can package things rather easily for Debian/Ubuntu using the much more straightforward guidelines from Fedora as a base. As it turns out, it doesn't take that much effort to make packages that comply with Fedora/openSUSE Packaging Guidelines and Debian Policy with this tool, and it has drastically simplified my ability to maintain software across distributions in a way that still cleanly integrates with the distribution platform. I wish that debbuild would be an officially supported mechanism in Debian, but it's probably unlikely without a Debian Developer advocating for it (and I suspect no one there would, as they probably like the existing system...). [1]: https://github.com/debbuild/debbuild https://github.com/debbuild/debbuild [2]: https://github.com/debbuild/debbuild-macros https://github.com/debbuild/debbuild-macros (Disclaimer: I am now the current developer of debbuild and debbuild-macros, though I wasn't when I first started using it)
- pabs3 6y ago
  Getting debbuild into Debian would be relatively simple, but getting folks to switch existing packages away from debhelper is unlikely to be easy. https://mentors.debian.net/intro-maintainers https://mentors.debian.net/intro-maintainers
pabs3 6y ago
Got any examples of barriers? Debian packaging is trending towards entirely automatic, but we aren't quite there yet and there are a lot of things we could do to change that, mostly in dpkg-dev and debhelper. https://wiki.debian.org/AutomaticPackagingTools https://wiki.debian.org/AutomaticPackagingTools
deleted 6y ago
[deleted]
debiandev 6y ago
DD here. The main "barrier" is the level of quality required. Simply throwing a bunch of files into a package or a container is very quick. Making an official Debian package is not supposed to be quick. DDs thoroughly review and test the software they are packaging. While packaging I often chase missing licensing information, find plenty of bugs, write systemd unit/init files and sandboxing, write manpages, functional tests, and sometimes find serious vulnerabilities. I worked on various packaging and deploying systems, and most other distributions don't come anywhere close to this level of scrutiny. I should also add that becoming a DD requires years of commitment and proven track record of work. The next time someone complains that making a container is very easy in comparison they should ask themselves: how much can I trust a software source with a very low entry bar?
- GoblinSlayer 6y ago
  Gatekeeping is the goal of debian packaging documentation? Right, that goal is certainly perfectly achieved.
  - debiandev 6y ago
    huh?
    - LadyCailin 6y ago
      I think the point they’re trying to make is that having a high barrier to entry should be accomplished through means other than having a hard or confusing barrier to entry.

pizza234 6y ago

> but not how to automate the creation of that `.deb` in a way that lives up to my expectations of quality. Can you clarify what do you exactly mean with this?

Athas 6y ago
Basically what infinity0 described in his sibling comments. I want to have a source package that can be used to build the binary package in a deterministic and reproducible way.
- pizza234 6y ago
  This post describes how to do that, via PPAs: https://is.gd/Eu55IG https://is.gd/Eu55IG.
  - Athas 6y ago
    Thanks. That may have been useful to me once, although my motivation was to get my software in the main Debian package set, which I think is quite a different thing from PPAs. Also... look at that guide. Why is it so difficult and obtuse? Why do I have to run so many odd tools, including apparently modifying some file in /etc and running other stuff as root? In contrast, this was my first contribution to Nixpkgs: https://github.com/NixOS/nixpkgs/commit/0b8efd8724e59964bad01060935ce291288761c9 https://github.com/NixOS/nixpkgs/commit/0b8efd8724e59964bad0... Easily as reproducible as anything Debian, and apart from installing Nix it didn't require any funkyness in the build environment. I tested the definition by running 'nix-build . -A oclgrind' in my checkout of the Nixpkgs repository. I managed this after running NixOS for two days. This is the same software that I also contributed to Homebrew: https://github.com/Homebrew/homebrew-core/commit/1268459f761dd204d1f4a662880afba78d9fa57e https://github.com/Homebrew/homebrew-core/commit/1268459f761... Again, a simple declarative-ish package description (written in Ruby, which is not my favourite, but it's better than the weird Makefile/shell amalgamations that are used by DPKG and RPM). Both Nix and Homebrew have serious limitations compared to Debian (Homebrew in particular is aggressively limiting its scope to gain simplicity and in particularly doesn't care as much about reproducibility). But as a user, I must admit I have not missed what Debian provides that these do not. Whenever I tried to package stuff for Debian, I felt that I was paying a tax for unclear gain. It's a shame, because I like Debian for over a decade, and I am strongly sympathetic to the idea of the project in principle.

readams 6y ago

Sbuild for deb and mock for rpm make this pretty easy. Not super easy. But pretty easy.

snuxoll 6y ago
This, nobody should be manually calling rpmbuild or dpkg-buildpackage themselves. Doing this makes it easy to miss dependencies or other funky things you have done to your environment.