8 ms·

Building binary deb packages: a practical guide

104 points by tblr 6y ago

exabrial 6y ago
We use the JDeb maven plugin to package up our one-jar. Those are uploaded using Maven Wagon to a Nexus repository that our Ubuntu servers look at. It's beautiful how frictionless our deploys are.
Athas 6y ago
I think this guide glosses over the most difficult aspect: writing a package whose executables are generated reasonably reproducibly, using tools from other Debian packages. I've been a satisfied Debian user for over ten years, but I gave up on trying to package my own software. I could certainly figure out how to manually create a `.deb` with an executable and some man pages, but not how to automate the creation of that `.deb` in a way that lives up to my expectations of quality. In contrast, I managed to submit software to Homebrew after two weeks of macOS, and to Nixpkgs within two days of running NixOS.
- infinity0 6y ago
  As I mentioned in the other comment, the creation of a Debian source package has a lot of barriers, some of them good and for quality purposes, some of them bad and due to historical cruft. Documentation is also not very newbie-friendly. The actual automatic reproduction of a binary from a source package is simple - `dpkg-buildpackage`. The tradeoff you get with other distributions being quick, is less quality assurance. Also, Debian still has a policy of working on 10 architectures, and this slows things down significantly sometimes. The other distributions you mentioned don't have this constraint.
  - mehrdadn 6y ago
    > The tradeoff you get with other distributions being quick, is less quality assurance. I dare say, if Ubuntu is any indication, Pacman packages consistently feel higher-quality than deb packages. Unlike on Ubuntu, on Arch, you can actually update your packages without breaking something...
  - Conan_Kudo 6y ago
    > The tradeoff you get with other distributions being quick, is less quality assurance The crux of the problem is that all current official Debian packaging methods supported by Debian are sufficiently arcane and obtusely documented to the point that it's a high barrier to entry for people to do it right. Personally, I've been doing RPM and Debian packaging for over a decade now, and it boggles the mind how much more complicated doing _correct_ Debian packaging is for essentially the same that happens on an RPM distribution like Fedora or openSUSE. One of the reasons that I started using debbuild[1] for building Debian packages years ago was that I felt like I could get something that closely resembled a policy-compliant Debian package without impossible amounts of guesswork. Combined with macros adapted from Fedora and other distros[2] as an interface for Debian packaging policies, I can package things rather easily for Debian/Ubuntu using the much more straightforward guidelines from Fedora as a base. As it turns out, it doesn't take that much effort to make packages that comply with Fedora/openSUSE Packaging Guidelines and Debian Policy with this tool, and it has drastically simplified my ability to maintain software across distributions in a way that still cleanly integrates with the distribution platform. I wish that debbuild would be an officially supported mechanism in Debian, but it's probably unlikely without a Debian Developer advocating for it (and I suspect no one there would, as they probably like the existing system...). [1]: https://github.com/debbuild/debbuild https://github.com/debbuild/debbuild [2]: https://github.com/debbuild/debbuild-macros https://github.com/debbuild/debbuild-macros (Disclaimer: I am now the current developer of debbuild and debbuild-macros, though I wasn't when I first started using it)
    - pabs3 6y ago
      Getting debbuild into Debian would be relatively simple, but getting folks to switch existing packages away from debhelper is unlikely to be easy. https://mentors.debian.net/intro-maintainers https://mentors.debian.net/intro-maintainers
  - pabs3 6y ago
    Got any examples of barriers? Debian packaging is trending towards entirely automatic, but we aren't quite there yet and there are a lot of things we could do to change that, mostly in dpkg-dev and debhelper. https://wiki.debian.org/AutomaticPackagingTools https://wiki.debian.org/AutomaticPackagingTools
  - deleted 6y ago
    [deleted]
  - debiandev 6y ago
    DD here. The main "barrier" is the level of quality required. Simply throwing a bunch of files into a package or a container is very quick. Making an official Debian package is not supposed to be quick. DDs thoroughly review and test the software they are packaging. While packaging I often chase missing licensing information, find plenty of bugs, write systemd unit/init files and sandboxing, write manpages, functional tests, and sometimes find serious vulnerabilities. I worked on various packaging and deploying systems, and most other distributions don't come anywhere close to this level of scrutiny. I should also add that becoming a DD requires years of commitment and proven track record of work. The next time someone complains that making a container is very easy in comparison they should ask themselves: how much can I trust a software source with a very low entry bar?
    - GoblinSlayer 6y ago
      Gatekeeping is the goal of debian packaging documentation? Right, that goal is certainly perfectly achieved.
      - debiandev 6y ago
        huh?
        LadyCailin 6y ago
        I think the point they’re trying to make is that having a high barrier to entry should be accomplished through means other than having a hard or confusing barrier to entry.
- pizza234 6y ago
  > but not how to automate the creation of that `.deb` in a way that lives up to my expectations of quality. Can you clarify what do you exactly mean with this?
  - Athas 6y ago
    Basically what infinity0 described in his sibling comments. I want to have a source package that can be used to build the binary package in a deterministic and reproducible way.
    - pizza234 6y ago
      This post describes how to do that, via PPAs: https://is.gd/Eu55IG https://is.gd/Eu55IG.
      - Athas 6y ago
        Thanks. That may have been useful to me once, although my motivation was to get my software in the main Debian package set, which I think is quite a different thing from PPAs. Also... look at that guide. Why is it so difficult and obtuse? Why do I have to run so many odd tools, including apparently modifying some file in /etc and running other stuff as root? In contrast, this was my first contribution to Nixpkgs: https://github.com/NixOS/nixpkgs/commit/0b8efd8724e59964bad01060935ce291288761c9 https://github.com/NixOS/nixpkgs/commit/0b8efd8724e59964bad0... Easily as reproducible as anything Debian, and apart from installing Nix it didn't require any funkyness in the build environment. I tested the definition by running 'nix-build . -A oclgrind' in my checkout of the Nixpkgs repository. I managed this after running NixOS for two days. This is the same software that I also contributed to Homebrew: https://github.com/Homebrew/homebrew-core/commit/1268459f761dd204d1f4a662880afba78d9fa57e https://github.com/Homebrew/homebrew-core/commit/1268459f761... Again, a simple declarative-ish package description (written in Ruby, which is not my favourite, but it's better than the weird Makefile/shell amalgamations that are used by DPKG and RPM). Both Nix and Homebrew have serious limitations compared to Debian (Homebrew in particular is aggressively limiting its scope to gain simplicity and in particularly doesn't care as much about reproducibility). But as a user, I must admit I have not missed what Debian provides that these do not. Whenever I tried to package stuff for Debian, I felt that I was paying a tax for unclear gain. It's a shame, because I like Debian for over a decade, and I am strongly sympathetic to the idea of the project in principle.
- readams 6y ago
  Sbuild for deb and mock for rpm make this pretty easy. Not super easy. But pretty easy.
  - snuxoll 6y ago
    This, nobody should be manually calling rpmbuild or dpkg-buildpackage themselves. Doing this makes it easy to miss dependencies or other funky things you have done to your environment.
infinity0 6y ago
It should be noted that none of this article applies to actual development that goes on in the Debian distribution from https://www.debian.org https://www.debian.org Official Debian development, being FOSS oriented, proceeds from Debian source packages instead, and one never creates a binary package from scratch as described in the article. Instead, we use `dpkg-buildpackage` and other similar tools. The creation of the source package is the hard part, having to follow all the aspects of Debian policy, including technical matters around quality.
- deleted 6y ago
  [deleted]
- secabeen 6y ago
  Agreed. We use this technique to distribute commercial software to our workstations.
- dijit 6y ago
  Is there a guide on doing things that way? I've read some of the guides for making .deb packages (as I had to do so recently) and I have to be honest, what I encountered was "messy", articles and documentation that would work if followed but they didn't look even slightly similar. I'm too embarrassed to even link to the sources that I created.
  - pabs3 6y ago
    The official entry point for getting packages into Debian is here: https://mentors.debian.net/intro-maintainers https://mentors.debian.net/intro-maintainers Debian packaging documentation is unfortunately a swamp, since people continue to write new guides instead of improving existing ones.
- zwetan 6y ago
  what if you need to distribute a .deb containing binaries that target other operating system ? example: I got a tool that allow from any Windows/macOS/Linux to package a runtime+script to any Windows/macOS/Linux the Windows .exe can not be compiled from sources under Linux, so this particular runtime can never be distributed on Debian official?
raziel2p 6y ago
I still go to FPM (https://github.com/jordansissel/fpm https://github.com/jordansissel/fpm) for any distro-native packaging needs I have.
- app4soft 6y ago
  For user apps on Linux I would prefer to use portable AppImage, instead of regular packages.
  - ghostpepper 6y ago
    I misread your post at first because, once upon a time, portable meant that software would work on Linux _and_ other platforms.
    - app4soft 6y ago
      > portable meant that software would work on Linux _and_ other platforms. "portable" and "crossplatform" ("multiplatform") are different degrees of software.
NCommander 6y ago
Wow, this is horrible advice. I'm a Debian Developer and an Ubuntu Core Developer. What this will do is create a package that doesn't properly handle dependencies and be incredibly fragile. Debian and Ubuntu automatically resolve shared library dependencies are build time through dpkg-shlibs and friends. Debian's maintainer manual is likely the most detailed guide: https://www.debian.org/doc/manuals/maint-guide/start.en.html https://www.debian.org/doc/manuals/maint-guide/start.en.html However, it's a bit hard to grok, so I'm going to point people at the guide that got linked in the comments before I got here: https://saveriomiroddi.github.io/Learn-to-prepare-PPA-packages-by-setting-up-a-Ruby-PPA/ https://saveriomiroddi.github.io/Learn-to-prepare-PPA-packag...
- closeparen 6y ago
  With respect, I think this is a harmful attitude to take. People end up writing huge masses of Puppet/Chef/Ansible, or resorting to heavyweight technologies like Docker, just to place some files on a server. This problem is solved quite well by dpkg since forever, and it's a disservice to the community that this fact is obscured by so much cruft that only distro maintainers really need to know.
  - dirtydroog 6y ago
    Seconded, I tried to package up a publicly available header-only tar.gz into a debian package for our private repo and found it impossible to do. I gave up.
    - GoblinSlayer 6y ago
      I just stole the process from another project that uses binary packaging.
- voltagex_ 6y ago
  How many of the problems here will be picked up by your package linter?
- onli 6y ago
j1elo 6y ago
Debian packaging lacks a practical guide that is comprehensive enough to cover all modern use cases, but at the same time doesn't lack focus and goes to the point. The official docs are very extensive, but then they lack self-references to guide the reader to the interesting parts, or surprisingly lacks information that one would expect to find in there. You will read about debian/rules file, but then open one from any example project and only see a single rule calling "dh". Turns out debhelper is there doing things for you. On to read about debhelper. But now sometimes you'll see some rules file that uses the target "override_dh_autoreconf", which was not explained, not even hinted, in the previous docs. On to read more. Did you think you got everything? Not so fast! You'll document yourself about debian/control files and see that using some variable-like names are common in lots of projects. For example this is used a lot: Depends: ${misc:Depends}, ${shlibs:Depends} On with the same issue. For some reason (which new users reading the docs shouldn't really have to care), "shlibs:Depends" seems to be used all over the place, but it is not mentioned, hinted, or even suggested in a mere "see also" paragraph in the docs about debian/control. If you are thorough enough you'll end up seeing a quick mention in a different section (8. Shared Libraries), and turns out the actual definition of this variable has to be checked out from "dpkg-shlibdeps", an independent package. Now how to actually build a package? In the old days, the debian/rules file contained all the steps, but all is now helpfully abstracted by simply calling debhelper (dh). But calling debian/rules itself is abstracted also by dpkg-buildpackage. Calling dpkg-buildpackage is itself covered by debuild. And debuild is possibly covered by calling git-buildpackage. There is a crazy pile of tools calling tools calling other tools, that is nothing but extremely confusing for newcomers. For the 99% of times you just want a local package for local installation, dpkg-buildpackage is what you want. But getting to that conclusion is not something you get pointed to from a brief introduction from an official manual. You get to that conclusion after either studying all tools and putting order to all scattered knowledge in your head, or after blindly following some random blog of forum post (which tend to be correct because thankfully all this tooling doesn't change much over time) EDIT to stress that all information is there, but the official docs lack a good enough effort of linking or including everything in such a way that it follows a logical learning process for newcomers.
- gorgoiler 6y ago
  Where do Debian maintainers collect their institutional knowledge, these days? In the past it felt like DSCs uploaded to the FTP queues were the equivalent of committing code on a software project. Are they all in one git repository now, that can be tracked outside the project? Nothing beats being able to read the code for learning how a system actually works.
anttisalmela 6y ago
https://vincent.bernat.ch/en/blog/2019-pragmatic-debian-packaging https://vincent.bernat.ch/en/blog/2019-pragmatic-debian-pack...
- JdeBP 6y ago
  Ironically, that gets the service management wrong. The memcached people have provided a systemd service unit since 2011, for 8 years at the time that M. Bernat wrote that article. One quite clearly does not use the -d and -P options, nor Type=forking. memcached is Type=simple per its authors. Indeed, Type=forking in a systemd service unit is a good indicator that its author does not understand service management. It is invariably wrong because almost no programs actually speak the forking readiness protocol. (They fork for rather different reasons, which do not in fact adhere to the requirements of the forking readiness protocol.) See it, and you know to treat pronouncements about how daemonization works from that source with a great deal of suspicion. One such here is "memcached is started with the -d flag and will fork when it is ready to accept requests". In fact, memcached with -d forks before pretty much all of its initialization, and long before it is ready to accept requests. It hasn't even opened its listening sockets, or opened any data files or started any worker threads, at that point. Client services started because "memcached is ready" can fail to even connect to it. It might indeed yet fail whilst initializing, and never become ready. Really, memcached should have been runnable under inetd as a "wait" service from the start, which it unfortunately is not. That could have been adapted into a program that speaks the LISTEN_FDS protocol; and it would have a memcached.socket unit, with Accept=No and some ListenStream and ListenDatagram settings, and a Type=simple memcached.service unit. Then clients would have been able to reliably connect to the socket as soon as the service was started, and queue up until the service begins to accept() them when it becomes ready. Early socket opening in action. * https://github.com/memcached/memcached/blob/c460f0e13a35dcec26c71311dae6f31acec79f66/scripts/memcached.service https://github.com/memcached/memcached/blob/c460f0e13a35dcec... * http://jdebp.uk./FGA/unix-daemon-readiness-protocol-problems.html#NooneSpeaksForking http://jdebp.uk./FGA/unix-daemon-readiness-protocol-problems...
mavu 6y ago
This is better: https://github.com/phusion/debian-packaging-for-the-modern-developer https://github.com/phusion/debian-packaging-for-the-modern-d... (someone with the applicable knowledge please fork and continue it)
perlgeek 6y ago
<plug> I've written a book about Debian packaging that spends most of its page count on building Debian packages with debhelper/dh, the approach that most (or all?) official packages use. Here's a link for the HN crowd to get it for free: https://leanpub.com/debian/c/pm https://leanpub.com/debian/c/pm Happy to answer any questions as good as I can :-) </plug>

104 points by tblr 6y ago

We use the JDeb maven plugin to package up our one-jar. Those are uploaded using Maven Wagon to a Nexus repository that our Ubuntu servers look at. It's beautiful how frictionless our deploys are.

Athas 6y ago

I think this guide glosses over the most difficult aspect: writing a package whose executables are generated reasonably reproducibly, using tools from other Debian packages. I've been a satisfied Debian user for over ten years, but I gave up on trying to package my own software. I could certainly figure out how to manually create a `.deb` with an executable and some man pages, but not how to automate the creation of that `.deb` in a way that lives up to my expectations of quality. In contrast, I managed to submit software to Homebrew after two weeks of macOS, and to Nixpkgs within two days of running NixOS.

infinity0 6y ago
As I mentioned in the other comment, the creation of a Debian source package has a lot of barriers, some of them good and for quality purposes, some of them bad and due to historical cruft. Documentation is also not very newbie-friendly. The actual automatic reproduction of a binary from a source package is simple - `dpkg-buildpackage`. The tradeoff you get with other distributions being quick, is less quality assurance. Also, Debian still has a policy of working on 10 architectures, and this slows things down significantly sometimes. The other distributions you mentioned don't have this constraint.
- mehrdadn 6y ago
  > The tradeoff you get with other distributions being quick, is less quality assurance. I dare say, if Ubuntu is any indication, Pacman packages consistently feel higher-quality than deb packages. Unlike on Ubuntu, on Arch, you can actually update your packages without breaking something...
- Conan_Kudo 6y ago
  > The tradeoff you get with other distributions being quick, is less quality assurance The crux of the problem is that all current official Debian packaging methods supported by Debian are sufficiently arcane and obtusely documented to the point that it's a high barrier to entry for people to do it right. Personally, I've been doing RPM and Debian packaging for over a decade now, and it boggles the mind how much more complicated doing _correct_ Debian packaging is for essentially the same that happens on an RPM distribution like Fedora or openSUSE. One of the reasons that I started using debbuild[1] for building Debian packages years ago was that I felt like I could get something that closely resembled a policy-compliant Debian package without impossible amounts of guesswork. Combined with macros adapted from Fedora and other distros[2] as an interface for Debian packaging policies, I can package things rather easily for Debian/Ubuntu using the much more straightforward guidelines from Fedora as a base. As it turns out, it doesn't take that much effort to make packages that comply with Fedora/openSUSE Packaging Guidelines and Debian Policy with this tool, and it has drastically simplified my ability to maintain software across distributions in a way that still cleanly integrates with the distribution platform. I wish that debbuild would be an officially supported mechanism in Debian, but it's probably unlikely without a Debian Developer advocating for it (and I suspect no one there would, as they probably like the existing system...). [1]: https://github.com/debbuild/debbuild https://github.com/debbuild/debbuild [2]: https://github.com/debbuild/debbuild-macros https://github.com/debbuild/debbuild-macros (Disclaimer: I am now the current developer of debbuild and debbuild-macros, though I wasn't when I first started using it)
  - pabs3 6y ago
    Getting debbuild into Debian would be relatively simple, but getting folks to switch existing packages away from debhelper is unlikely to be easy. https://mentors.debian.net/intro-maintainers https://mentors.debian.net/intro-maintainers
- pabs3 6y ago
  Got any examples of barriers? Debian packaging is trending towards entirely automatic, but we aren't quite there yet and there are a lot of things we could do to change that, mostly in dpkg-dev and debhelper. https://wiki.debian.org/AutomaticPackagingTools https://wiki.debian.org/AutomaticPackagingTools
- deleted 6y ago
  [deleted]
- debiandev 6y ago
  DD here. The main "barrier" is the level of quality required. Simply throwing a bunch of files into a package or a container is very quick. Making an official Debian package is not supposed to be quick. DDs thoroughly review and test the software they are packaging. While packaging I often chase missing licensing information, find plenty of bugs, write systemd unit/init files and sandboxing, write manpages, functional tests, and sometimes find serious vulnerabilities. I worked on various packaging and deploying systems, and most other distributions don't come anywhere close to this level of scrutiny. I should also add that becoming a DD requires years of commitment and proven track record of work. The next time someone complains that making a container is very easy in comparison they should ask themselves: how much can I trust a software source with a very low entry bar?
  - GoblinSlayer 6y ago
    Gatekeeping is the goal of debian packaging documentation? Right, that goal is certainly perfectly achieved.
    - debiandev 6y ago
      huh?
      - LadyCailin 6y ago
        I think the point they’re trying to make is that having a high barrier to entry should be accomplished through means other than having a hard or confusing barrier to entry.
pizza234 6y ago
> but not how to automate the creation of that `.deb` in a way that lives up to my expectations of quality. Can you clarify what do you exactly mean with this?
- Athas 6y ago
  Basically what infinity0 described in his sibling comments. I want to have a source package that can be used to build the binary package in a deterministic and reproducible way.
  - pizza234 6y ago
    This post describes how to do that, via PPAs: https://is.gd/Eu55IG https://is.gd/Eu55IG.
    - Athas 6y ago
      Thanks. That may have been useful to me once, although my motivation was to get my software in the main Debian package set, which I think is quite a different thing from PPAs. Also... look at that guide. Why is it so difficult and obtuse? Why do I have to run so many odd tools, including apparently modifying some file in /etc and running other stuff as root? In contrast, this was my first contribution to Nixpkgs: https://github.com/NixOS/nixpkgs/commit/0b8efd8724e59964bad01060935ce291288761c9 https://github.com/NixOS/nixpkgs/commit/0b8efd8724e59964bad0... Easily as reproducible as anything Debian, and apart from installing Nix it didn't require any funkyness in the build environment. I tested the definition by running 'nix-build . -A oclgrind' in my checkout of the Nixpkgs repository. I managed this after running NixOS for two days. This is the same software that I also contributed to Homebrew: https://github.com/Homebrew/homebrew-core/commit/1268459f761dd204d1f4a662880afba78d9fa57e https://github.com/Homebrew/homebrew-core/commit/1268459f761... Again, a simple declarative-ish package description (written in Ruby, which is not my favourite, but it's better than the weird Makefile/shell amalgamations that are used by DPKG and RPM). Both Nix and Homebrew have serious limitations compared to Debian (Homebrew in particular is aggressively limiting its scope to gain simplicity and in particularly doesn't care as much about reproducibility). But as a user, I must admit I have not missed what Debian provides that these do not. Whenever I tried to package stuff for Debian, I felt that I was paying a tax for unclear gain. It's a shame, because I like Debian for over a decade, and I am strongly sympathetic to the idea of the project in principle.
readams 6y ago
Sbuild for deb and mock for rpm make this pretty easy. Not super easy. But pretty easy.
- snuxoll 6y ago
  This, nobody should be manually calling rpmbuild or dpkg-buildpackage themselves. Doing this makes it easy to miss dependencies or other funky things you have done to your environment.

infinity0 6y ago

It should be noted that none of this article applies to actual development that goes on in the Debian distribution from https://www.debian.org https://www.debian.org Official Debian development, being FOSS oriented, proceeds from Debian source packages instead, and one never creates a binary package from scratch as described in the article. Instead, we use `dpkg-buildpackage` and other similar tools. The creation of the source package is the hard part, having to follow all the aspects of Debian policy, including technical matters around quality.

deleted 6y ago
[deleted]
secabeen 6y ago
Agreed. We use this technique to distribute commercial software to our workstations.
dijit 6y ago
Is there a guide on doing things that way? I've read some of the guides for making .deb packages (as I had to do so recently) and I have to be honest, what I encountered was "messy", articles and documentation that would work if followed but they didn't look even slightly similar. I'm too embarrassed to even link to the sources that I created.
- pabs3 6y ago
  The official entry point for getting packages into Debian is here: https://mentors.debian.net/intro-maintainers https://mentors.debian.net/intro-maintainers Debian packaging documentation is unfortunately a swamp, since people continue to write new guides instead of improving existing ones.
zwetan 6y ago
what if you need to distribute a .deb containing binaries that target other operating system ? example: I got a tool that allow from any Windows/macOS/Linux to package a runtime+script to any Windows/macOS/Linux the Windows .exe can not be compiled from sources under Linux, so this particular runtime can never be distributed on Debian official?

raziel2p 6y ago

I still go to FPM (https://github.com/jordansissel/fpm https://github.com/jordansissel/fpm) for any distro-native packaging needs I have.

app4soft 6y ago
For user apps on Linux I would prefer to use portable AppImage, instead of regular packages.
- ghostpepper 6y ago
  I misread your post at first because, once upon a time, portable meant that software would work on Linux _and_ other platforms.
  - app4soft 6y ago
    > portable meant that software would work on Linux _and_ other platforms. "portable" and "crossplatform" ("multiplatform") are different degrees of software.

NCommander 6y ago

Wow, this is horrible advice. I'm a Debian Developer and an Ubuntu Core Developer. What this will do is create a package that doesn't properly handle dependencies and be incredibly fragile. Debian and Ubuntu automatically resolve shared library dependencies are build time through dpkg-shlibs and friends. Debian's maintainer manual is likely the most detailed guide: https://www.debian.org/doc/manuals/maint-guide/start.en.html https://www.debian.org/doc/manuals/maint-guide/start.en.html However, it's a bit hard to grok, so I'm going to point people at the guide that got linked in the comments before I got here: https://saveriomiroddi.github.io/Learn-to-prepare-PPA-packages-by-setting-up-a-Ruby-PPA/ https://saveriomiroddi.github.io/Learn-to-prepare-PPA-packag...

closeparen 6y ago
With respect, I think this is a harmful attitude to take. People end up writing huge masses of Puppet/Chef/Ansible, or resorting to heavyweight technologies like Docker, just to place some files on a server. This problem is solved quite well by dpkg since forever, and it's a disservice to the community that this fact is obscured by so much cruft that only distro maintainers really need to know.
- dirtydroog 6y ago
  Seconded, I tried to package up a publicly available header-only tar.gz into a debian package for our private repo and found it impossible to do. I gave up.
  - GoblinSlayer 6y ago
    I just stole the process from another project that uses binary packaging.
voltagex_ 6y ago
How many of the problems here will be picked up by your package linter?
onli 6y ago

j1elo 6y ago

Debian packaging lacks a practical guide that is comprehensive enough to cover all modern use cases, but at the same time doesn't lack focus and goes to the point. The official docs are very extensive, but then they lack self-references to guide the reader to the interesting parts, or surprisingly lacks information that one would expect to find in there. You will read about debian/rules file, but then open one from any example project and only see a single rule calling "dh". Turns out debhelper is there doing things for you. On to read about debhelper. But now sometimes you'll see some rules file that uses the target "override_dh_autoreconf", which was not explained, not even hinted, in the previous docs. On to read more. Did you think you got everything? Not so fast! You'll document yourself about debian/control files and see that using some variable-like names are common in lots of projects. For example this is used a lot: Depends: ${misc:Depends}, ${shlibs:Depends} On with the same issue. For some reason (which new users reading the docs shouldn't really have to care), "shlibs:Depends" seems to be used all over the place, but it is not mentioned, hinted, or even suggested in a mere "see also" paragraph in the docs about debian/control. If you are thorough enough you'll end up seeing a quick mention in a different section (8. Shared Libraries), and turns out the actual definition of this variable has to be checked out from "dpkg-shlibdeps", an independent package. Now how to actually build a package? In the old days, the debian/rules file contained all the steps, but all is now helpfully abstracted by simply calling debhelper (dh). But calling debian/rules itself is abstracted also by dpkg-buildpackage. Calling dpkg-buildpackage is itself covered by debuild. And debuild is possibly covered by calling git-buildpackage. There is a crazy pile of tools calling tools calling other tools, that is nothing but extremely confusing for newcomers. For the 99% of times you just want a local package for local installation, dpkg-buildpackage is what you want. But getting to that conclusion is not something you get pointed to from a brief introduction from an official manual. You get to that conclusion after either studying all tools and putting order to all scattered knowledge in your head, or after blindly following some random blog of forum post (which tend to be correct because thankfully all this tooling doesn't change much over time) EDIT to stress that all information is there, but the official docs lack a good enough effort of linking or including everything in such a way that it follows a logical learning process for newcomers.

gorgoiler 6y ago
Where do Debian maintainers collect their institutional knowledge, these days? In the past it felt like DSCs uploaded to the FTP queues were the equivalent of committing code on a software project. Are they all in one git repository now, that can be tracked outside the project? Nothing beats being able to read the code for learning how a system actually works.

anttisalmela 6y ago

https://vincent.bernat.ch/en/blog/2019-pragmatic-debian-packaging https://vincent.bernat.ch/en/blog/2019-pragmatic-debian-pack...

JdeBP 6y ago
Ironically, that gets the service management wrong. The memcached people have provided a systemd service unit since 2011, for 8 years at the time that M. Bernat wrote that article. One quite clearly does not use the -d and -P options, nor Type=forking. memcached is Type=simple per its authors. Indeed, Type=forking in a systemd service unit is a good indicator that its author does not understand service management. It is invariably wrong because almost no programs actually speak the forking readiness protocol. (They fork for rather different reasons, which do not in fact adhere to the requirements of the forking readiness protocol.) See it, and you know to treat pronouncements about how daemonization works from that source with a great deal of suspicion. One such here is "memcached is started with the -d flag and will fork when it is ready to accept requests". In fact, memcached with -d forks before pretty much all of its initialization, and long before it is ready to accept requests. It hasn't even opened its listening sockets, or opened any data files or started any worker threads, at that point. Client services started because "memcached is ready" can fail to even connect to it. It might indeed yet fail whilst initializing, and never become ready. Really, memcached should have been runnable under inetd as a "wait" service from the start, which it unfortunately is not. That could have been adapted into a program that speaks the LISTEN_FDS protocol; and it would have a memcached.socket unit, with Accept=No and some ListenStream and ListenDatagram settings, and a Type=simple memcached.service unit. Then clients would have been able to reliably connect to the socket as soon as the service was started, and queue up until the service begins to accept() them when it becomes ready. Early socket opening in action. * https://github.com/memcached/memcached/blob/c460f0e13a35dcec26c71311dae6f31acec79f66/scripts/memcached.service https://github.com/memcached/memcached/blob/c460f0e13a35dcec... * http://jdebp.uk./FGA/unix-daemon-readiness-protocol-problems.html#NooneSpeaksForking http://jdebp.uk./FGA/unix-daemon-readiness-protocol-problems...

mavu 6y ago

This is better: https://github.com/phusion/debian-packaging-for-the-modern-developer https://github.com/phusion/debian-packaging-for-the-modern-d... (someone with the applicable knowledge please fork and continue it)

perlgeek 6y ago

<plug> I've written a book about Debian packaging that spends most of its page count on building Debian packages with debhelper/dh, the approach that most (or all?) official packages use. Here's a link for the HN crowd to get it for free: https://leanpub.com/debian/c/pm https://leanpub.com/debian/c/pm Happy to answer any questions as good as I can :-) </plug>