10 ms·

An annoyance with Debian postinstall scripts during package upgrades

52 points by zdw 5y ago

egberts1 5y ago
I have a use case to run both ntpd and chronyd NTP daemons. Debian will not allow me to install both: it will delete the other package if you install one.
ddtaylor 5y ago
As a side note I switched from Debian to Fedora recently and I was amazed at just how much faster in almost every way possible dnf (rpm) is compared to apt (deb). Downloading packages is faster and has less broken mirrors. Actually running the transactions is much faster. Oh, and dnf seems to be able to handle the problem where you have multiple kernels updating or installing and each one has to regenerate the initrd gzip.
- Breazy 5y ago
  Using deltas for updates is a huge win for Fedora. On distros without delta updates, it's always a real pain in the ass when Wesnoth updates and you have to download half a gig of assets that are probably only a few kilobytes different from the last version.
- uggedal 5y ago
  dnf has support for using the CoW support of Btrfs to make all the disk I/O more efficient. Might be why you're seeing dnf being more efficient, because my somewhat outdated experience is totally opposite.
  - eklitzke 5y ago
    As I understand it the CoW support is for (optionally) taking filesystem snapshots before/after applying package upgrades, I don't think it inherently makes disk I/O more efficient. There are some other neat tricks that dnf does though. For example, in my experience dnf/rpm are smarter in the case where N packages need to run the same post-install step and run it once instead of apt/dpkg which might run it N times (examples: updating the man database, updating icon caches, updating ldconfig cache). And as the article points out, rpm packages are smarter about skipping post-install steps altogether for package updates. In general it's my impression that dnf/rpm have been much more actively developed in the past few years than apt/dpkg. Depending on how long it's been since you last used it, they've made changes to the manifest format, the internal rpm database format, and rewritten a lot of parts of dnf in C for speed, and all of these have made a huge impact on the speed and usability of dnf.
  - BenjiWiebe 5y ago
    If your outdated experience is with YUM as opposed to DNF, that could be why. DNF is much faster than YUM. However if you have used DNF then idk.
  - chousuke 5y ago
    YUM/DNF (and RPM itself) are very different beasts from what they used to be some time in the past. For some reason, updating repository metadata is still slow compared to APT, but installing packages is speedy enough. However, in terms of features it's just... better. Two things that I especially appreciate: I can install things without knowing the package names, eg. "dnf install /usr/bin/foot 'perl(XML::Tiny)' 'pkgconfig(bwayland-client)' libaudit.so.1" will just work. These are especially useful when packaging stuff yourself because you can use these indirect references in your dependency list so it doesn't matter which package actually provides them. I can also "sync" my system to a set of repositories with distro-sync, downgrading and/or replacing packages as necessary. I've had a few times with Ubuntu that I wished APT had this. I also like that it does not have debconf. I don't have much love for debconf.
martinald 5y ago
Package management on Linux is a real mess. It works great if you're running a server with standard software from official distro repos that is infrequently updated. As soon as you go outside that it quickly goes wrong in my experience (I've been using Linux since around 1999).. Recently my RaspberryPi media server's packages completely blew up because a third party repo messed something up which now has half installed a broken package and I cannot figure out how to fix it. This has happened basically every time I have used desktop Linux with any third party PPAs/repos to vary degrees of brokenness. This has _never_ happened to me on Windows/Mac (at least since the NT4 days - obviously software has broken things but I've never managed to break software installation system wide like this on other OSs). Even homebrew which I have used in anger seems to break far far less often than apt/dkpg despite having much more bleeding edge software.
- Breazy 5y ago
  > It works great if you're running a server with standard software from official distro repos that is infrequently updated. It's not frequent updates that screwed you, it's using third party repos. OpenSUSE Tumbleweed is an RPM based rolling release distro with up-to-date software that I've found to be very suitable, and stable, for desktop use. Before switching to Tumbleweed, I previously used Debian Sid for a long time. Contrary to Sid's 'unstable' moniker, was always very stable for me when I wasn't pulling in 3rd party repos or trying to mix Sid with Stable (the notorious 'frankendebian'.)
  - martinald 5y ago
    By "infrequently updated", I mean software where it is "ok" to be running versions released months/years ago in the official repos and not having to resort to 3rd party repos to get more up to date software. Nearly everyone has to use 3rd party software at some point though. There just isn't a good solution for this on Linux. Too many people assume Linux users only use open source software that is in the official package repos. It totally excludes commercial software, niche stuff, etc.
- kevin_thibedeau 5y ago
  If you want software outside the official repo that isn't self contained you should use a container or build it yourself and install into the /usr/local tree. Don't cross the streams.
kijin 5y ago
A related gripe I have with Debian-based distros is that APT shuts down daemons and keeps them down for an unreasonably long time while it updates seemingly unrelated packages. This is especially annoying on desktop Ubuntu boxes with automatic updates turned on. You're working on some frontend code and bam! nginx is down, the database is down. It should only take a second to restart foo.service after updating foo. Why shut it down so early and wait while APT is updating the kernel and a whole bunch of other packages? Probably because shutting eveyrthing down makes it easier for the postinstall script to do a clean job. YUM/DNF don't do this. They just update things and let me restart things at my leisure.
- ex_amazon_sde 5y ago
  > APT shuts down daemons and keeps them down for an unreasonably long Are you doing live upgrades on production servers? Don't. If you are doing updates anywhere else a couple of seconds of downtime is negligible.
  - kijin 5y ago
    Who said it was a live server? And it can take much longer than a couple of seconds. Did you even read the parent comment?
- uggedal 5y ago
  dnf has recently added automatic restart support of systemd system services (and soon user services). It's opt-in per package, but expect things to restart more in the future.
  - kijin 5y ago
    Restart is fine. It only takes a fraction of a second. But APT doesn't do restarts. It stops services and doesn't start them back up until it's finished with all the other packages. This can take a few minutes if there are kernel updates that trigger multiple initramfs rebuilds, or indefinitely long if the update process throws up a dialog to confirm something. OP makes it clear why this is happening. Packages have no concept of an upgrade, so APT stops services before removing the old version, and starts them back up after installing the new version.
throwdbaaway 5y ago
As a gentoo user, this postinstall behavior on debian/fedora/etc was rather shocking to me when I first encountered it. With gentoo portage, either when initially installing a package or when upgrading it, nothing under /var/lib is created or modified, and the process itself is never automagically started or stopped. The postinstall behavior actually feels like a regression toward windows-style installer. The cassandra debian package is a good example. If I let the postinstall script runs, then it will use the default config to start a single-node cassandra cluster, creating a bunch of stuffs under /var/lib/cassandra, which then need to be removed first if I want to build a proper image from the disk. Doing the removal as part of the automation feels too risky to me, as it may only take one typo to accidentally delete /var/lib/cassandra in production. Instead, I usually just put a policy-rc.d script to disable all postinstall scripts from running.
- oshiar53-0 5y ago
  It's worth noting that Fedora/RHEL's postinstall is generally less intrusive than Debain's, though. For instance it doesn't just automatically create databases or starts/enables services.
  - marcosdumay 5y ago
    People expect `apt install package` to leave the package installed and running. That expectation does change from one distro to another. Slackware was one that wouldn't even create a start script for a deamon.
  - chousuke 5y ago
    Debian also has debconf, which attempts to be a universal configuration management system, so it'll generate and re-generate configuration files from templates. Unfortunately, in my experience it gets in my way more often than it helps since I usually have separate automation that configures the software anyway. I use Debian-based systems pretty rarely, so I still haven't figured out how you actually make dpkg regenerate your configuration files properly if you use automation to change a debconf selection and need to reconfigure a package. dpkg-reconfigure seems not to do what I want and just restores the previous value (or prompts interactively, which is not useful) for some reason.
hlieberman 5y ago
It is required by Debian Policy §6.2 (https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#maintainer-scripts-idempotency https://www.debian.org/doc/debian-policy/ch-maintainerscript...) that package maintainer scripts be idempotent. A maintainer could, theoretically, figure out whether its an upgrade or not by checking to see if a version is being passed into it. postinst configure is only called with a version on upgrade. See: https://www.debian.org/doc/debian-policy/ap-flowcharts.html https://www.debian.org/doc/debian-policy/ap-flowcharts.html
- zinekeller 5y ago
  1) that's actually reasonable and I believe that all Debian-maintained packages follows this, but unfortunately some third-party, usually proprietary, packages don't follow this. 2) Huh, so it's actually there, just isn't explicit, probably because of their requirement of idempotency.
  - rlpb 5y ago
    It is explicit: https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-configuration https://www.debian.org/doc/debian-policy/ch-maintainerscript... There's also a handy summary in the same policy document: https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#summary-of-ways-maintainer-scripts-are-called https://www.debian.org/doc/debian-policy/ch-maintainerscript...
  - asveikau 5y ago
    I'm imagining it's also possible that there could be bugs in a postinstall script that make it behave differently during an upgrade. This is actually somewhat an argument that postinstall scripts are maybe a bad idea. They're quite hard to test. It's better to have the package manager simply manipulate files in a declarative fashion than have it run package-provided code that may be buggy and introduce issues.
    - bestouff 5y ago
      Of course it'd be better, but I don't think a simple declarative language could easily cover the breadth of what is currently done in postinstall scripts.

52 points by zdw 5y ago

I have a use case to run both ntpd and chronyd NTP daemons. Debian will not allow me to install both: it will delete the other package if you install one.

ddtaylor 5y ago

As a side note I switched from Debian to Fedora recently and I was amazed at just how much faster in almost every way possible dnf (rpm) is compared to apt (deb). Downloading packages is faster and has less broken mirrors. Actually running the transactions is much faster. Oh, and dnf seems to be able to handle the problem where you have multiple kernels updating or installing and each one has to regenerate the initrd gzip.

Breazy 5y ago
Using deltas for updates is a huge win for Fedora. On distros without delta updates, it's always a real pain in the ass when Wesnoth updates and you have to download half a gig of assets that are probably only a few kilobytes different from the last version.
uggedal 5y ago
dnf has support for using the CoW support of Btrfs to make all the disk I/O more efficient. Might be why you're seeing dnf being more efficient, because my somewhat outdated experience is totally opposite.
- eklitzke 5y ago
  As I understand it the CoW support is for (optionally) taking filesystem snapshots before/after applying package upgrades, I don't think it inherently makes disk I/O more efficient. There are some other neat tricks that dnf does though. For example, in my experience dnf/rpm are smarter in the case where N packages need to run the same post-install step and run it once instead of apt/dpkg which might run it N times (examples: updating the man database, updating icon caches, updating ldconfig cache). And as the article points out, rpm packages are smarter about skipping post-install steps altogether for package updates. In general it's my impression that dnf/rpm have been much more actively developed in the past few years than apt/dpkg. Depending on how long it's been since you last used it, they've made changes to the manifest format, the internal rpm database format, and rewritten a lot of parts of dnf in C for speed, and all of these have made a huge impact on the speed and usability of dnf.
- BenjiWiebe 5y ago
  If your outdated experience is with YUM as opposed to DNF, that could be why. DNF is much faster than YUM. However if you have used DNF then idk.
- chousuke 5y ago
  YUM/DNF (and RPM itself) are very different beasts from what they used to be some time in the past. For some reason, updating repository metadata is still slow compared to APT, but installing packages is speedy enough. However, in terms of features it's just... better. Two things that I especially appreciate: I can install things without knowing the package names, eg. "dnf install /usr/bin/foot 'perl(XML::Tiny)' 'pkgconfig(bwayland-client)' libaudit.so.1" will just work. These are especially useful when packaging stuff yourself because you can use these indirect references in your dependency list so it doesn't matter which package actually provides them. I can also "sync" my system to a set of repositories with distro-sync, downgrading and/or replacing packages as necessary. I've had a few times with Ubuntu that I wished APT had this. I also like that it does not have debconf. I don't have much love for debconf.

martinald 5y ago

Package management on Linux is a real mess. It works great if you're running a server with standard software from official distro repos that is infrequently updated. As soon as you go outside that it quickly goes wrong in my experience (I've been using Linux since around 1999).. Recently my RaspberryPi media server's packages completely blew up because a third party repo messed something up which now has half installed a broken package and I cannot figure out how to fix it. This has happened basically every time I have used desktop Linux with any third party PPAs/repos to vary degrees of brokenness. This has _never_ happened to me on Windows/Mac (at least since the NT4 days - obviously software has broken things but I've never managed to break software installation system wide like this on other OSs). Even homebrew which I have used in anger seems to break far far less often than apt/dkpg despite having much more bleeding edge software.

Breazy 5y ago
> It works great if you're running a server with standard software from official distro repos that is infrequently updated. It's not frequent updates that screwed you, it's using third party repos. OpenSUSE Tumbleweed is an RPM based rolling release distro with up-to-date software that I've found to be very suitable, and stable, for desktop use. Before switching to Tumbleweed, I previously used Debian Sid for a long time. Contrary to Sid's 'unstable' moniker, was always very stable for me when I wasn't pulling in 3rd party repos or trying to mix Sid with Stable (the notorious 'frankendebian'.)
- martinald 5y ago
  By "infrequently updated", I mean software where it is "ok" to be running versions released months/years ago in the official repos and not having to resort to 3rd party repos to get more up to date software. Nearly everyone has to use 3rd party software at some point though. There just isn't a good solution for this on Linux. Too many people assume Linux users only use open source software that is in the official package repos. It totally excludes commercial software, niche stuff, etc.
kevin_thibedeau 5y ago
If you want software outside the official repo that isn't self contained you should use a container or build it yourself and install into the /usr/local tree. Don't cross the streams.

kijin 5y ago

A related gripe I have with Debian-based distros is that APT shuts down daemons and keeps them down for an unreasonably long time while it updates seemingly unrelated packages. This is especially annoying on desktop Ubuntu boxes with automatic updates turned on. You're working on some frontend code and bam! nginx is down, the database is down. It should only take a second to restart foo.service after updating foo. Why shut it down so early and wait while APT is updating the kernel and a whole bunch of other packages? Probably because shutting eveyrthing down makes it easier for the postinstall script to do a clean job. YUM/DNF don't do this. They just update things and let me restart things at my leisure.

ex_amazon_sde 5y ago
> APT shuts down daemons and keeps them down for an unreasonably long Are you doing live upgrades on production servers? Don't. If you are doing updates anywhere else a couple of seconds of downtime is negligible.
- kijin 5y ago
  Who said it was a live server? And it can take much longer than a couple of seconds. Did you even read the parent comment?
uggedal 5y ago
dnf has recently added automatic restart support of systemd system services (and soon user services). It's opt-in per package, but expect things to restart more in the future.
- kijin 5y ago
  Restart is fine. It only takes a fraction of a second. But APT doesn't do restarts. It stops services and doesn't start them back up until it's finished with all the other packages. This can take a few minutes if there are kernel updates that trigger multiple initramfs rebuilds, or indefinitely long if the update process throws up a dialog to confirm something. OP makes it clear why this is happening. Packages have no concept of an upgrade, so APT stops services before removing the old version, and starts them back up after installing the new version.

throwdbaaway 5y ago

As a gentoo user, this postinstall behavior on debian/fedora/etc was rather shocking to me when I first encountered it. With gentoo portage, either when initially installing a package or when upgrading it, nothing under /var/lib is created or modified, and the process itself is never automagically started or stopped. The postinstall behavior actually feels like a regression toward windows-style installer. The cassandra debian package is a good example. If I let the postinstall script runs, then it will use the default config to start a single-node cassandra cluster, creating a bunch of stuffs under /var/lib/cassandra, which then need to be removed first if I want to build a proper image from the disk. Doing the removal as part of the automation feels too risky to me, as it may only take one typo to accidentally delete /var/lib/cassandra in production. Instead, I usually just put a policy-rc.d script to disable all postinstall scripts from running.

oshiar53-0 5y ago
It's worth noting that Fedora/RHEL's postinstall is generally less intrusive than Debain's, though. For instance it doesn't just automatically create databases or starts/enables services.
- marcosdumay 5y ago
  People expect `apt install package` to leave the package installed and running. That expectation does change from one distro to another. Slackware was one that wouldn't even create a start script for a deamon.
- chousuke 5y ago
  Debian also has debconf, which attempts to be a universal configuration management system, so it'll generate and re-generate configuration files from templates. Unfortunately, in my experience it gets in my way more often than it helps since I usually have separate automation that configures the software anyway. I use Debian-based systems pretty rarely, so I still haven't figured out how you actually make dpkg regenerate your configuration files properly if you use automation to change a debconf selection and need to reconfigure a package. dpkg-reconfigure seems not to do what I want and just restores the previous value (or prompts interactively, which is not useful) for some reason.

hlieberman 5y ago

It is required by Debian Policy §6.2 (https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#maintainer-scripts-idempotency https://www.debian.org/doc/debian-policy/ch-maintainerscript...) that package maintainer scripts be idempotent. A maintainer could, theoretically, figure out whether its an upgrade or not by checking to see if a version is being passed into it. postinst configure is only called with a version on upgrade. See: https://www.debian.org/doc/debian-policy/ap-flowcharts.html https://www.debian.org/doc/debian-policy/ap-flowcharts.html

zinekeller 5y ago
1) that's actually reasonable and I believe that all Debian-maintained packages follows this, but unfortunately some third-party, usually proprietary, packages don't follow this. 2) Huh, so it's actually there, just isn't explicit, probably because of their requirement of idempotency.
- rlpb 5y ago
  It is explicit: https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-configuration https://www.debian.org/doc/debian-policy/ch-maintainerscript... There's also a handy summary in the same policy document: https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#summary-of-ways-maintainer-scripts-are-called https://www.debian.org/doc/debian-policy/ch-maintainerscript...
- asveikau 5y ago
  I'm imagining it's also possible that there could be bugs in a postinstall script that make it behave differently during an upgrade. This is actually somewhat an argument that postinstall scripts are maybe a bad idea. They're quite hard to test. It's better to have the package manager simply manipulate files in a declarative fashion than have it run package-provided code that may be buggy and introduce issues.
  - bestouff 5y ago
    Of course it'd be better, but I don't think a simple declarative language could easily cover the breadth of what is currently done in postinstall scripts.