7 ms·

RPM Packages Explained

85 points by qndev 7y ago

_pmf_ 7y ago
What's baffling to me is that there's support for signing,, but no support for an encryption layer. When delivering software updates for embedded devices, I need to do both, so I have to roll my own container format. This seems to also be the case with package formats like ipkg/opkg, which are targeted for the embedded use case.
- 0x006A 7y ago
  whats the use-case for an encryption layer?
  - ggg2 7y ago
    hiding packages you have installed from your ISP/NSA/etc. this discussion comes up time and time again (in rpm, apt et al). the consensus is: if you need that extra feature, manually download sensitive packages via ssl or something. everyone else (with nothing to hide, heh) keeps benefiting from a global cache of unencrypted transport of (mostly) open source data.
    - pwnna 7y ago
      Does that help? I thought the package size is quite revealing.
      - isostatic 7y ago
        In some cases (although the server could presumably send some random length data headers if that's a concern), but if you download multiple packages on a single connection can it still be tracked?
        vetinari 7y ago
        The sizes of all packages are a known information. So if someone is dedicated enough to track your downloaded packages, figuring out which ones were transferred with a single connection is relatively simple integer programming task. If you want to really hide what you are installing, make a local mirror of the entire repo and then pick and choose from that.
        scheveningen 7y ago
        I thought _pmf_ was describing packages that he authored, and certainly if the contents of them are confidential, they would be in a private repository. I don't think that the RPMs that I have created in my internal repository and deploy to my field systems are a 'known information' to anyone outside of my organization. If they are, I'm in serious trouble. I think a more realistic use case for package-level encryption is deploying RPMs that have secrets in them (either keys/creds in configuration or trade secrets in application logic). Ideally of course we should encapsulate these such that they aren't deployed to field/embedded devices but in embedded there certainly may be some use-cases and requirements that those of us used to working in data center and cloud computing aren't immediately thinking of.
        deleted 7y ago
        [deleted]
    - BuildTheRobots 7y ago
      Transport security & confidentiality makes sense (though at first I was trying to work out how an encrypted yum package would work). Yum with CentOS 6 and above does support SSL for mirror sites and a handful of global mirrors also support it (HEG being one). I suppose there's a slight race condition (eg how do I update the CA-Certificates bundle when I need the new CA-Certificates bundle to connect to the mirror site to download the update), however I tend to agree there should be some privacy as default.
    - ses1984 7y ago
      You can cache things that are encrypted too, or do you think drm protected Netflix videos are all streamed from the origin? Yeah it's a bit more complicated...
      - forgottenpass 7y ago
        If by "origin" you mean "box Netflix has root on"... yes, I do think that?
      - rhinoceraptor 7y ago
        Netflix runs a fleet of their own CDN boxes, that they put in ISP data centers.
    - solatic 7y ago
      As pwnna pointed out, package size gives you away. The real way to protect against this, if it's genuinely part of your threat model, is to maintain a complete local mirror: you can't tell what is installed and at what versions if you simply download everything. And if it's actually part of your threat model, then you likely have a large enough install base that you need a local mirror for performance/non-security reasons anyway. So it's really a non-issue.
    - eeZah7Ux 7y ago
      The combination of IP addresses and package sizes is way too revealing. That's why APT supports Tor as a transport protocol.
  - scheveningen 7y ago
    Personally I don't feel like the "hiding from the listener" use case discussed in the other response is very critical. What I think _pmf_ is getting at is an "only authorized devices may install my software, or view the RPMs". You could accomplish this having a keypair on your field/embedded devices, and then having the RPM distribution system pull each devices public key from the keyserver, build the RPM with encryption specifically for this device, and then push it out. Or maybe you choose to have a generic keypair for a class of devices. This could be used in cases where you have internal secrets in the RPMs you are building, or in the case of things like proprietary software and software licensing. I don't see how this applies to open source OS updates which is what I think the other sub-thread seems to be fixated on for some reason. Whether this belongs in the RPM system itself or in a wrapper format, I'm not so sure of.
  - _pmf_ 7y ago
    This is for embedded Linux devices with a set of proprietary applications (small volume commercial/industrial control, no consumer device). Mostly non networked, so we have to distribute application updates via USB (in the networked case, we a use VPNs separated by customer groups to deliver updates, so this layer provides the encryption without requiring the update archive to be encrypted).
- eeZah7Ux 7y ago
  The combination of IP addresses and package sizes is way too revealing. That's why APT supports Tor as a transport protocol.
- JetSpiegel 7y ago
  Why not have a password-protected repository per client? If you need public dependencies, build them once and copy them to all repositories. HTTP supports inline username and passwords, use HTTPS to keep the password encrypted on the wire: http://username:pass@server.tld/.. http://username:pass@server.tld/...
ggg2 7y ago
it doesn't explain anything! this is a indepth user guide, at most.
- isostatic 7y ago
  Thanks, I won't bother I know rpm is a cpio archive, with the files and some scripts, but the capabilities, when the scripts are run, etc I'm not sure of (I don't encounter rpms much - I'm a deb person, and I'm happy with my "ar -x" to explode my deb and look at the dependencies, conffiles, install/remove scripts, etc to install our custom packages on any rare centos servers we may need)
  - vetinari 7y ago
    It is a cpio archive with a custom header, so `rpm2cpio package.rpm | cpio ...` is needed to explode it. To look at scripts/dependencies/etc you don't need to explode it at all, rpm has switches for querying that.
ryanolsonx 7y ago
I found this very enlightening. Thank you!
Leace 7y ago
This article goes into more detail of RPM: https://xyrillian.de/thoughts/posts/argh-pm.html https://xyrillian.de/thoughts/posts/argh-pm.html
- majewsky 7y ago
  Came here to say the same thing. ;)
- dang 7y ago
  Should we change the URL to that above?
  - DyslexicAtheist 7y ago
    cruel :)
  - iamcreasy 7y ago
    Don't. It's a proof that the comment section is awesome!
- eeZah7Ux 7y ago
  This should be upvoted.
- gcbw2 7y ago
  Came for the click bait title of the original article, stayed for the awesome comments :)
- DyslexicAtheist 7y ago
  it's pretty crazy considering that RPM has been around since as long as I first installed Linux from floppy back in 1997. no wonder rpm no longer get's much love. RPM is too old, but sadly not old enough. My hope is in 20 years time @Foone[1] will warm our hearts with a thread on how he used RPM's to install the gnu toolchain on a fedora based IoT toaster from 2000-late. [1] https://twitter.com/foone https://twitter.com/foone
  - 7y ago
0xbadcafebee 7y ago
This does not explain RPM packages, title is clickbait. But i'll try my hand at it... RPM files are a compressed CPIO archive with some magic flags and an embedded key-value store. When installed or uninstalled, rpms execute various arbitrary stages to manage changes to an operating system before, during, and after install or uninstall. So they introduce not only file changes (including changes to the system's RPM database and its index/lock), but also arbitrary system state changes. Fun! RPM uses a global package database/index/lock to track what is installed and the dependencies. This can sometimes get corrupt, and then you may have to remove the lock and rebuild it. (edit: this partly applies to yum) Dependency resolution is crap, because dependency resolution is not dependent on a merkle tree or a transaction log. It's more of a lame recursive DAG that cares more about "what does this system currently have and what can I find in the package repo", versus "what was this thing actually built with/for and does this make sense at all to install". Packages are pretty much never built with a "base operating system" kind of dependency resolution, so it's possible to install a package from a completely different distro/version that was based on the one you are using. If you're lucky you won't be able to install the wrong package because of the recursive dependencies, but not everyone is lucky, and not all packages are built properly. It's also possible to create recursive dependencies so an installed package cannot be uninstalled. A .spec file defines what and how to build packages. A .srpm contains the source code to the application and the .spec file (handy!). A .rpm file is just one of potentially several packages that can result from building a .spec file, and the source can also be spread along multiple files. It's common for patches to be included in the .srpm and used at build time. A package repository typically contains both compiled binary packages organized by architecture (.rpm) and source packages (.srpm). If you add or remove a package from a repository, you need to re-generate the metadata files that the repo uses to communicate changes to tools like yum, or yum will have no idea about what you added/removed. RPM is actually fairly portable. A single .srpm file can build packages for Solaris, Windows, HP-UX, Linux, FreeBSD, etc. It can be a very good compliment to whatever the native packaging is, as long as you keep all packaged files in a unique file tree (like /opt/my_pkgs/).
- guggle 7y ago
  Is it any better/different than deb packages ? (I'm NOT trying to start a flame war or Debian vs. Fedora troll, just asking out of pure curiousity).
stirfrykitty 7y ago
After spending years messing with RPM packages in the sysadmin world, I now breathe a sigh of relief with Arch/pacman. Building/maintaining/installing RPMs is painful compared to Arch or Debian's dpkg.
- debiandev 7y ago
  Having worked a lot with RPM, other package managers, containers and proprietary build systems... I would choose debs every time.
  - stirfrykitty 7y ago
    Having said what I did above, though, can anything compare to SUSE's command to update? "zypper -up" Best command ever...
    - DyslexicAtheist 7y ago
      the initial (buggy) versions of zypper replacing Yast was what drove me away from SuSE for good. Never had much love for SuSe but what really killed it for me is when I worked as a consultant (a company that was a SuSE premium reseller) and I was the poor sod trying to integrate their solutions in large companies that didn't have IT/Tech as their core business (hospitals, government etc). I still have flashbacks to their SLOX (suse linux open exchange) and SLES which both were the buggiest distros I ever had shoved down my throat. I felt sorry for my customers. Once the whole IT of a hospital stopped working because of some bugs in their LDAP migration. I need therapy just by reflecting about this and their general way of how they handled root causes back then. From your comment it seems SuSE must have improved since 2005. I didn't follow them for long time now. It's Arch or Debian for me these days.
  - noselasd 7y ago
    I have the exact oposite experience. Building/packaging/maintainig my own software with RPMs is pretty straight foreward, whilst packaging the same software as .deb is a rollercoaster which often goes off track and down a deep hole.
throw7 7y ago
The reason I stayed with redhat/rpm so long ago was that rpm kept the native upstream source file untouched and pristine (most often a tarball). It was very easy to see what exactly was being patched and the build process driven by the specfile. This appealed to me as opposed to debian which often would just mash local patches together with the source (I hear debs do have the ability to keep the source "pristine", but for whatever reason dd's didn't usually do that).
- somepig 7y ago
  This is patently false. Source packages with Debian aren't even a single file. They consist of a pristine upstream tarball, a tarball of a /debian/ directory, and a dsc file that describes the package, including checksums of the two tarballs. The /debian/ directory contains all distro specific patches, as well as build rules (a Make file), and package scripts. It's possible to build a 'debian native' package where the /debian/ directory is bundled up with the orig source, but those are few and far between -- primarily packages consisting of debian-specific scripts/tools.
  - wahern 7y ago
    All true. Debian and Debian-derived distributions invariably manage their Debian build rules separately from upstream. In that sense there's little difference between RPM and Debian. However, if as a developer you want to include packaging support inside the project (for your own or your users' convenience), this is infinitely easier to accomplish with Debian than RPM packages. Various aspects of RPM--e.g. spec file template semantics, source and build directory locations--not only assume that packaging is done separately, but make violating those assumptions costly. Whereas Debian package build tools rarely make such assumptions. If I want to add "deb" and "rpm" targets to my Makefile, the deb target is a one-liner that directly invokes a dpkg tool using standard options. Whereas the rpm target may require generating a specfile (because some specfile variables aren't trivial to dynamically evaluate inline, if at all) and source tarball, or to go through additional contrivances to avoid either of those two things.
    - 7y ago
mrath 7y ago
Sometime back a sys admin at my friend's work place asked them to package Java applications as RPM for deployment. I thought that was not a great idea and may be wrong tool for the job. But I will take this opportunity to ask people here if it was an OK suggestion?

85 points by qndev 7y ago

What's baffling to me is that there's support for signing,, but no support for an encryption layer. When delivering software updates for embedded devices, I need to do both, so I have to roll my own container format. This seems to also be the case with package formats like ipkg/opkg, which are targeted for the embedded use case.

0x006A 7y ago
whats the use-case for an encryption layer?
- ggg2 7y ago
  hiding packages you have installed from your ISP/NSA/etc. this discussion comes up time and time again (in rpm, apt et al). the consensus is: if you need that extra feature, manually download sensitive packages via ssl or something. everyone else (with nothing to hide, heh) keeps benefiting from a global cache of unencrypted transport of (mostly) open source data.
  - pwnna 7y ago
    Does that help? I thought the package size is quite revealing.
    - isostatic 7y ago
      In some cases (although the server could presumably send some random length data headers if that's a concern), but if you download multiple packages on a single connection can it still be tracked?
      - vetinari 7y ago
        The sizes of all packages are a known information. So if someone is dedicated enough to track your downloaded packages, figuring out which ones were transferred with a single connection is relatively simple integer programming task. If you want to really hide what you are installing, make a local mirror of the entire repo and then pick and choose from that.
        scheveningen 7y ago
        I thought _pmf_ was describing packages that he authored, and certainly if the contents of them are confidential, they would be in a private repository. I don't think that the RPMs that I have created in my internal repository and deploy to my field systems are a 'known information' to anyone outside of my organization. If they are, I'm in serious trouble. I think a more realistic use case for package-level encryption is deploying RPMs that have secrets in them (either keys/creds in configuration or trade secrets in application logic). Ideally of course we should encapsulate these such that they aren't deployed to field/embedded devices but in embedded there certainly may be some use-cases and requirements that those of us used to working in data center and cloud computing aren't immediately thinking of.
        deleted 7y ago
        [deleted]
  - BuildTheRobots 7y ago
    Transport security & confidentiality makes sense (though at first I was trying to work out how an encrypted yum package would work). Yum with CentOS 6 and above does support SSL for mirror sites and a handful of global mirrors also support it (HEG being one). I suppose there's a slight race condition (eg how do I update the CA-Certificates bundle when I need the new CA-Certificates bundle to connect to the mirror site to download the update), however I tend to agree there should be some privacy as default.
  - ses1984 7y ago
    You can cache things that are encrypted too, or do you think drm protected Netflix videos are all streamed from the origin? Yeah it's a bit more complicated...
    - forgottenpass 7y ago
      If by "origin" you mean "box Netflix has root on"... yes, I do think that?
    - rhinoceraptor 7y ago
      Netflix runs a fleet of their own CDN boxes, that they put in ISP data centers.
  - solatic 7y ago
    As pwnna pointed out, package size gives you away. The real way to protect against this, if it's genuinely part of your threat model, is to maintain a complete local mirror: you can't tell what is installed and at what versions if you simply download everything. And if it's actually part of your threat model, then you likely have a large enough install base that you need a local mirror for performance/non-security reasons anyway. So it's really a non-issue.
  - eeZah7Ux 7y ago
    The combination of IP addresses and package sizes is way too revealing. That's why APT supports Tor as a transport protocol.
- scheveningen 7y ago
  Personally I don't feel like the "hiding from the listener" use case discussed in the other response is very critical. What I think _pmf_ is getting at is an "only authorized devices may install my software, or view the RPMs". You could accomplish this having a keypair on your field/embedded devices, and then having the RPM distribution system pull each devices public key from the keyserver, build the RPM with encryption specifically for this device, and then push it out. Or maybe you choose to have a generic keypair for a class of devices. This could be used in cases where you have internal secrets in the RPMs you are building, or in the case of things like proprietary software and software licensing. I don't see how this applies to open source OS updates which is what I think the other sub-thread seems to be fixated on for some reason. Whether this belongs in the RPM system itself or in a wrapper format, I'm not so sure of.
- _pmf_ 7y ago
  This is for embedded Linux devices with a set of proprietary applications (small volume commercial/industrial control, no consumer device). Mostly non networked, so we have to distribute application updates via USB (in the networked case, we a use VPNs separated by customer groups to deliver updates, so this layer provides the encryption without requiring the update archive to be encrypted).
eeZah7Ux 7y ago
The combination of IP addresses and package sizes is way too revealing. That's why APT supports Tor as a transport protocol.
JetSpiegel 7y ago
Why not have a password-protected repository per client? If you need public dependencies, build them once and copy them to all repositories. HTTP supports inline username and passwords, use HTTPS to keep the password encrypted on the wire: http://username:pass@server.tld/.. http://username:pass@server.tld/...

ggg2 7y ago

it doesn't explain anything! this is a indepth user guide, at most.

isostatic 7y ago
Thanks, I won't bother I know rpm is a cpio archive, with the files and some scripts, but the capabilities, when the scripts are run, etc I'm not sure of (I don't encounter rpms much - I'm a deb person, and I'm happy with my "ar -x" to explode my deb and look at the dependencies, conffiles, install/remove scripts, etc to install our custom packages on any rare centos servers we may need)
- vetinari 7y ago
  It is a cpio archive with a custom header, so `rpm2cpio package.rpm | cpio ...` is needed to explode it. To look at scripts/dependencies/etc you don't need to explode it at all, rpm has switches for querying that.

ryanolsonx 7y ago

I found this very enlightening. Thank you!

Leace 7y ago

This article goes into more detail of RPM: https://xyrillian.de/thoughts/posts/argh-pm.html https://xyrillian.de/thoughts/posts/argh-pm.html

majewsky 7y ago
Came here to say the same thing. ;)
dang 7y ago
Should we change the URL to that above?
- DyslexicAtheist 7y ago
  cruel :)
- iamcreasy 7y ago
  Don't. It's a proof that the comment section is awesome!
eeZah7Ux 7y ago
This should be upvoted.
gcbw2 7y ago
Came for the click bait title of the original article, stayed for the awesome comments :)
DyslexicAtheist 7y ago
it's pretty crazy considering that RPM has been around since as long as I first installed Linux from floppy back in 1997. no wonder rpm no longer get's much love. RPM is too old, but sadly not old enough. My hope is in 20 years time @Foone[1] will warm our hearts with a thread on how he used RPM's to install the gnu toolchain on a fedora based IoT toaster from 2000-late. [1] https://twitter.com/foone https://twitter.com/foone
- 7y ago

0xbadcafebee 7y ago

This does not explain RPM packages, title is clickbait. But i'll try my hand at it... RPM files are a compressed CPIO archive with some magic flags and an embedded key-value store. When installed or uninstalled, rpms execute various arbitrary stages to manage changes to an operating system before, during, and after install or uninstall. So they introduce not only file changes (including changes to the system's RPM database and its index/lock), but also arbitrary system state changes. Fun! RPM uses a global package database/index/lock to track what is installed and the dependencies. This can sometimes get corrupt, and then you may have to remove the lock and rebuild it. (edit: this partly applies to yum) Dependency resolution is crap, because dependency resolution is not dependent on a merkle tree or a transaction log. It's more of a lame recursive DAG that cares more about "what does this system currently have and what can I find in the package repo", versus "what was this thing actually built with/for and does this make sense at all to install". Packages are pretty much never built with a "base operating system" kind of dependency resolution, so it's possible to install a package from a completely different distro/version that was based on the one you are using. If you're lucky you won't be able to install the wrong package because of the recursive dependencies, but not everyone is lucky, and not all packages are built properly. It's also possible to create recursive dependencies so an installed package cannot be uninstalled. A .spec file defines what and how to build packages. A .srpm contains the source code to the application and the .spec file (handy!). A .rpm file is just one of potentially several packages that can result from building a .spec file, and the source can also be spread along multiple files. It's common for patches to be included in the .srpm and used at build time. A package repository typically contains both compiled binary packages organized by architecture (.rpm) and source packages (.srpm). If you add or remove a package from a repository, you need to re-generate the metadata files that the repo uses to communicate changes to tools like yum, or yum will have no idea about what you added/removed. RPM is actually fairly portable. A single .srpm file can build packages for Solaris, Windows, HP-UX, Linux, FreeBSD, etc. It can be a very good compliment to whatever the native packaging is, as long as you keep all packaged files in a unique file tree (like /opt/my_pkgs/).

guggle 7y ago
Is it any better/different than deb packages ? (I'm NOT trying to start a flame war or Debian vs. Fedora troll, just asking out of pure curiousity).

stirfrykitty 7y ago

After spending years messing with RPM packages in the sysadmin world, I now breathe a sigh of relief with Arch/pacman. Building/maintaining/installing RPMs is painful compared to Arch or Debian's dpkg.

debiandev 7y ago
Having worked a lot with RPM, other package managers, containers and proprietary build systems... I would choose debs every time.
- stirfrykitty 7y ago
  Having said what I did above, though, can anything compare to SUSE's command to update? "zypper -up" Best command ever...
  - DyslexicAtheist 7y ago
    the initial (buggy) versions of zypper replacing Yast was what drove me away from SuSE for good. Never had much love for SuSe but what really killed it for me is when I worked as a consultant (a company that was a SuSE premium reseller) and I was the poor sod trying to integrate their solutions in large companies that didn't have IT/Tech as their core business (hospitals, government etc). I still have flashbacks to their SLOX (suse linux open exchange) and SLES which both were the buggiest distros I ever had shoved down my throat. I felt sorry for my customers. Once the whole IT of a hospital stopped working because of some bugs in their LDAP migration. I need therapy just by reflecting about this and their general way of how they handled root causes back then. From your comment it seems SuSE must have improved since 2005. I didn't follow them for long time now. It's Arch or Debian for me these days.
- noselasd 7y ago
  I have the exact oposite experience. Building/packaging/maintainig my own software with RPMs is pretty straight foreward, whilst packaging the same software as .deb is a rollercoaster which often goes off track and down a deep hole.

throw7 7y ago

The reason I stayed with redhat/rpm so long ago was that rpm kept the native upstream source file untouched and pristine (most often a tarball). It was very easy to see what exactly was being patched and the build process driven by the specfile. This appealed to me as opposed to debian which often would just mash local patches together with the source (I hear debs do have the ability to keep the source "pristine", but for whatever reason dd's didn't usually do that).

somepig 7y ago
This is patently false. Source packages with Debian aren't even a single file. They consist of a pristine upstream tarball, a tarball of a /debian/ directory, and a dsc file that describes the package, including checksums of the two tarballs. The /debian/ directory contains all distro specific patches, as well as build rules (a Make file), and package scripts. It's possible to build a 'debian native' package where the /debian/ directory is bundled up with the orig source, but those are few and far between -- primarily packages consisting of debian-specific scripts/tools.
- wahern 7y ago
  All true. Debian and Debian-derived distributions invariably manage their Debian build rules separately from upstream. In that sense there's little difference between RPM and Debian. However, if as a developer you want to include packaging support inside the project (for your own or your users' convenience), this is infinitely easier to accomplish with Debian than RPM packages. Various aspects of RPM--e.g. spec file template semantics, source and build directory locations--not only assume that packaging is done separately, but make violating those assumptions costly. Whereas Debian package build tools rarely make such assumptions. If I want to add "deb" and "rpm" targets to my Makefile, the deb target is a one-liner that directly invokes a dpkg tool using standard options. Whereas the rpm target may require generating a specfile (because some specfile variables aren't trivial to dynamically evaluate inline, if at all) and source tarball, or to go through additional contrivances to avoid either of those two things.
  - 7y ago

mrath 7y ago

Sometime back a sys admin at my friend's work place asked them to package Java applications as RPM for deployment. I thought that was not a great idea and may be wrong tool for the job. But I will take this opportunity to ask people here if it was an OK suggestion?