12 ms·
Argh-P-M – Dissecting the RPM file format (2016)
- okket 8y ago(2016)
- Subsentient 8y agoTrue, RPM is pretty messed up as far as complexity. But, let's consider that I have never had an RPM-using system have its package database corrupted beyond repair (you can rebuild it with rpmdb --rebuilddb), and unlike dpkg/deb, rpm fully supports downgrading packages, which is something I have missed on Debian multiple times. RPM is too complex, but it's also a good format from a reliability standpoint.
- RVuRnvbM2e 8y ago> I have never had an RPM-using system have its package database corrupted beyond repair you've had that on Debian? > rpm fully supports downgrading packages, which is something I have missed on Debian multiple times Huh? Downgrades are fully supported by dpkg and apt*.
- yjftsjthsd-h 8y agoI've had a Debian system break because I couldn't fix the package database, yes. I might well have just missed a trick, but I did google around for fixes and got nowhere.
- Subsentient 8y ago> Huh? Downgrades are fully supported by dpkg and apt. No, they aren't. You need to remove the newer version prior to installing the old one. RPM based systems allow you to downgrade to an entirely different distro release with one command. > you've had that on Debian? More than once, sadly.
- CyberShadow 8y agoHow are these qualities descriptive of the file format and not of the particular implementation of the package manager used by the distribution in question?
- Subsentient 8y agoI am talking about the official implementation of the RPM package manager from Red Hat.
- CyberShadow 8y agoThe file format and the quality of the package manager implementation seem to be rather orthogonal, so I don't understand how your top comment is relevant to the article.
- sandGorgon 8y agothere was a chance to unify linux packaging in the last year around creating universal packages - what ended up happening instead was yet another war between snap and flatpak. Everyone flaunts some superiority or another - https://github.com/AppImage/AppImageKit/wiki/Similar-projects#comparison https://github.com/AppImage/AppImageKit/wiki/Similar-project... however, it would be incredible to have a single format
- digi_owl 8y agoIf you want a universal format, you have a compressed source tar-ball. Anything else will run into the whims (and on the ground circumstances) of maintainers and admins.
- bboozzoo 8y agoOr cpio. IIRC tar had its own set of issues once you consider old Linux distros or non-Linux systems in general.
- yjftsjthsd-h 8y agoFun story: I once had to exchange data with a Solaris box, and discovered that its tar had no "z" flag. Ended up just exchanging uncompressed tar files because I didn't want to figure out .tar.Z files.
- digi_owl 8y ago.Z is compress/uncompress, no?
- yjftsjthsd-h 8y agoThink so. And it was probably doable, but I realized that my network connection was fast enough that the transfer would be done before I could check.
- rwmj 8y ago
- nailer 8y agoThis person tries inspecting the format with a hex debugger, implying he hasn't done any research. Rpm2cpio then extract the RPM file. It's a binary and a text .spec file. The spec file format is divided into sections to build, make etc the app and while ancient (like dpkg) not hard to learn.
- icebraining 8y agoFrom what I can tell, spec files are just for source packages, whereas the author was analyzing (and trying to build) a binary package directly.
- nailer 8y agoThat's even worse. Why would you analyse a binary RPM to inspect it, when present that as a normal way to analyse or build RPMs? > the author was analysing trying to build a binary package directly. To analyse, there's two approaches that have been well documented for around 20 years now: - open the .src.rpm for a package - Run the rpm binary over the .rpm, which can dump spec file contents too To build: - Make a spec file Again, there's lots to dislike about RPM (my main issue is how the sections are macros and so are the autoconf macros), which confused a lot of new people) but this article in incredibly unresearched.
- yjftsjthsd-h 8y agoIn fairness, the author was trying to write his own RPM encoder, so not using existing tools was... less of a bad idea than usual. That said, I agree that this was under-researched.
- icebraining 8y agoWho cares why you do it? Maybe you're just curious. How does that invalidate the analysis?
- nailer 8y agoIt makes it seem like RPM is an incredibly difficult packaging system, when in reality the author has deliberately chosen an obscure and unusual way of working with it.
- lmilcin 8y agoThe only reason RPM is still being in use is RedHat, the company, that did well to provide the very needed service for actual businesses. There is no competition between package managers, there is only competition between distributions.
- rwmj 8y agoHow about SUSE and Mageia?
- CptMauli 8y agoTo plug it here: If you want to build an RPM via Maven (or just want to build one with Java using the underlying lib) https://github.com/ctron/rpm-builder https://github.com/ctron/rpm-builder
- neonihil 8y agoI've authored many RPM packages, it's dead simple. Have a spec file, tar it, and run rpmbuild on it. I don't really know why this guy was freaking out. I also like the fact that transient dependency and repo handling is separated from the package handler itself. We have rpm to handle a single package, and we have dnf (yum) to handle trees of packages.
- bmn__ 8y ago> I don't really know why this guy was freaking out. That's clearly explained in the article. This is not about authoring from a specfile, so your comment is off. He wants to serialise a package into RPM without the traditional toolchain; it turns out to be much more difficult than the other serialisers he already has because of the rpm format's bad design decisions and accretions.
- yjftsjthsd-h 8y agoI don't think they're bad decisions; I think he has no idea why it's that way and so is assuming that it's terrible. The RPM format was designed when computers were smaller and slower, and is extremely flexible while forcing you to declare enough that it can validate the file contents as it goes. It's over-engineered, and there are certainly things I would change, but it's not just random stupid decisions. If you can't think of any good reason why it would be written the way it is, the main options are that you're missing something or the original spec authors were stupid. Of course, sometimes they were stupid, but maybe don't immediately assume that after your very first investigation into the format.
- paol 8y ago> RPM format was designed when computers were smaller and slower So was DEB. That's not a very good excuse.
- icebraining 8y agoThe Deb file format was designed even earlier, yet it doesn't contain a matryoshka of metadata.
- notacoward 8y agoYes, the RPM format is crappy. A normal person would have seen that quickly (or known it already), gotten their business done, and moved on. What's the point of dissecting every little piece to criticize, or of criticizing even the oversights or practices normal for their time as much as the things that are truly insane? It's just rage-bait.
- fuzzy2 8y agoTo learn from past mistakes, probably. If you want to create something better, you first need to determine what’s bad.
- yjftsjthsd-h 8y agoYou also need to understand how it got that way. In this case, a lot of the things the author takes issue with are a combination of eternal backwards compatibility, really wide portability, and having originally been devised when memory was tiny and processors were slow. It's one of those things where if you rebuilt it yourself, by the time you covered all features of the original, you would be just as ugly.
- cesarb 8y agoI agree. For instance, one of the things the author complains about is the rpmlib(...) dependencies. To me, they are clearly a way to make older versions of rpm (which do not know anything about these features) fail cleanly with a "missing dependency" error.
- rsynnott 8y agoFor the benefit of the next person who has to figure this out, with bonus angry commentary?
- rwmj 8y agoAlmost no one who uses RPMs needs to know the internal format, and if they do they should use rpmlib (or rpm2archive if they just want to unpack the files). Does anyone not use JPEG because they think the internals of JPEG binary format are a bit complicated?
- bmn__ 8y agoThe point of the exercise was to not use rpmlib. The comparison to JPEG is inappropriate because it has dozens of interoperable implementations. Formats defined by a single implementation are sickly and much less viable (remember the lessons of http://libmng.com/ http://libmng.com/ ?); I applaud the reverse engineering effort from the article.
- pornel 8y ago> Does anyone not use JPEG because they think the internals of JPEG binary format are a bit complicated? Yes! There are lots of examples like this. • Plenty of small tools write BMP or Targa instead of JPEG or PNG, because these formats are dead simple to reinvent. • JPEG as we know it is JFIF, which has clearly won over alternative JPEG data containers, such as more complicated TIFF (mode 6). BTW JPEG, for what it does, is brilliantly simple. • MNG never took off, but dumber APNG did. Formats that are intended for interoperability need multiple implementations, and it does happen that authors say "no, fuck that" and choose a different format instead.
- renox 8y agoThat's funny, I've always thought that the various Linux distribution using different packaging format was a huge waste of resources and wanted everyone to standardize on RPM. After reading this article, I still thinks it is a waste of resources but I'd prefer that everyone standardize on .deb not RPM!
- jiveturkey 8y agoLike @nailer says, this article is under-researched and starting from a prejudiced viewpoint. However I did love one comment. > The <rpm/rpmtag.h> header has an enum with all acceptable values, but please consider the environment before trying to print it.
- deleted 8y ago[deleted]