4 ms·

There are two levels of isolation when building Linux packages

30 points by ingve 3y ago

Arnavion 3y ago
>(At the time I set up my RPM configuration, the default RPM setup of package source wasn't ideal because it could comingle components of all packages together. These defaults may have changed since then.) It's still the same unless you manually make the effort to use a different topdir for each build. >Neither RPM nor Debian packages provide hermetic builds out of the box. For RPMs, mock provides an all-in-one solution that's generally very easy to use. You set up a chroot / container and build inside that. For OpenSUSE, the `osc build` command wraps `rpmbuild` to do that, and... >Debian has the sbuild collection of tools (also, sbuild(1)) that, based on my reading, provide the tools you need to do this (I only recently found out about sbuild and haven't tried to use it). ... according to its documentation that's what sbuild does too. Though the author's concern is also about how easy to use it is, which I have no experience with. My Debian packaging experience has been to use dpkg-buildpackage in a container.
- snuxoll 3y ago
  For RPM's you would use mock (https://rpm-software-management.github.io/mock/ https://rpm-software-management.github.io/mock/) to handle reproducible/hermetic build environments; it's the underlying tool used by Koji (the Fedora and Red Hat package build system) so it's regularly getting the tires kicked too.
throwawaaarrgh 3y ago
Use containers...
- throwawaaarrgh 3y ago
  [flagged]
dlor 3y ago
In Wolfi's packaging system (melange) we setup a hermetic build environment. See here: http://github.com/wolfi-dev/os http://github.com/wolfi-dev/os https://github.com/chainguard-dev/melange https://github.com/chainguard-dev/melange We use this to build APK packages from source for a large set of software.
dathinab 3y ago
For building packages you want to distribute I would generally recommend an environment well isolated, well defined and reset to a known immutable state every time before you build anything _independent/additionally of any isolation a build system provided_ (except if the build system provides exactly this kind of isolation). A common way to archive this is to run the steps in OCI containers (e.g. docker, CI systems like github actions, etc.). Through systems like nix can archive similar results. Be aware that just using an OCI image isn't separating the "build software for packaging" and "process of packaging software" steps, any decent packaging system should provide a reasonable version of this and not doing so is "a mayor wtf. it's 2023 get your shit together" moment. Through weather it's with multi stage docker containers or dependent CI jobs there are many ways to work around it. EDIT: To be clear I'm speaking about system packages and similar. Constraints for pure source code packages are similar but not quite the same and what is/isn't needed is also a lot language dependent.
Paul-Craft 3y ago
My goal for my emacs config has always been to be able to just clone the git repo it's in, then, either start up emacs and let it self-bootstrap, or maybe run a Makefile or build script. I want that to work flawlessly on any system, with any software installed (subject to having the right version of emacs, of course), so I've spent a good amount of time working on a pseudo-hermetic build that works on random Linux distros and OSX. I don't use Windows much, but I think with WSL2, I could just run the Linux build these days.
deleted 3y ago
[deleted]
suprjami 3y ago
>The second sort of isolation is isolation of the entire build process from your regular user environment and your system's particular set of installed packages (or packages that aren't installed). I have stopped building on my base install altogether. I now build in a throwaway Rootless Podman container which I can create in seconds when needed. It's very good. Next I want to automate installing all dependencies and doing the build, so i just run a container and get packages out the other end.
m463 3y ago
Docker is a good tool for building packages in a "hermetic" sort of environment. (although dockerfiles themselves are not completely perfect at reproducibility) I honestly wish docker had a sort of "build my docker images from fundamentals" mode, where you could build everything on your machine in a systematic way instead of including the docker.io cloud.

30 points by ingve 3y ago

>(At the time I set up my RPM configuration, the default RPM setup of package source wasn't ideal because it could comingle components of all packages together. These defaults may have changed since then.) It's still the same unless you manually make the effort to use a different topdir for each build. >Neither RPM nor Debian packages provide hermetic builds out of the box. For RPMs, mock provides an all-in-one solution that's generally very easy to use. You set up a chroot / container and build inside that. For OpenSUSE, the `osc build` command wraps `rpmbuild` to do that, and... >Debian has the sbuild collection of tools (also, sbuild(1)) that, based on my reading, provide the tools you need to do this (I only recently found out about sbuild and haven't tried to use it). ... according to its documentation that's what sbuild does too. Though the author's concern is also about how easy to use it is, which I have no experience with. My Debian packaging experience has been to use dpkg-buildpackage in a container.

snuxoll 3y ago
For RPM's you would use mock (https://rpm-software-management.github.io/mock/ https://rpm-software-management.github.io/mock/) to handle reproducible/hermetic build environments; it's the underlying tool used by Koji (the Fedora and Red Hat package build system) so it's regularly getting the tires kicked too.

throwawaaarrgh 3y ago

Use containers...

throwawaaarrgh 3y ago
[flagged]

dlor 3y ago

In Wolfi's packaging system (melange) we setup a hermetic build environment. See here: http://github.com/wolfi-dev/os http://github.com/wolfi-dev/os https://github.com/chainguard-dev/melange https://github.com/chainguard-dev/melange We use this to build APK packages from source for a large set of software.

dathinab 3y ago

For building packages you want to distribute I would generally recommend an environment well isolated, well defined and reset to a known immutable state every time before you build anything _independent/additionally of any isolation a build system provided_ (except if the build system provides exactly this kind of isolation). A common way to archive this is to run the steps in OCI containers (e.g. docker, CI systems like github actions, etc.). Through systems like nix can archive similar results. Be aware that just using an OCI image isn't separating the "build software for packaging" and "process of packaging software" steps, any decent packaging system should provide a reasonable version of this and not doing so is "a mayor wtf. it's 2023 get your shit together" moment. Through weather it's with multi stage docker containers or dependent CI jobs there are many ways to work around it. EDIT: To be clear I'm speaking about system packages and similar. Constraints for pure source code packages are similar but not quite the same and what is/isn't needed is also a lot language dependent.

Paul-Craft 3y ago

My goal for my emacs config has always been to be able to just clone the git repo it's in, then, either start up emacs and let it self-bootstrap, or maybe run a Makefile or build script. I want that to work flawlessly on any system, with any software installed (subject to having the right version of emacs, of course), so I've spent a good amount of time working on a pseudo-hermetic build that works on random Linux distros and OSX. I don't use Windows much, but I think with WSL2, I could just run the Linux build these days.

deleted 3y ago

[deleted]

suprjami 3y ago

>The second sort of isolation is isolation of the entire build process from your regular user environment and your system's particular set of installed packages (or packages that aren't installed). I have stopped building on my base install altogether. I now build in a throwaway Rootless Podman container which I can create in seconds when needed. It's very good. Next I want to automate installing all dependencies and doing the build, so i just run a container and get packages out the other end.

m463 3y ago

Docker is a good tool for building packages in a "hermetic" sort of environment. (although dockerfiles themselves are not completely perfect at reproducibility) I honestly wish docker had a sort of "build my docker images from fundamentals" mode, where you could build everything on your machine in a systematic way instead of including the docker.io cloud.