5 ms·

Arch Linux minimal container userland 100% reproducible – now what?

103 points by kpcyrd 3y ago

sph 3y ago
Oh, might be the chance to chuck my Alpine, Debian and Ubuntu containers into the flaming sun and just use an Arch Linux base image. No more dealing with apt nonsense; no more dealing with Debian packaging philosophy which is antithetical to containers; and no more musl, which has been working fine, but I keep wondering whether any DNS issue is because of it.
- lrvick 3y ago
  IMO musl is the right way forward if you want to optimize for minimalism, security, and making it easy to build static binaries in most languages.
  - anthk 3y ago
    Not just static builds. An alpine chroot with bubblewrap and mosh for non-supported distros weights less than 60MB.
    - musicale 3y ago
      I wonder how much larger that is than, say, classic Unix distributions like 4.3BSD.
  - musicale 3y ago
    And glibc is bad until it can be used reliably for static builds.
- rascul 3y ago
  > I keep wondering whether any DNS issue is because of it If you're referring to the lack of TCP for DNS, that was fixed almost a year ago in 1.2.4. https://musl.libc.org/releases.html https://musl.libc.org/releases.html https://news.ycombinator.com/item?id=36933028 https://news.ycombinator.com/item?id=36933028
  - sph 3y ago
    So apart from unknown bugs, musl should now be as good as glibc for networked services?
    - deleted 3y ago
      [deleted]
    - rascul 3y ago
      I suspect so. I'm not really qualified to answer that definitively, though.
devaiops9001 3y ago
flake.nix and Brazil exist. So does Guix. This news is great for the Arch Linux project. But for the reproducible world this is meh.
- lrvick 3y ago
  Nix is mostly reproducible but does not require maintainers to sign their packages or commits, and most do not, which is a bare minimum for any security sensitive environment. In Guix signing is mandated and it is mostly reproducible, but the choice of scheme and lack of base container images make it unapproachable for many. Debian lead the way on signing and reproducibility, but package versions of things like rust are too far behind to be useful to most orgs. Arch in contrast to these is IMO easy to package for, has recent well maintained signed packages, has well maintained OCI images published, and is rapidly improving on reproducibility. Having at least one glibc distro that can meet this criteria is a big win for many use cases. Different tools for different projects and threat models.
  - devaiops9001 3y ago
    >> base container images This is very easy to solve for. >> choice of scheme There was more wisdom when everyone at least tacitly acknowledge that maybe not everyone should be touching servers.
- sph 3y ago
  With all due respect to Nix/Guix, to me they swim uphill against the current of "worse is better" than UNIX is built upon. They are trying to tame the complexity of the world by making it declarative. A lofty, and a little too idealistic goal. I much prefer the immutable approach (rpm-ostree) or even an unsophisticated approach like a Dockerfile, which is worse but much closer to the status quo, than to create a perfect world from scratch and hope that everybody follows the Light. Software is too large, complex, hacky, buggy and nuanced to expect it to fit into neat preordained categories. Because unless you do very simple things, you'll soon find yourself out of the happy path and have to resort to doing the hacky way anyway. This is what some believe killed Lisp machines vs UNIX. I guess, like Lisp, in decades there will a vocal contingent of people lamenting the fact that we all run immutable Microsoft Kubernetes Ubuntu (MSKU) instead of using the more refined NixOS approach. https://www.dreamsongs.com/WorseIsBetter.html https://www.dreamsongs.com/WorseIsBetter.html
lrvick 3y ago
Since OCI determinism was mentioned, I feel it may be of interest that some friends and I bootstrapped a linux distro from 0 to solve exactly this problem. [Stageˣ]: The container native, full source bootstrapped, musl based, and reproducible toolchain to build all of the things. https://codeberg.org/stagex/stagex https://codeberg.org/stagex/stagex A lot more to come on this soon. This was only possible by carefully incorporating and cribbing a huge amount of bootstrapping and reproducibility work from the Stage0, live-bootstrap, Docker, Guix, Nix, mrustc, Debian, Alpine, and Arch teams, many of whose members have generously helped unblock us several times.

103 points by kpcyrd 3y ago

sph 3y ago

Oh, might be the chance to chuck my Alpine, Debian and Ubuntu containers into the flaming sun and just use an Arch Linux base image. No more dealing with apt nonsense; no more dealing with Debian packaging philosophy which is antithetical to containers; and no more musl, which has been working fine, but I keep wondering whether any DNS issue is because of it.

lrvick 3y ago
IMO musl is the right way forward if you want to optimize for minimalism, security, and making it easy to build static binaries in most languages.
- anthk 3y ago
  Not just static builds. An alpine chroot with bubblewrap and mosh for non-supported distros weights less than 60MB.
  - musicale 3y ago
    I wonder how much larger that is than, say, classic Unix distributions like 4.3BSD.
- musicale 3y ago
  And glibc is bad until it can be used reliably for static builds.
rascul 3y ago
> I keep wondering whether any DNS issue is because of it If you're referring to the lack of TCP for DNS, that was fixed almost a year ago in 1.2.4. https://musl.libc.org/releases.html https://musl.libc.org/releases.html https://news.ycombinator.com/item?id=36933028 https://news.ycombinator.com/item?id=36933028
- sph 3y ago
  So apart from unknown bugs, musl should now be as good as glibc for networked services?
  - deleted 3y ago
    [deleted]
  - rascul 3y ago
    I suspect so. I'm not really qualified to answer that definitively, though.

devaiops9001 3y ago

flake.nix and Brazil exist. So does Guix. This news is great for the Arch Linux project. But for the reproducible world this is meh.

lrvick 3y ago
Nix is mostly reproducible but does not require maintainers to sign their packages or commits, and most do not, which is a bare minimum for any security sensitive environment. In Guix signing is mandated and it is mostly reproducible, but the choice of scheme and lack of base container images make it unapproachable for many. Debian lead the way on signing and reproducibility, but package versions of things like rust are too far behind to be useful to most orgs. Arch in contrast to these is IMO easy to package for, has recent well maintained signed packages, has well maintained OCI images published, and is rapidly improving on reproducibility. Having at least one glibc distro that can meet this criteria is a big win for many use cases. Different tools for different projects and threat models.
- devaiops9001 3y ago
  >> base container images This is very easy to solve for. >> choice of scheme There was more wisdom when everyone at least tacitly acknowledge that maybe not everyone should be touching servers.
sph 3y ago
With all due respect to Nix/Guix, to me they swim uphill against the current of "worse is better" than UNIX is built upon. They are trying to tame the complexity of the world by making it declarative. A lofty, and a little too idealistic goal. I much prefer the immutable approach (rpm-ostree) or even an unsophisticated approach like a Dockerfile, which is worse but much closer to the status quo, than to create a perfect world from scratch and hope that everybody follows the Light. Software is too large, complex, hacky, buggy and nuanced to expect it to fit into neat preordained categories. Because unless you do very simple things, you'll soon find yourself out of the happy path and have to resort to doing the hacky way anyway. This is what some believe killed Lisp machines vs UNIX. I guess, like Lisp, in decades there will a vocal contingent of people lamenting the fact that we all run immutable Microsoft Kubernetes Ubuntu (MSKU) instead of using the more refined NixOS approach. https://www.dreamsongs.com/WorseIsBetter.html https://www.dreamsongs.com/WorseIsBetter.html

lrvick 3y ago

Since OCI determinism was mentioned, I feel it may be of interest that some friends and I bootstrapped a linux distro from 0 to solve exactly this problem. [Stageˣ]: The container native, full source bootstrapped, musl based, and reproducible toolchain to build all of the things. https://codeberg.org/stagex/stagex https://codeberg.org/stagex/stagex A lot more to come on this soon. This was only possible by carefully incorporating and cribbing a huge amount of bootstrapping and reproducibility work from the Stage0, live-bootstrap, Docker, Guix, Nix, mrustc, Debian, Alpine, and Arch teams, many of whose members have generously helped unblock us several times.