5 ms·
Build Initramfs Rootless
- usr1106 3y agoEditorialized title.
- m-p-3 3y agoKind of necessary, a post simply titled Initramfs wouldn't be useful.
- usr1106 3y agoBut isn't the article about building initramfs bottom-up, presenting the most basic tools? I did not see anything special that would make it rootless. That seemed to be a point made by the submitter.
- Retr0id 3y agoThe article itself says: >Note this is rootless, we don't need to re-create the filesystem tree locally or use fakeroot to create special file in the archive.
- usr1106 3y agoSure it does. But that's not because the article shows us any new trick, it's just because they use the original and most basic tool to generate an initramfs.
- Retr0id 3y agogen_init_cpio is not the original cpio archiver utility. It's only obvious to know to use it if you already know about it. The Linux kernel docs cover it, sure, but the whole point of the OP is a more approachable introduction to the same information.
- nonameiguess 3y agoThe article says it is but that is potentially misleading. Building an initramfs is very simple. Copy all the files you want in it and make a cpio archive. Arch's tool for doing it is just a 80 line or so bash script. In principle, it doesn't matter what user does this but only if all the files you need are readable by a non-root user.
- oasisaimlessly 3y agoYou can't make device nodes as non-root.
- Brian_K_White 3y agoYou can create any data as non-root, including the description of a device node in a cpio.
- mikepurvis 3y agoBut the point is that the tooling to do this is non-obvious. The same issue exists for all these kinds of archives— container image layers, ISOs, VMDKs, debootstrap outputs, etc. All of them are things that should be possible to just emit directly into a user-owned archive, but often require mounts, overlays, chown, and who knows what else. Another related one is about installing grub into an archived filesystem or disk image. Should be trivial, right? But most instructions for this require mounting it, which immediately means you need SYS_ADMIN and therefore a privileged container in your CI setup.
- zygology 3y agoYou can't make them on the filesystem itself, gen_init_cpio lets you specify device nodes to be created within the archive, but they are not actually created on the live system and do not require privileges. I made my own CPIO library: https://github.com/desultory/pycpio https://github.com/desultory/pycpio and one of the main goals was this behavior, so I could create CPIO archives with device nodes without privileges. It's really as simple as writing some bytes to a file saying "when you extract this, create this device node". It's not like any archive actually has a device node, just instructions for where to put them, and what properties they have. Certainly, you can't extract these archives as non-root, but you have to go out of your way to make an initramfs have non-root privileges.
- pzmarzly 3y agoI wish that there were some more tools to build archives based on script-like description. E.g. "i want this file under this path with this metadata, go build me a zip". Without it, my favorite way of building images rootless is still fakeroot. fakeroot + mknod + cpio can fully replace gen_init_cpio
- Retr0id 3y agoThe python `zipfile` module (`tarfile` likewise) is fairly straight-forward to use for this use case (just with a literal script instead of a script-like).
- danudey 3y agoUsing the zipfile/tarfile/etc. modules' functionality, it's easy to write a script that would take an input file of any format you want (such as the one in the example) and create an output file the same way it works in the article. It also allows for you to create archives compressed with gz, bz2, lzma, xz (via lzma), or potentially any other option that Python can provide.
- Retr0id 3y agoI did consider writing such a script as an example, but it occurred to me that just using ad-hoc imperative python scripts might be simpler/clearer (i.e. do we actually benefit from using a "DSL" here, even if it's a trivial one?)
- DHowett 3y agobsdtar can do this! It supports reading an mtree file (a text description of a filesystem or of the contents of an archive) as an input and producing ZIP as an output! Given an mtree file foo.mtree: #mtree baz/bar type=file content=quux You can produce a ZIP containing baz/bar with: bsdtar -c --format=zip -f out.zip @foo.mtree There’s a much better explanation in the bsdtar manual at https://man.freebsd.org/cgi/man.cgi?query=bsdtar&sektion=1&format=html#end https://man.freebsd.org/cgi/man.cgi?query=bsdtar&sektion=1&f... If you’re on Windows, “tar.exe” is bsdtar. With the caveat that the mtree file should be saved with LF line endings, this should still work.
- csdvrx 3y agoAre there FreeBSD/NetBSD equivalents to do the same without creating a UFS volume: - programatically declare the /dev/ files and their permissions - link the binaries you want to use I've learned about crunched binaries (like busybox, but in reverse), so I wonder if there's a BSD way to do that.
- LargoLasskhyfv 3y agohttps://duckduckgo.com/?t=ftsa&q=mfsbsd https://duckduckgo.com/?t=ftsa&q=mfsbsd ? There have been others in the past, but this seems to be the most polished and ready to use ATM(for FBSD). Another would be NanoBSD (also FBSD). For NetBSD you're on your own, starting from there https://wiki.netbsd.org/tutorials/how_to_create_bootable_netbsd_image/ https://wiki.netbsd.org/tutorials/how_to_create_bootable_net... , and/or asking on https://daemonforums.org/ https://daemonforums.org/ , https://www.unitedbsd.com/ https://www.unitedbsd.com/ (taking inspiration from some 'live-distro' discussed there. like 'OS-108'), reddit(?), 'crap-overflow', and even https://www.linuxquestions.org/ https://www.linuxquestions.org/. Of course you're free to use the official NetBSD mailing lists, and some obscure IRC-channels in even more obscure IRC-networks also :-) (You won't be spoon-fed, and are expected to have read the manuals and other documentations...)
- csdvrx 3y agoI'm very new to BSD in general, but I find it very fun and interesting! However, I need pointers to get started. > You won't be spoon-fed, and are expected to have read the manuals and other documentations... I read a lot of FreeBSD and NetBSD documentation to get to the point of compiling my own kernels, but I don't think I ever read about the equivalent concept of Linux cpio/initramfs for BSD. My minimal images use a UFS filesystem. Here, after checking https://mfsbsd.vx.sk/ https://mfsbsd.vx.sk/ and https://github.com/mmatuska/mfsbsd/blob/master/scripts/mdinit https://github.com/mmatuska/mfsbsd/blob/master/scripts/mdini... I think mfsbsd is just a using tmpfs so it may not exactly the same thing as initramfs, that allows booting linux from a bzImage + initrd I'll keep searching, it's not super high priority at the moment, but it's something I'd like to do with (Free|Net)BSD.
- kosolam 3y agoA good tool for creating images, also initrd images: http://mkosi.systemd.io/ http://mkosi.systemd.io/
- mathiasgredal 3y agoJust use bsdtar, which will convert a tar archive to cpio: https://unix.stackexchange.com/a/581014 https://unix.stackexchange.com/a/581014 I have used it to convert docker images to Linux bootable initramfs archives for Rpi4.
- akdev1l 3y agoDo you have link to that conversion? I wanna use containers to provision the pis, such a pain.
- zygology 3y agoI've had good luck making environments in a qemu user chroot, then you can more or less just copy the whole filesystem to a SD card and boot it, considering the boot partition of course.
- throwway120385 3y agoYou can also roll the same CPIO archive into your linux kernel by going into the build system and specifying that you want the initramfs to contain the files in the gen_init_cpio list referenced here. I forget exactly which flags do this but I use it all the time right now.
- lrvick 3y agoJust point CONFIG_INITRAMFS_SOURCE at a folder or cpio archive
- tekknolagi 3y agoThis kind of thing always makes me want to build a programming language with monadic file IO where the FS actually gets logically copied and modified at each write (but maybe this can get optimized away by a compiler).
- lrvick 3y agoHere is a tiny containerized and deterministic Linux build process with a minimal init system in rust for secure enclaves: https://git.distrust.co/public/enclaveos/src/branch/master/Containerfile https://git.distrust.co/public/enclaveos/src/branch/master/C... And another one for a simple busybox based ISO for airgap operations: https://git.distrust.co/public/airgap/src/branch/stagex-rewrite/Containerfile https://git.distrust.co/public/airgap/src/branch/stagex-rewr... Making Linux images for server/appliance use cases is always more secure and almost always easier than trying to adapt workstation distros like debian. Lots of ways to do it depending on your tooling preferences.
- zygology 3y agoI've been working on this for the last year or so: https://github.com/desultory/ugrd https://github.com/desultory/ugrd It's mostly made to work with Gentoo (uses pax-utils for finding dependencies), but should work with any distro. It does most of the initramfs creation process rootless. The main consideration is that you need privileges to make device nodes, but I just create them synthetically within the cpio image. On most systems, this tool still requires root to copy the mount binary.
- snac 3y agoThis is right up my alley, I’d love for a more programmatic way to generate initramfs. Being able to template different use cases would be great for specific appliances. One of my projects aims to create a hackable minimal Linux distro, and thinking about extension has always been on my mind. https://github.com/snacsnoc/snacklinux https://github.com/snacsnoc/snacklinux Have you found any specific niches that scratch an itch with your tool?
- bradfa 3y agoIf you want devtmpfs mounted automatically by the kernel within an initramfs, then you may need to patch your kernel slightly, like: https://lore.kernel.org/lkml/25e7e777-19f9-6280-b456-6c9c782c7901@landley.net/ https://lore.kernel.org/lkml/25e7e777-19f9-6280-b456-6c9c782... By default the kernel will not mount devtmpfs automatically within an initramfs, even if you've configured your kernel to automatically mount devtmpfs, because it waits until the real rootfs has been mounted to do it. This is fine unless you never plan to actually mount a real rootfs as you've made your initramfs have everything a normal rootfs would have and you want devtmpfs to Just Work.
- milkey_mouse 3y agoThanks--you just saved me an hour or two of debugging. I thought, CONFIG_DEVTMPFS_MOUNT=y, what more do I need to do? I should've just RTFM though: even an unpatched kernel mentions this in the help for that config option.
- zygology 3y agoIt's also easy enough to just add a mount command to your initramfs init, or embed required device nodes in the CPIO.
- bradfa 3y agoIt is, unless you're just trying to use an off the shelf distro as your initramfs and that distro's default init choice makes the assumption that the kernel will mount devtmpfs for it.
- zygology 3y agoMy understanding is that pretty much every modern init system has a service for mounting the devtmpfs, even if it's not in the fstab, and depending on the switch_root binary being used, mounts made in the initramfs may or may not be re-mounted in the new root. You don't even need a devtmpfs in the initramfs environment, and it may or may not even persist into the fully booted system. I believe Systemd has a tmpfs unit file, and OpenRC has a service for it which is enabled by default.
- deleted 3y ago[deleted]
- wrs 3y agoprintf '%s\n' '#include <stdio.h>' '#include <unistd.h>' 'int main() { printf("Hello world!\n"); sleep(999999999); }' | gcc -fpic -static -xc - -o init This is completely irrelevant, but that line reminded me of the old “real programmers write device drivers with cat” joke. I guess even realer programmers would write device drivers directly into gcc stdin.