8 ms·
How modern Linux systems boot
- pmalynin 8y agoFor those interested: nowadays Linux has support for what is called the EFI Stub boot, which allows the kernel to be booted directly by UEFI. So in a sense steps 1-3 are rolled into the kernel, and the kernel acts as its own boot loader.
- chungy 8y agoWouldn't it rather just be UEFI taking place of steps 1-3? The kernel becomes a PE binary instead of an ELF one, but I get the sense that it's basically the same thing...?
- secure 8y agoDo you know of any major distributions using this mechanism? I’ve only ever seen GRUB on UEFI.
- rl3 8y agoArch supports it: https://wiki.archlinux.org/index.php/EFISTUB https://wiki.archlinux.org/index.php/EFISTUB Being able to directly edit your motherboard's EFI boot order via command line is admittedly pretty neat. That said, using a traditional bootloader seems to be a popular choice still for simplicity and portability reasons. EFISTUB also can make certain crypto setups more difficult as far as I'm aware.
- checkyoursudo 8y agoOn Gentoo, for a couple of years, I used an initramfs compiled into the kernel to boot via EFI_STUB with full disk encryption. It's possible. It's been a while. I don't remember it being very easy to wrangle. I don't use that laptop anymore, but I still boot via EFI_STUB on my desktop. It's super simple to set up when you're not using FDE. I don't remember there being any specific problem that make some crypto setups more difficult than others, with the caveat that implementing FDE at all makes EFI_STUB somewhat more complicated no matter what. Like I said, it's been a couple of years since I figured all that stuff out, so the passage of time may have softened the terror of configuration and clouded my memories.
- rl3 8y ago>... the terror of configuration ... That's an excellent way of phrasing it.
- checkyoursudo 8y agoI have been booting an EFI_STUB kernel in Gentoo for several years. It's pretty easy to do. See, e.g., https://wiki.gentoo.org/wiki/EFI_stub_kernel https://wiki.gentoo.org/wiki/EFI_stub_kernel
- 21 8y agoI have a full disk encryption setup for my Ubuntu. It uses LVM, LUKS, and cryptosetup in GRUB. The problem is that it takes forever to boot (5 min, not kidding). Apparently the whole boot partition is decrypted to a RAM disk or something.
- mjevans 8y agoThis depends heavily on your security model... Are you trying to use just a literal 'usb key' as a fob for booting the computer? Decrypting the LUKS key will take some time, but you can ship the key for decrypting that in the initramfs that you'd store with the kernel on the USB drive.
- deleted 8y ago[deleted]
- lgunsch 8y agoI have my whole disk encrypted, including swap, and use LVM with BTRFS on top. My key is included in the initramfs. After I type my password in the Grub prompt, it only takes about 60 to 90 seconds to do a full boot. Although, the disk is a SSD. I'm sure if I didn't have a SSD disk, the boot process would take a very long time.
- scintill76 8y agoI've similarly included my key in the initramfs, until I noticed the initramfs file in /boot is world-readable. (On Linux Mint.) I'm a bit over-paranoid maybe, but I don't want a compromise of my user account turning into compromising my keys. Have you had that issue, and fixed it cleanly? (I could just chmod, but I don't want to have to remember.) For now, I just enter my password twice -- once for GRUB to load Linux, then again in Linux's early boot. I've been tempted to implement some way for GRUB to pass the key to Linux in RAM, avoiding the need for a key file.
- lgunsch 8y agoMy initramfs is only readable by root (0600). The key is also in a file only readable by root (0600). The key file is added into the initramfs image when it's built. edit: I'm on Arch Linux
- Florin_Andrei 8y agoIt's not really that different from how things were 20 years ago. There's a few bells and whistles added now, but for the most part it's still the old sequence. (added this comment because of the "modern" qualifier in the title)
- thatcks 8y agoHow much things have changed depends on what level you look at the boot process at. For example, before 2.3.41 introduced the pivot_root system call, the starting process in the initial ramdisk wasn't run as PID 1 (as far as I remember and can tell from the remaining kernel code for this). The kernel also used to be far more willing to do things itself, such as assemble software RAID devices; modern Linux pushes all of that into the initial ramdisk user level code, or even later in boot. At the broad level, though, yes absolutely. Unixes with System V init have been drawing a distinction between 'single user' boot activities like fsck'ing the filesystems and 'multi-user' ones like starting daemons for a long time, so that's a two stage boot. Linux booting became three stage once it added initial ramdisks so that the core kernel didn't have to have to build in all of the pieces necessary to get the root filesystem. (I'm the author of the linked-to entry.)
- JdeBP 8y agoYou did, however, proffer a discussion of the Linux boot process, and Linux operating systems do not and did not have AT&T System 5 init. They have/had something resembling it, written by Miquel van Smoorenburg in the 1990s. But it isn't AT&T System 5 init, and it actively diverged from it early on. Ironically, 20 years ago was almost three years after van Smoorenburg init+rc had diverged from the old idea of one "single-user" mode and had instead taken the route of two modes, emergency and rescue. In fact, we only need to wait a year or so until it has been 20 years since those names can be found in widespread use. It has already been more than 20 years since the name "emergency" gained traction. * http://jdebp.info./FGA/emergency-and-rescue-mode-bootstrap.html#history http://jdebp.info./FGA/emergency-and-rescue-mode-bootstrap.h... So whilst things are like they were 20 years ago, how things were 20 years ago isn't in fact the old AT&T System 5 world that people nowadays tell one another it was. That was the 1980s, and it wasn't Linux-based. Indeed, by 25 years ago the AT&T world itself had already introduced ideas changing the original model of multi-user login, such as the Service Access Facility.
- scott_s 8y agoIf anyone is curious what it looked like in 2004: http://www.scott-a-s.com/the-linux-boot-process-of-2004/ http://www.scott-a-s.com/the-linux-boot-process-of-2004/
- amorousf00p 8y agoWas a much better time imho.
- herpderperator 8y agoWhat this article fails to mention is that having an initramfs is not required. It's only necessary if additional tools are needed to mount a particular filesystem, otherwise the kernel can run init directly from the root.
- textmode 8y agoTrue or false: An initrd (cf. initramfs) is still required, where "rd" stands for ramdisk. As a BSD user, I prefer to use a mfs root (ramdisk) for personal reasons, but AFAIK BSD has never required a ramdisk in order to boot. If I recall correctly Linux needed to use a ramdisk in its early days. Is this still true today?
- jschwartzi 8y agoNo, neither are required. initrd is obsolete and has been replaced by initramfs.
- kijiki 8y agoEarly on, Linux supported initrd, which was just a ramdisk with a FS image written to it and mounted. Later, initramfs came along, which is all anyone uses anymore. It is a cpio archive that is extracted onto a special form of tmpfs. One neat trick is that you can append multiple cpio archives, and the kernel will expand them one after another, so you can easily customize your distro-provided initramfs. Only the first one can be compressed, and you have to be careful about it getting overwritten by a kernel update, but it is handy for quick experiments. There is also pivot_root and switch_root, which have man pages if you're interested in the gory details. For all of initrd/initramfs's history, you've only needed one if your monolithic kernel couldn't mount the root filesystem. So if you needed a module, or had some special config that needed to happen that the kernel couldn't do internally.
- ajross 8y agoLinux has never needed a ramdisk. If it's built with drivers for the boot device and it probes in an unambiguous manner, you can pass root=/dev/whatever99. That's as true today as it was in 1992.
- forgot-my-pw 8y agoBrief overview on how ChromeOS boots (8 years ago): https://www.youtube.com/watch?v=mTFfl7AjNfI https://www.youtube.com/watch?v=mTFfl7AjNfI Would love to watch an updated version today, if anything changes.
- voidmain 8y agoOne day, through the primeval wood, A calf walked home, as good calves should... [1] http://holyjoe.org/poetry/foss3.htm http://holyjoe.org/poetry/foss3.htm
- bjackman 8y agoThis is wonderful. By "S.W. Foss" as well...
- AceJohnny2 8y ago> How the initramfs /init pivots into running your real system's init daemon on your real system's root filesystem is beyond the scope of this entry. The commands may be simple (systemd just runs 'systemctl switch-root'), but how they work is complicated. Darn, I was actually hoping this would be elaborated on. Does anyone know of other (just as understandable) sources?
- xfs 8y agodracut does the work. The reason it is complicated is because it supports booting from btrfs, DM RAID, MD RAID, LVM2, device mapper multipath I/O, dm-crypt, cifs, FCoE, iSCSI, NBD and NFS. https://en.wikipedia.org/wiki/Dracut_(software) https://en.wikipedia.org/wiki/Dracut_(software)
- AceJohnny2 8y agoHm, Dracut looks like a userspace framework for building initramfs. I was wondering more about what's going on at the kernel level to switch the root. What depends on / ? Do any processes survive whose CWD is in the initramfs? Later in the article it's pointed out that systemd(from initramfs) passes the torch to systemd(in final root) while preserving state, so are any processes/ressources from the initramfs leftover or is everything cleared?
- xfs 8y agoA process spawned from initramfs can certainly live on but you wouldn't want to do that because that would prevent initramfs from being unmounted. By switching to the new root you can run real binaries. You can also not switch the root and also run real binaries from the new mount but that's not very convenient is it? Pivoting root is not the complicated. The complicated is the process of detecting and setting up the real root filesystem which can involve loading kernel modules to dhcp negotiation to http requests and more. Dracut handles all these.
- cmurf 8y agoYou can boot with two additional boot parameters: systemd.log_level=debug rd.debug That will get you very verbose logging to the journal, it'll include all the dracut stuff, and systemd setting up jobs and going through all sorts of logic. To read the result: sudo journalctl -b -o short-monotonic I prefer monotonic time even though with these two boot options the startup will be significantly slower, it'll give you a better idea of the relative cost of everything than date/time.
- jbuzbee 8y agoBack in the day, I seem to recall having to binary patch LILO every time I installed a new kernel to tell it what partition to boot. I suppose that dates me...
- rl3 8y agoRelated: http://iam.tj/prototype/guides/boot/ http://iam.tj/prototype/guides/boot/ There's something to be said for flow charts when trying to explain a complex topic such as the boot process.
- ericnyamu12 8y agowow.check out this cool forum http://mawazo.byethost7.com/index.php http://mawazo.byethost7.com/index.php
- JdeBP 8y agoContrast a different explanation discussed at https://news.ycombinator.com/item?id=17326691 https://news.ycombinator.com/item?id=17326691
- JdeBP 8y agoI notice that I and M. Siebenmann reacted to this in similar ways. (-:
- qazwsxedcrfv1 8y agowow.check out this cool forum http://mawazo.byethost7.com/index.php http://mawazo.byethost7.com/index.php
- organsnyder 8y agoWhen I was first learning Linux (almost 20 years ago [yikes]), I installed it by hand using the Linux From Scratch guide. It was invaluable for learning a ton about how Linux works at a deep level. While a lot has changed since then (mainly due to UEFI), that knowledge is still valuable for understanding how a system functions. http://linuxfromscratch.org/ http://linuxfromscratch.org/
- BJanecke 8y agoA bit more lightweight version of that is https://wiki.archlinux.org/index.php/installation_guide https://wiki.archlinux.org/index.php/installation_guide The first time I tried to install arch I had many aha! moments
- zshrdlu 8y agoSimilarly, the Gentoo Installation Handbook: https://wiki.gentoo.org/wiki/Handbook:AMD64 https://wiki.gentoo.org/wiki/Handbook:AMD64
- watersb 8y agoGentoo taught me Linux. The Handbooks are really good. I think that much of early Gentoo ideas - the OpenRC init system, the ports system, the handbooks - were inspired by BSD traditions. OpenBSD and FreeBSD each have good documentation. Working through the FreeBSD handbook, and then study of McKusik's BSD Book [0], are a good way to get another perspective if you get into this sort of thing. [0] http://www.worldcat.org/title/design-and-implementation-of-the-freebsd-operating-system/oclc/870847694 http://www.worldcat.org/title/design-and-implementation-of-t...
- BrandoElFollito 8y agoWhen I was hiring junior sydadmins. I had them go through an install of Gentoo (some 15 years ago so things may have changed). It was always an eyes opening experience for them.
- watersb 8y agoGoogle terms: "Early Userspace"