4 ms·

The problem is, ZFS support on GRUB2 hasn't been great, partly due to CDDL/GPL licensing incompatibility requiring lots of ZFS internals to be re-implemented in

by sirn 4y ago

The problem is, ZFS support on GRUB2 hasn't been great, partly due to CDDL/GPL licensing incompatibility requiring lots of ZFS internals to be re-implemented in GRUB. This resulted in issues such as grub-probe unable to detect ZFS pools due to unsupported ZFS features[1] (including native ZFS encryption, which is a deal-breaker for many) ZBM took another approach. It provides a small initramfs image that are built on the host machine via standard method such as dracut or mkinitcpio. This image provides an interface for decrypting/mounting ZFS filesystems using the very same ZFS kernel module and tools installed on the host. After the filesystem is mounted, it then kexec'd into the host kernel. This also means ZBM doesn't completely replaced GRUB2 or syslinux. Instead, it rely on those intermediate bootloader (including EFI bootloader such as rEFId/gummiboot) to load ZBM itself. (Though ZBM itself only has built-in hooks for syslinux and gummiboot). Being an initramfs give an extra benefit of providing interesting mechanism during boot e.g. providing a SSH server for entering an encryption key on a headless server[2], ability to discover, managing/booting from ZFS snapshots, etc. (No affiliate; just a very happy user.) [1]: https://savannah.gnu.org/bugs/?58555 https://savannah.gnu.org/bugs/?58555 [2]: https://github.com/zbm-dev/zfsbootmenu/wiki/Remote-Access-to-ZBM https://github.com/zbm-dev/zfsbootmenu/wiki/Remote-Access-to...

matja 4y ago
Wouldn't kexec with a mounted filesystem lose the filesystem state because the kernel heap+stack is overwritten? I think ZBM copies the kernel/initramfs from the ZFS dataset (presumably to tmpfs), unmounts/exports, then kexec's, and the new initramfs imports/mounts the pool/dataset as usual?
- sirn 4y ago
  My understanding is that during the boot process using ZBM's initramfs: 1. ZBM prompt for encryption passphrase, decrypts the filesystem, locate kernel/initramfs on ZFS datasets, then display boot menu 2. ZBM kexec into the kernel on the filesystem using the chosen kernel/initramfs while appending root=zfs:... to the kernel parameter 3. The target kernel decrypts the filesystem[^] and mounts the root ZFS again and boot into final system [^]: In this case, ZBM requires the encryption key to be placed in the target initramfs (not ZBM's) for the target kernel to load (dataset need to be decrypted again since kernel state is disregarded). This initramfs is located inside the encrypted filesystem itself, only accessible after initial decryption/mount by ZBM in step 1, so the only way to obtain this key is to already have access to encrypted filesystem in the first place.
  - E39M5S62 4y ago
    That's exactly right. We also append spl.spl_hostid to the command line, to work around any possible hostid mismatches inside the boot environment.
    - sirn 4y ago
      Thank you for such a great tool. I've recently migrated from one server to another server in different contenient via `zfs send | zfs recv` (using hrmpf), and `generate-zbm` inside the chroot was all I need to get it working again.
- E39M5S62 4y ago
  Since the pool itself is imported read-only by default, there's no state to keep. We don't even need to export the pool, no txg's can be generated and left in a pending state. If a pool is switched to read-write so that the default kernel can be set, a snapshot cloned to a new BE, etc, we check for that and then export the pool just before kexec. Once kexec is done, your BEs kernel and initramfs essentially start fresh and actas if it's a fresh boot.

by sirn 4y ago

matja 4y ago

Wouldn't kexec with a mounted filesystem lose the filesystem state because the kernel heap+stack is overwritten? I think ZBM copies the kernel/initramfs from the ZFS dataset (presumably to tmpfs), unmounts/exports, then kexec's, and the new initramfs imports/mounts the pool/dataset as usual?

sirn 4y ago
My understanding is that during the boot process using ZBM's initramfs: 1. ZBM prompt for encryption passphrase, decrypts the filesystem, locate kernel/initramfs on ZFS datasets, then display boot menu 2. ZBM kexec into the kernel on the filesystem using the chosen kernel/initramfs while appending root=zfs:... to the kernel parameter 3. The target kernel decrypts the filesystem[^] and mounts the root ZFS again and boot into final system [^]: In this case, ZBM requires the encryption key to be placed in the target initramfs (not ZBM's) for the target kernel to load (dataset need to be decrypted again since kernel state is disregarded). This initramfs is located inside the encrypted filesystem itself, only accessible after initial decryption/mount by ZBM in step 1, so the only way to obtain this key is to already have access to encrypted filesystem in the first place.
- E39M5S62 4y ago
  That's exactly right. We also append spl.spl_hostid to the command line, to work around any possible hostid mismatches inside the boot environment.
  - sirn 4y ago
    Thank you for such a great tool. I've recently migrated from one server to another server in different contenient via `zfs send | zfs recv` (using hrmpf), and `generate-zbm` inside the chroot was all I need to get it working again.
E39M5S62 4y ago
Since the pool itself is imported read-only by default, there's no state to keep. We don't even need to export the pool, no txg's can be generated and left in a pending state. If a pool is switched to read-write so that the default kernel can be set, a snapshot cloned to a new BE, etc, we check for that and then export the pool just before kexec. Once kexec is done, your BEs kernel and initramfs essentially start fresh and actas if it's a fresh boot.