Have my server infrastructure on NixOS. Huge boost in productivity and stability, would never go back to standard linux. But man if something breaks it's a nuke

Have my server infrastructure on NixOS. Huge boost in productivity and stability, would never go back to standard linux. But man if something breaks it's a nuke going off. Sry for the long post, but thought I share my experience: Mass storage on a big encrypted RaidZ array of spinning rust, no issues. Bootloader, /boot, Encrypted Root and Databases on Mirrored NVME Drives. And man is that a nailbiter on each update. Setup my drives during 22.11 following NixOS Root on ZFS instructions [1], which were amended following reports of systems becoming unbootable [2] and mostly removed later [3]. Besides initially being a broken setup [4] with an increasing amount of mounts each update stalling any system writes and causing updates to fail, it became a well running machine after that was fixed. Then during the 24.05 update and with no config change, the system became unbootable [5]. After a tough recovery [6] I never figured out how to do mirrored bootloaders again, switched to a single bootloader setup. To this day I have interactions I don't understand and am trying to fix [7], which sometimes causes updates to knock services offline due to the `nixos-rebuild switch` process stopping services, going to update the bootloader, failing due to missing mounts and exiting with services being offline, prompting manual intervention. [1] https://github.com/openzfs/openzfs-docs/blob/1211e98faf1f37af1de5eb8f3ce0a1c87f71a0e6/docs/Getting%20Started/NixOS/Root%20on%20ZFS/1-preparation.rst https://github.com/openzfs/openzfs-docs/blob/1211e98faf1f37a... [2] https://github.com/openzfs/openzfs-docs/commit/1211e98faf1f37af1de5eb8f3ce0a1c87f71a0e6 https://github.com/openzfs/openzfs-docs/commit/1211e98faf1f3... [3] https://github.com/openzfs/openzfs-docs/commit/4fb5fb694f44c81639d28c1b73fce2093e7ccac4 https://github.com/openzfs/openzfs-docs/commit/4fb5fb694f44c... [4] https://github.com/NixOS/nixpkgs/issues/214871 https://github.com/NixOS/nixpkgs/issues/214871 [5] https://github.com/openzfs/openzfs-docs/issues/531 https://github.com/openzfs/openzfs-docs/issues/531 [6] https://github.com/openzfs/openzfs-docs/issues/531#issuecomment-2430689864 https://github.com/openzfs/openzfs-docs/issues/531#issuecomm... [7] https://discord.com/channels/568306982717751326/1328541099672997888/1328541099672997888 https://discord.com/channels/568306982717751326/132854109967...