3 ms·

Kudos to everyone involved in this! Love everything about this. Using it on my notebook, on dedicated servers rented at Hetzner as well as on Hetzner-Cloud, as

by nisa 3y ago

Kudos to everyone involved in this! Love everything about this. Using it on my notebook, on dedicated servers rented at Hetzner as well as on Hetzner-Cloud, as well on a bunch of dedicated servers in a rack. Solves almost all problems related to ZFS and Linux. Booting this from SYSLINUX works very well as well as UEFI - it's extensible and you can run it with the ZFS git version if you use the generate-zbm command. Saved my ass quite a few times already.

SushiHippie 3y ago
May I ask how you use this on the dedicated server?
- wongarsu 3y ago
  Hetzner's dedicated servers give you KVM access for stuff like tweaking your bios settings, installing an OS, or I guess using your boot menu. You have to request them via support ticket, but last time I did that I got it within 5 minutes, no questions asked. If you have your own server in a rack somewhere chances are you bought one with a similar web interface (IMPI/BMC/whatever your brand calls it) on a separate always-on NIC on the mainboard. https://docs.hetzner.com/robot/dedicated-server/maintainance/kvm-console/ https://docs.hetzner.com/robot/dedicated-server/maintainance...
  - codetrotter 3y ago
    When I installed FreeBSD on my Hetzner servers, I did so by booting the servers into the Linux based rescue mode and then I think I used dd to write the mfsBSD media onto one of the hard drives. This way I didn’t have to request KVM access for my servers. Perhaps a similar method can be used in order to install ZFSBootMenu
    - themoonisachees 3y ago
      Nice tip but if you fail your server is toast and that's an interesting support ticket.
      - codetrotter 3y ago
        Their Linux based rescue mode boots from network. I should probably specify though that I do this when setting up a new server. If you fail then it should still be possible to select their Linux rescue image again from the Hetzner control panel and force reboot the server. I seem to recall that I had to try a couple of times before I got it right. And that I did exactly that; select their rescue image again and force reboot.
      - wkat4242 3y ago
        Not a big deal though when you're doing a fresh install. Just cancel the server and provision a new one.
      - numpad0 3y ago
        rootfs should be network mounted by NIC feature on most clouds, so it should just be the matter of clicking "Force shutdown" and "Reset disk to default install"
- nisa 3y ago
  As others have pointed out you can request a KVM-console (called LARA there IIRC) for 3 hours for free - however if you install it correctly you can setup everything in the rescue system without requesting a KVM console. We have lot's of data to store and I'm using it with additional 16TB disks in a server and using the 2x enterprise SSD for the special metadata vdev - it's really nice to do a setfacl -R on 5 million files and 12TB data in 15seconds.
imiric 3y ago
Void is my favorite distro, and I'm interested in giving this setup a try. So far I've been reluctant to use ZFS on Linux out of reliability concerns. Have you had any issues with this setup, or have any suggestions?
- nisa 3y ago
  Setup works fine. Biggest issue was messing up the initramfs and the kernel-module build (I've used the now 2.2.0rc and before that git because I wanted to have overlayfs support and idmapped mounts - machines running docker and lxd - so everything is native in ZFS). One issue to look out for is the activated features and the ZFS module in the kernel and in ZFSBootMenu - these should match as depending on the features it's not possible to import the pool otherwise. zfsbootmenu just looks for kernels and boots them. If something fails you can go back to an older snapshot or use an older kernel - and you have an emergency shell.
tehlike 3y ago
how do you use this with hetzner-cloud? Do you use it as a vm or as part of, say, k8s?
- nisa 3y ago
  I wanted to have snapshots and installed just Ubuntu on ZFS using debboostrap - basically using the tutorial on the page here almost verbatim. Hetzner Cloud is not using UEFI afaik so you need to setup a small SYSLINUX setup to boot. But there is no ready cloud image for that. Another idea I didn't implement yet is to have a minimal image that I can bootstrap using send/recv but at the moment there's no need for that.

by nisa 3y ago

SushiHippie 3y ago

May I ask how you use this on the dedicated server?

wongarsu 3y ago
Hetzner's dedicated servers give you KVM access for stuff like tweaking your bios settings, installing an OS, or I guess using your boot menu. You have to request them via support ticket, but last time I did that I got it within 5 minutes, no questions asked. If you have your own server in a rack somewhere chances are you bought one with a similar web interface (IMPI/BMC/whatever your brand calls it) on a separate always-on NIC on the mainboard. https://docs.hetzner.com/robot/dedicated-server/maintainance/kvm-console/ https://docs.hetzner.com/robot/dedicated-server/maintainance...
- codetrotter 3y ago
  When I installed FreeBSD on my Hetzner servers, I did so by booting the servers into the Linux based rescue mode and then I think I used dd to write the mfsBSD media onto one of the hard drives. This way I didn’t have to request KVM access for my servers. Perhaps a similar method can be used in order to install ZFSBootMenu
  - themoonisachees 3y ago
    Nice tip but if you fail your server is toast and that's an interesting support ticket.
    - codetrotter 3y ago
      Their Linux based rescue mode boots from network. I should probably specify though that I do this when setting up a new server. If you fail then it should still be possible to select their Linux rescue image again from the Hetzner control panel and force reboot the server. I seem to recall that I had to try a couple of times before I got it right. And that I did exactly that; select their rescue image again and force reboot.
    - wkat4242 3y ago
      Not a big deal though when you're doing a fresh install. Just cancel the server and provision a new one.
    - numpad0 3y ago
      rootfs should be network mounted by NIC feature on most clouds, so it should just be the matter of clicking "Force shutdown" and "Reset disk to default install"
nisa 3y ago
As others have pointed out you can request a KVM-console (called LARA there IIRC) for 3 hours for free - however if you install it correctly you can setup everything in the rescue system without requesting a KVM console. We have lot's of data to store and I'm using it with additional 16TB disks in a server and using the 2x enterprise SSD for the special metadata vdev - it's really nice to do a setfacl -R on 5 million files and 12TB data in 15seconds.

imiric 3y ago

Void is my favorite distro, and I'm interested in giving this setup a try. So far I've been reluctant to use ZFS on Linux out of reliability concerns. Have you had any issues with this setup, or have any suggestions?

nisa 3y ago
Setup works fine. Biggest issue was messing up the initramfs and the kernel-module build (I've used the now 2.2.0rc and before that git because I wanted to have overlayfs support and idmapped mounts - machines running docker and lxd - so everything is native in ZFS). One issue to look out for is the activated features and the ZFS module in the kernel and in ZFSBootMenu - these should match as depending on the features it's not possible to import the pool otherwise. zfsbootmenu just looks for kernels and boots them. If something fails you can go back to an older snapshot or use an older kernel - and you have an emergency shell.

tehlike 3y ago

how do you use this with hetzner-cloud? Do you use it as a vm or as part of, say, k8s?

nisa 3y ago
I wanted to have snapshots and installed just Ubuntu on ZFS using debboostrap - basically using the tutorial on the page here almost verbatim. Hetzner Cloud is not using UEFI afaik so you need to setup a small SYSLINUX setup to boot. But there is no ready cloud image for that. Another idea I didn't implement yet is to have a minimal image that I can bootstrap using send/recv but at the moment there's no need for that.