4 ms·

Somewhat related, I recently watched: "By the numbers: ZFS Performance Results from Six Operating Systems and Their Derivatives" https://www.youtube.com/watch

by e12e 7y ago

Somewhat related, I recently watched: "By the numbers: ZFS Performance Results from Six Operating Systems and Their Derivatives" https://www.youtube.com/watch?v=HrUvjocWopI https://www.youtube.com/watch?v=HrUvjocWopI Slides: https://callfortesting.org/log/vBSDcon-2019-Dexter-BytheNumbers.pdf https://callfortesting.org/log/vBSDcon-2019-Dexter-BytheNumb... And was surprised that zfs for windows (and hyper-v core) appear to be alive and comming along. Anyone using zfs on windows? Are they also going off of zfsonlinux - and if so how's support for encryption comming? Finally having shared, read write, encrypted home that works on both Linux and windows safely would be great!

kop316 7y ago
I used zfs on Linux with encrypted / for a while on Debian. It works really well, but unfortunately when I upgraded from Jessie to Buster, it did not recognize how I set it up and broke the system. I ended up having to reinstall.
- oarsinsync 7y ago
  Did you take a snapshot before performing the upgrade, and did you try to restore the previous snapshot before the reinstall? Curious if snapshots can protect you even as far as OS upgrades gone wrong. I'd assume they can, if all filesystems are ZFS + snapshotted.
  - ohthehugemanate 7y ago
    I've used (conceptually equivalent) btrfs snapshots to let me install arch and gentoo, always easily rolling back to my own heavily customized Ubuntu system. I've never tried it, but given a copy of the snapshot tools, you should even be able to rollback from formatting your root partition.
    - pm7 7y ago
      I think there are even some tools to do this directly in GRUB, at least for btrfs. We can have automatically created list of available snapshots.
  - kop316 7y ago
    To be honest I didn't feel like tinkering with it, I just reinstalled. Right now it is on ext4. My issue was that for some reason Linux no longer could find the encrypted partition to decrypt, so I couldn't get it up to a point where I could even work with it.
    - oarsinsync 7y ago
      Ah that's a shame. If you'd had a bootable USB stick or some pre-boot environment to use, you probably could have rolled back from a snapshot. At least, that's my understanding of the theory. Your experience has definitely not discouraged me from wanting to ZFS my root partitions though. If only to provide the potential / theoretical ability to roll back a snapshot if necessary!
      - kop316 7y ago
        To be honest, it was laziness on my part, I could have probably figured it out and fixed it. The snapshot ability was incredibly nice, don't get me wrong. I really liked it on my system. I may switch back in the near future. I can post my "script" when I get home if you want to try it. I use script very liberally, it was my notes of the commands I used in order in order to get ZFS/dmcrypt working.
- e12e 7y ago
  Hm, you used zfsonlinux 0.8 under Jessie? Or was this prior to general release of native encryption support?
  - kop316 7y ago
    What do you mean by "native" encryption? I used dmcrypt+LUKS. That is file system agonistic.
    - e12e 7y ago
      That's not zfs encryption, that's "just" encryption (ed: with zfs on top). (open) Zfs now has native encryption: https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.8.0 https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.8.0 (ed: there are some potential benefits to zfs encryption - it's as cross platform as zfs (encrypt an USB drive, use on Linux, os x, windows, freebsd). Zfs send/receive - receiver can take your encrypted backup - key doesn't have to travel. Mount drives across dual boot - share encrypted /home between dual boot windows and Linux? Not sure - but I think volume encryption is supported too. Might simplify encrypted swap (if you already "pay" for zfs). Note: there are apparently some issues with swap on zfs (volumes at least, possibly files too) If you don't need suspend/resume - suppose you could share one encrypted swap volume across multiboot environments..) )
      - kop316 7y ago
        Heh, ok neat! I did not use that then.
        e12e 7y ago
        If you did on Jessie I'd be more than a little impressed! :)
- stiray 7y ago
  zfs import -af?
  - kop316 7y ago
    Like I said elsewhere, I could have probably fixed it, but I was lazy. I also wanted to try out Wayland, but I ended up going back to Xfce. Maybe when Xfce moves to Wayland I will try again.
- e12e 7y ago
  Related: I moved to a new laptop at work with a zfs send over ssh of /home/myuser - and a mount+manual copy of what I wanted. Very straight forward - but not that different from an rsync to /home/myuser/olfstuff. Zfs encryption was still a bit new, so both boxes uses dmcrypt on full disk plus efi plus /boot (technically redundant, but it works). Looking forward to hopefully root on encrypted zfs in next Ubuntu lts.
- e12e 7y ago
  BTW, did you do a traditional in-place upgrade via apt? I've once or twice managed to break boot between Debian releases.. But not since lilo/early grub days. Sounds like you probably ran into zfs not being in Debian proper - and buster unlocking dmcrypt, but not being able to mount zfs? Should normally be salvageable via booting previous Jessie kernel and initial ram/root disk (initrd). At any rate, I gather you've moved on - just sound like an unlucky break - even with root on unsupported zfs.
  - kop316 7y ago
    Yeah I did an in-place upgrade via apt. I for some reason thought zfs was in debian proper (contrib vs. main). Like I said in another post, I may try doing that again. But then again, the main reason I haven't is because all of my stuff is backed up via Nextcloud, so reinstalling Debian is pretty quick/painless for me.
jiggawatts 7y ago
The ZFS-equivalent on Windows is meant to be ReFS, which is roughly equivalent to ZFS when layered on top of Storage Spaces. I'm really keen to give ReFS+SS a go, but I keep hearing horror stories of abysmal parity-spaces performance, which can only be explained by poor coding and testing. Even if ZFS has better performance, it's never going to have full support, and it wasn't written for the NT kernel IO model, so I'm dubious about it ever working well...
- p_l 7y ago
  There is less mismatch between ZFS and WinNT IO model than between ZFS and Linux VFS. Sincerely, someone who had to port kernel level code from Linux to NT.
- fuzzy2 7y ago
  I once tried ReFS, back on Windows 8.1 IIRC. Everything was going fine until suddenly, in the middle of the day, Windows decided that the ReFS volume was corrupt. And then you discover there’s virtually no rescue tools. It nearly gave me a heart attack. After rebooting three times it decided everything was okay again. I quickly moved away my data and never used ReFS since.
  - xenospn 7y ago
    That happened to me with a beta version of APFS. I never got my data back.
- e12e 7y ago
  This entirely misses the fact that zfs (open zfs/zfsonlinux) is open and cross platform. Which is almost more interesting than all the good stuff that comes with zfs (volume management, disk management, software raid-like functionality, encryption, snapshots, send/receive). Its the first time we truly have a half-modern filesystem with safe write support and sane performance across many different os's. Even if just made dual booting freebsd and Linux this would be a fmgreat win imnho. If the choice is between zfs and fat32...
  - 7y ago
aasasd 7y ago
Isn't encryption supposed to be a separate layer from the filesystem? Or is the merger specific to Windows? Or, is this ZFS' attitude toward features, like with the volume manager?
- cyphar 7y ago
  ZFS "native encryption" is an upstream feature. The argument for doing it in the filesystem rather than as a separate layer is that it allows you to have far more flexibility than classic FDE (encrypted send/recv, you can use different keys for each filesystem, the encryption applies to all data written by ZFS so you don't need to worry about forgetting to set up LUKS on a new drive added to your pool, etc.)
- e12e 7y ago
  In addition to what @cyphar mentions - zfs is "storage manager". Disk, volume and filesystem. Some of the power and data guarantees relate to this design decision. Encrypted /home/username filesystem with user-owned keys, is one neat trick,for example.
- pm7 7y ago
  > How many drives can you fit into a laptop? Why? One more addition to other replies: ZFS needs access to disks for self-healing (bit-rot detection and repair). It means that in the past if we wanted encryption, same date needed to be encrypted twice (one for every mirror). Now with encryption in ZFS, date is encrypted before creating redundancy. We can even do a scrub without encryption key.
- kop316 7y ago
  That is how Linux and FreeBSD handle it. They both present a block layer device that you can then put ZFS on. The ZFS specific encryption IIRC is owned by Oracle, and they do not open source it.
  - e12e 7y ago
    Oracle "closed" (forked) zfs "back" and added encryption. But encryption is (now) in openzfs (who's canonical upstream has gravitated to "zfsonlinux" which is just the open project that ended up having the major mind-share). https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.8.0 https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.8.0

by e12e 7y ago

kop316 7y ago

I used zfs on Linux with encrypted / for a while on Debian. It works really well, but unfortunately when I upgraded from Jessie to Buster, it did not recognize how I set it up and broke the system. I ended up having to reinstall.

oarsinsync 7y ago
Did you take a snapshot before performing the upgrade, and did you try to restore the previous snapshot before the reinstall? Curious if snapshots can protect you even as far as OS upgrades gone wrong. I'd assume they can, if all filesystems are ZFS + snapshotted.
- ohthehugemanate 7y ago
  I've used (conceptually equivalent) btrfs snapshots to let me install arch and gentoo, always easily rolling back to my own heavily customized Ubuntu system. I've never tried it, but given a copy of the snapshot tools, you should even be able to rollback from formatting your root partition.
  - pm7 7y ago
    I think there are even some tools to do this directly in GRUB, at least for btrfs. We can have automatically created list of available snapshots.
- kop316 7y ago
  To be honest I didn't feel like tinkering with it, I just reinstalled. Right now it is on ext4. My issue was that for some reason Linux no longer could find the encrypted partition to decrypt, so I couldn't get it up to a point where I could even work with it.
  - oarsinsync 7y ago
    Ah that's a shame. If you'd had a bootable USB stick or some pre-boot environment to use, you probably could have rolled back from a snapshot. At least, that's my understanding of the theory. Your experience has definitely not discouraged me from wanting to ZFS my root partitions though. If only to provide the potential / theoretical ability to roll back a snapshot if necessary!
    - kop316 7y ago
      To be honest, it was laziness on my part, I could have probably figured it out and fixed it. The snapshot ability was incredibly nice, don't get me wrong. I really liked it on my system. I may switch back in the near future. I can post my "script" when I get home if you want to try it. I use script very liberally, it was my notes of the commands I used in order in order to get ZFS/dmcrypt working.
e12e 7y ago
Hm, you used zfsonlinux 0.8 under Jessie? Or was this prior to general release of native encryption support?
- kop316 7y ago
  What do you mean by "native" encryption? I used dmcrypt+LUKS. That is file system agonistic.
  - e12e 7y ago
    That's not zfs encryption, that's "just" encryption (ed: with zfs on top). (open) Zfs now has native encryption: https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.8.0 https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.8.0 (ed: there are some potential benefits to zfs encryption - it's as cross platform as zfs (encrypt an USB drive, use on Linux, os x, windows, freebsd). Zfs send/receive - receiver can take your encrypted backup - key doesn't have to travel. Mount drives across dual boot - share encrypted /home between dual boot windows and Linux? Not sure - but I think volume encryption is supported too. Might simplify encrypted swap (if you already "pay" for zfs). Note: there are apparently some issues with swap on zfs (volumes at least, possibly files too) If you don't need suspend/resume - suppose you could share one encrypted swap volume across multiboot environments..) )
    - kop316 7y ago
      Heh, ok neat! I did not use that then.
      - e12e 7y ago
        If you did on Jessie I'd be more than a little impressed! :)
stiray 7y ago
zfs import -af?
- kop316 7y ago
  Like I said elsewhere, I could have probably fixed it, but I was lazy. I also wanted to try out Wayland, but I ended up going back to Xfce. Maybe when Xfce moves to Wayland I will try again.
e12e 7y ago
Related: I moved to a new laptop at work with a zfs send over ssh of /home/myuser - and a mount+manual copy of what I wanted. Very straight forward - but not that different from an rsync to /home/myuser/olfstuff. Zfs encryption was still a bit new, so both boxes uses dmcrypt on full disk plus efi plus /boot (technically redundant, but it works). Looking forward to hopefully root on encrypted zfs in next Ubuntu lts.
e12e 7y ago
BTW, did you do a traditional in-place upgrade via apt? I've once or twice managed to break boot between Debian releases.. But not since lilo/early grub days. Sounds like you probably ran into zfs not being in Debian proper - and buster unlocking dmcrypt, but not being able to mount zfs? Should normally be salvageable via booting previous Jessie kernel and initial ram/root disk (initrd). At any rate, I gather you've moved on - just sound like an unlucky break - even with root on unsupported zfs.
- kop316 7y ago
  Yeah I did an in-place upgrade via apt. I for some reason thought zfs was in debian proper (contrib vs. main). Like I said in another post, I may try doing that again. But then again, the main reason I haven't is because all of my stuff is backed up via Nextcloud, so reinstalling Debian is pretty quick/painless for me.

jiggawatts 7y ago

The ZFS-equivalent on Windows is meant to be ReFS, which is roughly equivalent to ZFS when layered on top of Storage Spaces. I'm really keen to give ReFS+SS a go, but I keep hearing horror stories of abysmal parity-spaces performance, which can only be explained by poor coding and testing. Even if ZFS has better performance, it's never going to have full support, and it wasn't written for the NT kernel IO model, so I'm dubious about it ever working well...

p_l 7y ago
There is less mismatch between ZFS and WinNT IO model than between ZFS and Linux VFS. Sincerely, someone who had to port kernel level code from Linux to NT.
fuzzy2 7y ago
I once tried ReFS, back on Windows 8.1 IIRC. Everything was going fine until suddenly, in the middle of the day, Windows decided that the ReFS volume was corrupt. And then you discover there’s virtually no rescue tools. It nearly gave me a heart attack. After rebooting three times it decided everything was okay again. I quickly moved away my data and never used ReFS since.
- xenospn 7y ago
  That happened to me with a beta version of APFS. I never got my data back.
e12e 7y ago
This entirely misses the fact that zfs (open zfs/zfsonlinux) is open and cross platform. Which is almost more interesting than all the good stuff that comes with zfs (volume management, disk management, software raid-like functionality, encryption, snapshots, send/receive). Its the first time we truly have a half-modern filesystem with safe write support and sane performance across many different os's. Even if just made dual booting freebsd and Linux this would be a fmgreat win imnho. If the choice is between zfs and fat32...
- 7y ago

aasasd 7y ago

Isn't encryption supposed to be a separate layer from the filesystem? Or is the merger specific to Windows? Or, is this ZFS' attitude toward features, like with the volume manager?

cyphar 7y ago
ZFS "native encryption" is an upstream feature. The argument for doing it in the filesystem rather than as a separate layer is that it allows you to have far more flexibility than classic FDE (encrypted send/recv, you can use different keys for each filesystem, the encryption applies to all data written by ZFS so you don't need to worry about forgetting to set up LUKS on a new drive added to your pool, etc.)
e12e 7y ago
In addition to what @cyphar mentions - zfs is "storage manager". Disk, volume and filesystem. Some of the power and data guarantees relate to this design decision. Encrypted /home/username filesystem with user-owned keys, is one neat trick,for example.
pm7 7y ago
> How many drives can you fit into a laptop? Why? One more addition to other replies: ZFS needs access to disks for self-healing (bit-rot detection and repair). It means that in the past if we wanted encryption, same date needed to be encrypted twice (one for every mirror). Now with encryption in ZFS, date is encrypted before creating redundancy. We can even do a scrub without encryption key.
kop316 7y ago
That is how Linux and FreeBSD handle it. They both present a block layer device that you can then put ZFS on. The ZFS specific encryption IIRC is owned by Oracle, and they do not open source it.
- e12e 7y ago
  Oracle "closed" (forked) zfs "back" and added encryption. But encryption is (now) in openzfs (who's canonical upstream has gravitated to "zfsonlinux" which is just the open project that ended up having the major mind-share). https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.8.0 https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.8.0