6 ms·

Enhancing our ZFS support on Ubuntu 19.10

116 points by anuragsoni 7y ago

gen3 7y ago
I've been using zfs on Debian for over half a year now. I've been really happy with the stability, going through a handful of power outages and the addition of another drive.
- dev_dull 7y ago
  Me too. I’ve been really surprised at how easy it is. Even migrating to a fresh OS was a breeze and didn’t require me backing up any configs.
  - steeleduncan 7y ago
    Do you mean ZFS as root or as a data partition? and when you say migrating to a fresh OS do you mean the Stretch to Buster upgrade? If so, it is great to hear it is working so well now.
    - GrayShade 7y ago
      Not the grandparent, but I've moved a running system from one ZFS pool to another without physically touching the computer. The OS was running off a ZFS root.
    - dev_dull 7y ago
      ZFS data volume. Haven't tried root yet (wasn't easily available when I created the volume). Still, a data volume in ZFS is so much better than the traditional md raid. Won't go back.
- gralx 7y ago
  ZFS lets me back up every `apt-get` action, which is useful on Debian Testing: /etc/apt/apt.conf.d/71backup DPkg::Pre-Install-Pkgs {"/sbin/zfs snapshot rpool/ROOT/debian@apt-get_$(date '+%Y-%m-%d-%H%M%S')";}; When I inevitably break Debian with no time to troubleshoot, I do # zfs list -t snapshot | grep apt\-get ... # zfs rollback -r rpool/ROOT/debian@apt-get_<insert time stamp here>
  - dddddaviddddd 7y ago
    I've set up something similar on FreeBSD with just a shell function wrapping pkg and port, very handy.
  - isostatic 7y ago
    Nice. I haven't touched ZFS since it first came out on Solaris around 2008ish (we had a 48 drive "x4500", great hardware, but solaris :/ ) Are the snapshots instant? I'd be tempted to replace "sudo" with something that snapshotted them too!
    - GrayShade 7y ago
      > Are the snapshots instant? Yes. But you have to remember to delete the old ones at some point.
    - masklinn 7y ago
      > Are the snapshots instant? Yes, it just copies & stores the block at the top of the filesystem[0], in the same way naming a git tree will keep its content recursively alive. However all the data in the snapshot is retained as-is until the snapshot is destroyed, so as the system keeps being used and diverges from the snapshot the snapshot is slowly going to "increase in size" compared to the live FS. [0] there's also a more recent "checkpoint" concept which "saves" the entire pool by copying the uberblock.
    - zenexer 7y ago
      The actual act of snapshotting doesn’t do a whole lot. I believe ZFS uses CoW for that purpose. If you modify a file after a snapshot, the file (or block) will be copied first. Normally, with CoW, the original would then be marked as free space available for writing, but if a snapshot exists, it will be preserved. (This is a general concept; I don’t know the nuances of ZFS’ approach.)
      - isostatic 7y ago
        I thought that was the case, but in more recent times they only snapshotting I've done is on AWS instances which take forever
flyinghamster 7y ago
Having been a ZFS fan since the twilight of OpenSolaris, I'm very glad to see ZoL taking off. Rolling it into Ubuntu and making it officially supported was a great move - after some frustration with trying to run ZFS on a CentOS box and having it occasionally break after a kernel update, having it easily available on Ubuntu was like a breath of fresh air. Having it readily available as a root filesystem, and having TRIM support at long last, is great news.
lousken 7y ago
I wonder how's the performance compared to ext4. I've only seen some old articles and not sure if it's comparable to the latest version
- pizza234 7y ago
  Here's a recent test: https://www.phoronix.com/scan.php?page=article&item=bcachefs-linux-2019 https://www.phoronix.com/scan.php?page=article&item=bcachefs... It needs to be said though, that ZFS requires tweaking for some workloads. In particular, database volumes needs to have the recordsize set to 8 KiB, in order to match the DB page size (I think the benchmark did not do that).
rewtraw 7y ago
Happy ZoL user on my home NAS for over 6 years now. 60TB+, not a single byte lost. Very, very happy to see the continued support.
Wowfunhappy 7y ago
Semi-offtopic, but in the article, they link the reason why Ubuntu considers it legal to include binary ZFS: > The CDDL cannot apply to the Linux kernel because zfs.ko is a self-contained file system module — the kernel itself is quite obviously not a derivative work of this new file system. And zfs.ko, as a self-contained file system module, is clearly not a derivative work of the Linux kernel but rather quite obviously a derivative work of OpenZFS and OpenSolaris. This makes lots of sense to me as a layperson. Can anyone explain what the counter argument is? If a non-GPL binary zfs.ko isn't allowed, why is a fully-proprietary Celeste.sh (a commercial video game) okay? Because Celeste has fewer privileges?
- snuxoll 7y ago
  > why is a fully-proprietary Celeste.sh (a commercial video game) okay? Because Celeste has fewer privileges? Because Celeste doesn't link into the kernel, there is a system call exemption in the kernel license to make it abundantly clear that user-mode applications are not considered combined or derived works of the kernel. As far as ZFS.ko is concerned, there's already history of non-GPL filesystem modules being included in the kernel; namely afs. It's the same argument, AFS was designed for another UNIX-like operating system with a similar VFS layer and - rationale that Linus agreed with. It's really other maintainers at this point that are stonewalling ZFS.ko, an issue that will ultimately come down to what a court decides constitutes a derived work.
- cesarb 7y ago
  It depends on the definition of "derived work", which is a legal definition, not a technical definition. For "user programs that use kernel services by normal system calls", everyone agrees that it's not a derived work ("[...] this is merely considered normal use of the kernel, and does not fall under the heading of "derived work"."). For kernel modules, which use the internal kernel API instead of system calls, it's less clear. Also, you cannot say that zfs.ko "is clearly not a derivative work of the Linux kernel but rather quite obviously a derivative work of OpenZFS and OpenSolaris" - if that were the case, that kernel module clearly wouldn't work, since the internal kernel API of the Linux kernel and of the Solaris kernel is completely different. Whether the changes necessary for zfs.ko to work with the Linux kernel make zfs.ko as a whole a "derived work" of the Linux kernel or not is a more complex question. But that's not the worst issue. The real problem is: under what license is the whole kernel, once you linked together the common Linux parts (which are under GPL2) and the ZFS parts (which are under the CDDL)? The "mere aggregation" provisions of GPL2 clearly don't apply, and the licenses are incompatible (on purpose, it's been documented that CDDL was intentionally made incompatible with the GPL). Since the CDDL can't apply (due to the GPL parts), and the GPL can't apply (due to the CDDL parts), distributing the combined whole would be a copyright violation.

116 points by anuragsoni 7y ago

gen3 7y ago

I've been using zfs on Debian for over half a year now. I've been really happy with the stability, going through a handful of power outages and the addition of another drive.

dev_dull 7y ago
Me too. I’ve been really surprised at how easy it is. Even migrating to a fresh OS was a breeze and didn’t require me backing up any configs.
- steeleduncan 7y ago
  Do you mean ZFS as root or as a data partition? and when you say migrating to a fresh OS do you mean the Stretch to Buster upgrade? If so, it is great to hear it is working so well now.
  - GrayShade 7y ago
    Not the grandparent, but I've moved a running system from one ZFS pool to another without physically touching the computer. The OS was running off a ZFS root.
  - dev_dull 7y ago
    ZFS data volume. Haven't tried root yet (wasn't easily available when I created the volume). Still, a data volume in ZFS is so much better than the traditional md raid. Won't go back.
gralx 7y ago
ZFS lets me back up every `apt-get` action, which is useful on Debian Testing: /etc/apt/apt.conf.d/71backup DPkg::Pre-Install-Pkgs {"/sbin/zfs snapshot rpool/ROOT/debian@apt-get_$(date '+%Y-%m-%d-%H%M%S')";}; When I inevitably break Debian with no time to troubleshoot, I do # zfs list -t snapshot | grep apt\-get ... # zfs rollback -r rpool/ROOT/debian@apt-get_<insert time stamp here>
- dddddaviddddd 7y ago
  I've set up something similar on FreeBSD with just a shell function wrapping pkg and port, very handy.
- isostatic 7y ago
  Nice. I haven't touched ZFS since it first came out on Solaris around 2008ish (we had a 48 drive "x4500", great hardware, but solaris :/ ) Are the snapshots instant? I'd be tempted to replace "sudo" with something that snapshotted them too!
  - GrayShade 7y ago
    > Are the snapshots instant? Yes. But you have to remember to delete the old ones at some point.
  - masklinn 7y ago
    > Are the snapshots instant? Yes, it just copies & stores the block at the top of the filesystem[0], in the same way naming a git tree will keep its content recursively alive. However all the data in the snapshot is retained as-is until the snapshot is destroyed, so as the system keeps being used and diverges from the snapshot the snapshot is slowly going to "increase in size" compared to the live FS. [0] there's also a more recent "checkpoint" concept which "saves" the entire pool by copying the uberblock.
  - zenexer 7y ago
    The actual act of snapshotting doesn’t do a whole lot. I believe ZFS uses CoW for that purpose. If you modify a file after a snapshot, the file (or block) will be copied first. Normally, with CoW, the original would then be marked as free space available for writing, but if a snapshot exists, it will be preserved. (This is a general concept; I don’t know the nuances of ZFS’ approach.)
    - isostatic 7y ago
      I thought that was the case, but in more recent times they only snapshotting I've done is on AWS instances which take forever

flyinghamster 7y ago

Having been a ZFS fan since the twilight of OpenSolaris, I'm very glad to see ZoL taking off. Rolling it into Ubuntu and making it officially supported was a great move - after some frustration with trying to run ZFS on a CentOS box and having it occasionally break after a kernel update, having it easily available on Ubuntu was like a breath of fresh air. Having it readily available as a root filesystem, and having TRIM support at long last, is great news.

lousken 7y ago

I wonder how's the performance compared to ext4. I've only seen some old articles and not sure if it's comparable to the latest version

pizza234 7y ago
Here's a recent test: https://www.phoronix.com/scan.php?page=article&item=bcachefs-linux-2019 https://www.phoronix.com/scan.php?page=article&item=bcachefs... It needs to be said though, that ZFS requires tweaking for some workloads. In particular, database volumes needs to have the recordsize set to 8 KiB, in order to match the DB page size (I think the benchmark did not do that).

rewtraw 7y ago

Happy ZoL user on my home NAS for over 6 years now. 60TB+, not a single byte lost. Very, very happy to see the continued support.

Wowfunhappy 7y ago

Semi-offtopic, but in the article, they link the reason why Ubuntu considers it legal to include binary ZFS: > The CDDL cannot apply to the Linux kernel because zfs.ko is a self-contained file system module — the kernel itself is quite obviously not a derivative work of this new file system. And zfs.ko, as a self-contained file system module, is clearly not a derivative work of the Linux kernel but rather quite obviously a derivative work of OpenZFS and OpenSolaris. This makes lots of sense to me as a layperson. Can anyone explain what the counter argument is? If a non-GPL binary zfs.ko isn't allowed, why is a fully-proprietary Celeste.sh (a commercial video game) okay? Because Celeste has fewer privileges?

snuxoll 7y ago
> why is a fully-proprietary Celeste.sh (a commercial video game) okay? Because Celeste has fewer privileges? Because Celeste doesn't link into the kernel, there is a system call exemption in the kernel license to make it abundantly clear that user-mode applications are not considered combined or derived works of the kernel. As far as ZFS.ko is concerned, there's already history of non-GPL filesystem modules being included in the kernel; namely afs. It's the same argument, AFS was designed for another UNIX-like operating system with a similar VFS layer and - rationale that Linus agreed with. It's really other maintainers at this point that are stonewalling ZFS.ko, an issue that will ultimately come down to what a court decides constitutes a derived work.
cesarb 7y ago
It depends on the definition of "derived work", which is a legal definition, not a technical definition. For "user programs that use kernel services by normal system calls", everyone agrees that it's not a derived work ("[...] this is merely considered normal use of the kernel, and does not fall under the heading of "derived work"."). For kernel modules, which use the internal kernel API instead of system calls, it's less clear. Also, you cannot say that zfs.ko "is clearly not a derivative work of the Linux kernel but rather quite obviously a derivative work of OpenZFS and OpenSolaris" - if that were the case, that kernel module clearly wouldn't work, since the internal kernel API of the Linux kernel and of the Solaris kernel is completely different. Whether the changes necessary for zfs.ko to work with the Linux kernel make zfs.ko as a whole a "derived work" of the Linux kernel or not is a more complex question. But that's not the worst issue. The real problem is: under what license is the whole kernel, once you linked together the common Linux parts (which are under GPL2) and the ZFS parts (which are under the CDDL)? The "mere aggregation" provisions of GPL2 clearly don't apply, and the licenses are incompatible (on purpose, it's been documented that CDDL was intentionally made incompatible with the GPL). Since the CDDL can't apply (due to the GPL parts), and the GPL can't apply (due to the CDDL parts), distributing the combined whole would be a copyright violation.