4 ms·

I don’t really know that this is a particularly appropriate comparison. XFS is a fine filesystem, but ZFS is more than a filesystem, it’s RAID, LVM, deduplicati

I don’t really know that this is a particularly appropriate comparison. XFS is a fine filesystem, but ZFS is more than a filesystem, it’s RAID, LVM, deduplication, etc. The real competitors to ZFS from my understanding are largely BTRFS and proprietary technologies such as apple’s new APFS. I’d very much like to see a comparison between those. I’ve run ZFS on my systems for years, but moved away from it towards btrfs on my laptop, as it was just too much pain to keep the kernel modules up to date. I wish the CDDL were compatible with the GPL, cause if the drivers were in-tree there wouldn’t be any argument at all.

josteink 7y ago
With Ubuntu you get binary modules updated together with the kernel. I moved the opposite direction (btrfs->zfs) because I had issues with my main btrfs rootfs constantly degrading to read-only mode, causing my system to crash and require rebooting. With ZFS I’ve had no such issues. ZFS uses noticeably much more RAM though.
- noodlesUK 7y ago
  Yeah, I was on Gentoo at the time, which didn’t make it particularly difficult (when everything is source code it doesn’t really matter), but I’m on Fedora now, where the story is a bit different. Ubuntu shipping the binary modules is good. Do they support booting from ZFS simply (that is, are the kernel modules in the initramfs)?
  - josteink 7y ago
    Yeah. All my systems are pure ZFS systems. The initial install is a bit involved (live-cd, manual partitioning + debootstrap) but everything just works (tm) after that, including booting into specific snapshots from grub. There’s a good guide on github: https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-ZFS https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-...
vbezhenar 7y ago
If you need RAID and LVM, just use them? Separation of concepts is not a bad thing.
- madhadron 7y ago
  If there are really separate concepts, yes, but ZFS's creators, who came from virtual memory systems, not filesystems, made a compelling case that there weren't, and that these layers were historical hacks that cost more than they were worth. It's worth spending some time reading about this design.
zeec123 7y ago
I hope bcachefs is ready soon as another alternative.
Mister_Snuggles 7y ago
Personally, BTRFS feels like it has a ways to go before it's ready for prime-time. I've had two major and one minor BTRFS-related issues that have scared me away from it. 1) One of my computers got its BTRFS filesystem into a state where it would hang when trying to mount read/write. What I suspect is that there was some filesystem thing happening in the background when I rebooted the machine. I rebooted via the GUI and there was no sign that something was happening in the background, so this was really a normal thing that a user would do. No amount of fixing was able to get it back, but I was able to boot from the installation media, mount it read-only, and copy the data elsewhere. 2) Virtually all of the Linux servers at work will randomly hang for between minutes and hours. This was eventually traced to a BTRFS-scrub process that the OS vendor schedules to run weekly. The length and impact of the hang seemed to be based on how much write activity happens - servers where all the heavy activity happens on NFS mounts saw no impact, but servers that write a lot of logs to the local filesystem would get severely crippled on a weekly basis. We've moved a bunch of our write-heavy filesystems to non-BTRFS options as a result of this. 3) This is a more minor issue, but still speaks to my experience. I had a VM that was basically a remote desktop for me to use. Generally speaking it would hang hard after a few days of uptime with no actual usage. When I reinstalled it on a non-BTRFS (sorry, can't remember which filesystem I used) filesystem it was rock solid. I have no proof that this had anything to do with BTRFS. All of these were things that happened around a year ago, they may not be a true representation of the current state of BTRFS. But they've burned me badly, so now any use of BTRFS will be evaluated very carefully. In contrast, I've been running ZFS on a couple of FreeBSD servers, with fairly write-heavy loads, and have had no issues that were filesystem-related. Even under severe space and memory constraints ZFS has been rock solid.

josteink 7y ago

With Ubuntu you get binary modules updated together with the kernel. I moved the opposite direction (btrfs->zfs) because I had issues with my main btrfs rootfs constantly degrading to read-only mode, causing my system to crash and require rebooting. With ZFS I’ve had no such issues. ZFS uses noticeably much more RAM though.

noodlesUK 7y ago
Yeah, I was on Gentoo at the time, which didn’t make it particularly difficult (when everything is source code it doesn’t really matter), but I’m on Fedora now, where the story is a bit different. Ubuntu shipping the binary modules is good. Do they support booting from ZFS simply (that is, are the kernel modules in the initramfs)?
- josteink 7y ago
  Yeah. All my systems are pure ZFS systems. The initial install is a bit involved (live-cd, manual partitioning + debootstrap) but everything just works (tm) after that, including booting into specific snapshots from grub. There’s a good guide on github: https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-ZFS https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-...

vbezhenar 7y ago

If you need RAID and LVM, just use them? Separation of concepts is not a bad thing.

madhadron 7y ago
If there are really separate concepts, yes, but ZFS's creators, who came from virtual memory systems, not filesystems, made a compelling case that there weren't, and that these layers were historical hacks that cost more than they were worth. It's worth spending some time reading about this design.

zeec123 7y ago

I hope bcachefs is ready soon as another alternative.

Mister_Snuggles 7y ago

Personally, BTRFS feels like it has a ways to go before it's ready for prime-time. I've had two major and one minor BTRFS-related issues that have scared me away from it. 1) One of my computers got its BTRFS filesystem into a state where it would hang when trying to mount read/write. What I suspect is that there was some filesystem thing happening in the background when I rebooted the machine. I rebooted via the GUI and there was no sign that something was happening in the background, so this was really a normal thing that a user would do. No amount of fixing was able to get it back, but I was able to boot from the installation media, mount it read-only, and copy the data elsewhere. 2) Virtually all of the Linux servers at work will randomly hang for between minutes and hours. This was eventually traced to a BTRFS-scrub process that the OS vendor schedules to run weekly. The length and impact of the hang seemed to be based on how much write activity happens - servers where all the heavy activity happens on NFS mounts saw no impact, but servers that write a lot of logs to the local filesystem would get severely crippled on a weekly basis. We've moved a bunch of our write-heavy filesystems to non-BTRFS options as a result of this. 3) This is a more minor issue, but still speaks to my experience. I had a VM that was basically a remote desktop for me to use. Generally speaking it would hang hard after a few days of uptime with no actual usage. When I reinstalled it on a non-BTRFS (sorry, can't remember which filesystem I used) filesystem it was rock solid. I have no proof that this had anything to do with BTRFS. All of these were things that happened around a year ago, they may not be a true representation of the current state of BTRFS. But they've burned me badly, so now any use of BTRFS will be evaluated very carefully. In contrast, I've been running ZFS on a couple of FreeBSD servers, with fairly write-heavy loads, and have had no issues that were filesystem-related. Even under severe space and memory constraints ZFS has been rock solid.