6 ms·

File Systems – ZFS vs. XFS

50 points by truth_seeker 7y ago

hnlmorg 7y ago
On the topic of ZFS on Linux, does anyone know the expected timeline for version 8 of ZoL? Or is it just a case of "it will be ready when it's ready"? I've had to import an pool from FreeBSD but can only mount the volume as readonly because of a feature flag enabled that isn't in v7 but is currently in the RC versions of 8. I'm a little nervous about running the RC versions because this is a "production" pool (albeit just a home server).
mikece 7y ago
"For the sake of clarity, when we say ZFS we mean OpenZFS and not the commercial version supported by Oracle Inc." Just curious: what are the differences between [Oracle] ZFS and OpenZFS?
- georgyo 7y ago
  At this point, the list is very long. Development forked many years ago when Oracle closed the source. Both sides have been in very active development since.
buster 7y ago
I was expecting some more technical proof, benchmarks or similar. Sadly, this post reads as a pure advertisement piece to use OpenZFS. What should i take away in this sentence? "XFS is optimized for huge files and for parallel I/O this makes it the easier choice for use cases like NASA Advanced Supercomputing Division." The whole post sounds like "ZFS is the superior technology with much more bells and whistles and XFS is old tech but XFS is what you choose if you're the NASA..". Having interest in XFS/ZFS/BTRFS and what to choose on my machines, i was hoping for some more substantial and objective facts.
- ahje 7y ago
  Yeah, I expected something more substantial. While my personal experience confirms that XFS is more suitable for I/O intensive stuff, it's still nothing more than anectdotal evidence, and it's not indicative for other people's use cases.
- truth_seeker 7y ago
  It might not what exactly you are looking for, but i think it's close. MySQL performance bench suite against ZFS vs XFS https://www.percona.com/blog/2018/05/15/about-zfs-performance/ https://www.percona.com/blog/2018/05/15/about-zfs-performanc...
DiabloD3 7y ago
This article is mildly ironic to me: I use XFS for my boot file system, a small 16GB in an mdraid mirror, to boot my ZFS server. They are both useful, and the ultimate solution, for different cases. BTRFS has no future, and ext4 is purely a legacy solution (also ironic, given XFS is older than ext4).
- peatfreak 7y ago
  > BTRFS has no future This is FUD. Btrfs is actively developed and its problems and limitations are acknowledged by upstream in an honest way. Many people DO use it with no more problems than they have with XFS or OpenZFS. The only people who say "Btrfs is dead" are those who don't use it. It's a bit like people who snigger at the whole "Perl looks like line noise" trope without ever having writting a line of Perl in their lives. In every report that I have seen of Btrfs doing something bad is when somebody (myself included) has done something incorrectly, not read the docs, not consulted an expert before doing something complicated, not taken the time to understand how it works, using a suboptimal or flakey hardware/software stack underlying it. Have you seen how strict the FreeNAS community is about using proper, server-grade hardware and configuring the system properly to prevent data loss? This is a very good line to stick to. Unfortunately Btrfs doesn't have the same consistent message but if it did then there would be fewer problems. I regularly use Btrfs at home and so far I haven't had any problems. I would not use it in production when other people's data is on it YET. One reason is technical, and it's too much of a cognitive burden these days when combined with other people's data. The other reason is because of customer expectations, i.e., because of FUD that people like you are spreading. If there was some data loss, the customer would be angry at me for choosing Btrfs because "everybody knows" that it's unstable /s. If I am using ZFS then there's less of a chance that customers think I make bad decisions and there is a small chance they will be more forgiving. Mdadm + LVM + XFS is an excellent alternative. The only reason that I don't use this more often is because snapshotting requires twice as much disk space as you plan to use. Otherwise I probably would be using this stack exclusively. Getting back to the original "Btrfs has no future" nonsense, the only reason this is true is if people make it true. Therefore, one important reason I use Btrfs is that I want to see it succeed. That can only happen when people actually use it and advocate for it in a positive and honest way.
fsiefken 7y ago
I/O on ZFS might be faster if you turn on compression at the expense of memory and cpu. XFS has no such feature, BTRFS has though.
noodlesUK 7y ago
I don’t really know that this is a particularly appropriate comparison. XFS is a fine filesystem, but ZFS is more than a filesystem, it’s RAID, LVM, deduplication, etc. The real competitors to ZFS from my understanding are largely BTRFS and proprietary technologies such as apple’s new APFS. I’d very much like to see a comparison between those. I’ve run ZFS on my systems for years, but moved away from it towards btrfs on my laptop, as it was just too much pain to keep the kernel modules up to date. I wish the CDDL were compatible with the GPL, cause if the drivers were in-tree there wouldn’t be any argument at all.
- josteink 7y ago
  With Ubuntu you get binary modules updated together with the kernel. I moved the opposite direction (btrfs->zfs) because I had issues with my main btrfs rootfs constantly degrading to read-only mode, causing my system to crash and require rebooting. With ZFS I’ve had no such issues. ZFS uses noticeably much more RAM though.
  - noodlesUK 7y ago
    Yeah, I was on Gentoo at the time, which didn’t make it particularly difficult (when everything is source code it doesn’t really matter), but I’m on Fedora now, where the story is a bit different. Ubuntu shipping the binary modules is good. Do they support booting from ZFS simply (that is, are the kernel modules in the initramfs)?
    - josteink 7y ago
      Yeah. All my systems are pure ZFS systems. The initial install is a bit involved (live-cd, manual partitioning + debootstrap) but everything just works (tm) after that, including booting into specific snapshots from grub. There’s a good guide on github: https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-ZFS https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-...
- vbezhenar 7y ago
  If you need RAID and LVM, just use them? Separation of concepts is not a bad thing.
tinus_hn 7y ago
Wow, I thought XFS was dead but apparently it is still supported.
- amelius 7y ago
  I'm glad there is XFS because it supports more than 64k hardlinks (which I use for backups and for structural sharing of immutable data). This is where ext4 fails for me.
  - sterwill 7y ago
    That's the original reason I started using XFS on Linux about 15 years ago. I needed all those hardlinks for dirvish. I haven't had a need for dirvish since then, but I still use XFS for largeish volumes with lots of files (won't run out of inodes). LVM does a good job at volume management.
- okl 7y ago
  I am using and have used xfs on many systems, from server to embedded. Xfs is reliable and performs well. Debian support is fine. I wouldn't want to switch to any other file system.
- gdfasfklshg4 7y ago
  It is the FS for Red Hat 7 and CentOS 7.
  - Tepix 7y ago
    Also on RHEL 8. XFS gains the XFS Copy-On-Write Data Extents feature in RHEL 8 which can speed things up and save space.
- cyphar 7y ago
  XFS is supported by effectively every distribution and has very significant development each release (even the test-suite for all filesystems is called xfstests because it used to be only part of XFS). And despite being older than ext (and older than Linux itself), XFS has lots of features which ext doesn't have -- such as reflinks and CoW (though XFS also doesn't have ext4-style encryption).
gdfasfklshg4 7y ago
Should a fair comparison not include the features of LVM etc that Red Hat etc are layering together with XFS?
psankar 7y ago
> One last point of architectural difference is this — XFS has B+ tree implementation for searching files and allocating space. This makes searching and fetching the data much faster. OpenZFS has no such feature. I wish I could downvote this article, just for this one point alone. I wonder what kind of benchmarking the author did to come to the conclusion of "searching and fetching the data much faster" and how much the B+ tree of xfs contributed to it. Speed depends on a lot of factors (copy on write, compression, journaling, crash recovery, etc.) and I wonder how the author narrowed it down to the B+ tree of XFS. Also, there could be variants of various B family trees even within a same filesystem. So I am not really sure, what the author means by "XFS has B+ tree implementation". If you are a RedHat shop and deploy linux extensively already, then xfs may be a better option, if you are not looking for most zfs features like volume management, etc. Also, installation, support etc. would be much easier. This article is shallow in many ways technically, but arrives at conclusions strongly. It just seems like a paid article to promote zfs. May be an article on LWN would be more meaningful and technically competent to make a comparison than this.
- lallysingh 7y ago
  I'd like to see some RAM usage comparisons.
- theamk 7y ago
  It looks to me like this was an attempt to make XFS look better? The author probably looks at the things from "the reliability is #1, and the rest is secondary" standpoint, so they are having a hard time coming with pro-XFS arguments.
deleted 7y ago
[deleted]
nineteen999 7y ago
> XFS is largely supported on Linux along with IRIX Use of the present tense regarding IRIX here made me chuckle. The last version of IRIX is 6.5.30, released in August 2006.
- lallysingh 7y ago
  And by that point, there wasn't much going on in Irix development. 6.5 was out in the late 1990s, IIRC.
lasermike026 7y ago
Doesn't ZFS consume more resources?
ofrzeta 7y ago
I don't know about ZFS but you can't shrink XFS filesystems (opposed to Ext3/4). It may not be the most common use case but I tend to forget about it, used XFS as the fs and then got into problems because of that non-feature. http://xfs.org/index.php/Shrinking_Support http://xfs.org/index.php/Shrinking_Support

50 points by truth_seeker 7y ago

On the topic of ZFS on Linux, does anyone know the expected timeline for version 8 of ZoL? Or is it just a case of "it will be ready when it's ready"? I've had to import an pool from FreeBSD but can only mount the volume as readonly because of a feature flag enabled that isn't in v7 but is currently in the RC versions of 8. I'm a little nervous about running the RC versions because this is a "production" pool (albeit just a home server).

mikece 7y ago

"For the sake of clarity, when we say ZFS we mean OpenZFS and not the commercial version supported by Oracle Inc." Just curious: what are the differences between [Oracle] ZFS and OpenZFS?

georgyo 7y ago
At this point, the list is very long. Development forked many years ago when Oracle closed the source. Both sides have been in very active development since.

buster 7y ago

I was expecting some more technical proof, benchmarks or similar. Sadly, this post reads as a pure advertisement piece to use OpenZFS. What should i take away in this sentence? "XFS is optimized for huge files and for parallel I/O this makes it the easier choice for use cases like NASA Advanced Supercomputing Division." The whole post sounds like "ZFS is the superior technology with much more bells and whistles and XFS is old tech but XFS is what you choose if you're the NASA..". Having interest in XFS/ZFS/BTRFS and what to choose on my machines, i was hoping for some more substantial and objective facts.

ahje 7y ago
Yeah, I expected something more substantial. While my personal experience confirms that XFS is more suitable for I/O intensive stuff, it's still nothing more than anectdotal evidence, and it's not indicative for other people's use cases.
truth_seeker 7y ago
It might not what exactly you are looking for, but i think it's close. MySQL performance bench suite against ZFS vs XFS https://www.percona.com/blog/2018/05/15/about-zfs-performance/ https://www.percona.com/blog/2018/05/15/about-zfs-performanc...

DiabloD3 7y ago

This article is mildly ironic to me: I use XFS for my boot file system, a small 16GB in an mdraid mirror, to boot my ZFS server. They are both useful, and the ultimate solution, for different cases. BTRFS has no future, and ext4 is purely a legacy solution (also ironic, given XFS is older than ext4).

peatfreak 7y ago
> BTRFS has no future This is FUD. Btrfs is actively developed and its problems and limitations are acknowledged by upstream in an honest way. Many people DO use it with no more problems than they have with XFS or OpenZFS. The only people who say "Btrfs is dead" are those who don't use it. It's a bit like people who snigger at the whole "Perl looks like line noise" trope without ever having writting a line of Perl in their lives. In every report that I have seen of Btrfs doing something bad is when somebody (myself included) has done something incorrectly, not read the docs, not consulted an expert before doing something complicated, not taken the time to understand how it works, using a suboptimal or flakey hardware/software stack underlying it. Have you seen how strict the FreeNAS community is about using proper, server-grade hardware and configuring the system properly to prevent data loss? This is a very good line to stick to. Unfortunately Btrfs doesn't have the same consistent message but if it did then there would be fewer problems. I regularly use Btrfs at home and so far I haven't had any problems. I would not use it in production when other people's data is on it YET. One reason is technical, and it's too much of a cognitive burden these days when combined with other people's data. The other reason is because of customer expectations, i.e., because of FUD that people like you are spreading. If there was some data loss, the customer would be angry at me for choosing Btrfs because "everybody knows" that it's unstable /s. If I am using ZFS then there's less of a chance that customers think I make bad decisions and there is a small chance they will be more forgiving. Mdadm + LVM + XFS is an excellent alternative. The only reason that I don't use this more often is because snapshotting requires twice as much disk space as you plan to use. Otherwise I probably would be using this stack exclusively. Getting back to the original "Btrfs has no future" nonsense, the only reason this is true is if people make it true. Therefore, one important reason I use Btrfs is that I want to see it succeed. That can only happen when people actually use it and advocate for it in a positive and honest way.

fsiefken 7y ago

I/O on ZFS might be faster if you turn on compression at the expense of memory and cpu. XFS has no such feature, BTRFS has though.

noodlesUK 7y ago

I don’t really know that this is a particularly appropriate comparison. XFS is a fine filesystem, but ZFS is more than a filesystem, it’s RAID, LVM, deduplication, etc. The real competitors to ZFS from my understanding are largely BTRFS and proprietary technologies such as apple’s new APFS. I’d very much like to see a comparison between those. I’ve run ZFS on my systems for years, but moved away from it towards btrfs on my laptop, as it was just too much pain to keep the kernel modules up to date. I wish the CDDL were compatible with the GPL, cause if the drivers were in-tree there wouldn’t be any argument at all.

josteink 7y ago
With Ubuntu you get binary modules updated together with the kernel. I moved the opposite direction (btrfs->zfs) because I had issues with my main btrfs rootfs constantly degrading to read-only mode, causing my system to crash and require rebooting. With ZFS I’ve had no such issues. ZFS uses noticeably much more RAM though.
- noodlesUK 7y ago
  Yeah, I was on Gentoo at the time, which didn’t make it particularly difficult (when everything is source code it doesn’t really matter), but I’m on Fedora now, where the story is a bit different. Ubuntu shipping the binary modules is good. Do they support booting from ZFS simply (that is, are the kernel modules in the initramfs)?
  - josteink 7y ago
    Yeah. All my systems are pure ZFS systems. The initial install is a bit involved (live-cd, manual partitioning + debootstrap) but everything just works (tm) after that, including booting into specific snapshots from grub. There’s a good guide on github: https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-ZFS https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-...
vbezhenar 7y ago
If you need RAID and LVM, just use them? Separation of concepts is not a bad thing.

tinus_hn 7y ago

Wow, I thought XFS was dead but apparently it is still supported.

amelius 7y ago
I'm glad there is XFS because it supports more than 64k hardlinks (which I use for backups and for structural sharing of immutable data). This is where ext4 fails for me.
- sterwill 7y ago
  That's the original reason I started using XFS on Linux about 15 years ago. I needed all those hardlinks for dirvish. I haven't had a need for dirvish since then, but I still use XFS for largeish volumes with lots of files (won't run out of inodes). LVM does a good job at volume management.
okl 7y ago
I am using and have used xfs on many systems, from server to embedded. Xfs is reliable and performs well. Debian support is fine. I wouldn't want to switch to any other file system.
gdfasfklshg4 7y ago
It is the FS for Red Hat 7 and CentOS 7.
- Tepix 7y ago
  Also on RHEL 8. XFS gains the XFS Copy-On-Write Data Extents feature in RHEL 8 which can speed things up and save space.
cyphar 7y ago
XFS is supported by effectively every distribution and has very significant development each release (even the test-suite for all filesystems is called xfstests because it used to be only part of XFS). And despite being older than ext (and older than Linux itself), XFS has lots of features which ext doesn't have -- such as reflinks and CoW (though XFS also doesn't have ext4-style encryption).

gdfasfklshg4 7y ago

Should a fair comparison not include the features of LVM etc that Red Hat etc are layering together with XFS?

psankar 7y ago

> One last point of architectural difference is this — XFS has B+ tree implementation for searching files and allocating space. This makes searching and fetching the data much faster. OpenZFS has no such feature. I wish I could downvote this article, just for this one point alone. I wonder what kind of benchmarking the author did to come to the conclusion of "searching and fetching the data much faster" and how much the B+ tree of xfs contributed to it. Speed depends on a lot of factors (copy on write, compression, journaling, crash recovery, etc.) and I wonder how the author narrowed it down to the B+ tree of XFS. Also, there could be variants of various B family trees even within a same filesystem. So I am not really sure, what the author means by "XFS has B+ tree implementation". If you are a RedHat shop and deploy linux extensively already, then xfs may be a better option, if you are not looking for most zfs features like volume management, etc. Also, installation, support etc. would be much easier. This article is shallow in many ways technically, but arrives at conclusions strongly. It just seems like a paid article to promote zfs. May be an article on LWN would be more meaningful and technically competent to make a comparison than this.

lallysingh 7y ago
I'd like to see some RAM usage comparisons.
theamk 7y ago
It looks to me like this was an attempt to make XFS look better? The author probably looks at the things from "the reliability is #1, and the rest is secondary" standpoint, so they are having a hard time coming with pro-XFS arguments.

deleted 7y ago

[deleted]

nineteen999 7y ago

> XFS is largely supported on Linux along with IRIX Use of the present tense regarding IRIX here made me chuckle. The last version of IRIX is 6.5.30, released in August 2006.

lallysingh 7y ago
And by that point, there wasn't much going on in Irix development. 6.5 was out in the late 1990s, IIRC.

lasermike026 7y ago

Doesn't ZFS consume more resources?

ofrzeta 7y ago

I don't know about ZFS but you can't shrink XFS filesystems (opposed to Ext3/4). It may not be the most common use case but I tend to forget about it, used XFS as the fs and then got into problems because of that non-feature. http://xfs.org/index.php/Shrinking_Support http://xfs.org/index.php/Shrinking_Support