3 ms·

I've been burned by Btrfs enough times to avoid it. I had a VM running OpenSuse Tumbleweed - I know, rolling release, don't expect stability, etc - and it woul

I've been burned by Btrfs enough times to avoid it. I had a VM running OpenSuse Tumbleweed - I know, rolling release, don't expect stability, etc - and it would consistently freeze after being up and unused for a few days. Rebuilt the VM but used XFS instead of Btrfs, it was rock solid. This happened within the last four months. At work, we have a number of machines using Btrfs. At some point they get into a state where the weekly Btrfs scrub process hangs the machine for hours at a time. The solution in every case was to move that filesystem onto a non-Btrfs volume. This is an ongoing issue, but I'm not in the team responsible so I don't know what they're doing with this. Anecdotally, this seems to be a problem when there is a lot of writing going on with the volumes. Machines that use Btrfs root but do most of their work on NFS volumes don't seem to experience this issue. I also have a laptop running OpenSuse Tumbleweed with Btrfs. One day it somehow got itself into a state where it would hang when mounting the filesystem read/write, but it would be fine if it was mounted read-only. I can't recall exactly what happened to cause it, but I'm pretty sure I didn't do anything exceptional with the computer - I think I rebooted it from the GUI while it was doing a scrub in the background, but that's just a guess. I think this happened about a year ago. This is all disappointing because some of the things Btrfs enables are awesome. In particular, OpenSuse's snapper has saved me countless hour of work. I haven't played with the send/receive stuff, but it seems like a great solution for backups. In the meantime, I'm using ZFS on a variety of FreeBSD boxes with a variety of workloads (including one running more VMs than its RAM should support) and it's been rock solid. At this point it will be a long time before I give Btrfs another look. It has the potential to be really good, and I want it to be good, but I don't believe it's there yet.

cookiecaper 8y ago
Yes, this is similar to my fairly recent experience with btrfs. While it's competitive on paper and there are several meta-arguments in its favor (being in mainline, for example), the cold hard facts just don't stack up. btrfs fares far worse and requires much more tuning than a generic ZFS installation. A great example is that if you ever try to run a database or a large virtual machine on a btrfs mount, you'll quickly become acquainted with chattr +C for "nocow". Any write-intensive applications grinds the filesystem to a halt until you disable the copy-on-write functionality for that file specifically, making it so snapshots, etc., are ineffective and the FS stops tracking the changes that would make them possible. Whereas on ZFS, like any application that uses fixed block sizes, virtualization and databases _may_ require some alignment tuning to avoid a read-modify-write cycle, and ZFS works without fuss from that moment on. I know that Facebook is investing heavily in btrfs right now, but to be frank, I don't see the point. btrfs doesn't have its youth as an excuse anymore and the core issues that exist seem like they're pretty fundamental. It's a gargantuan waste of effort when people could just take the tack that Ubuntu has taken and decide all that FUD around the CDDL, kicked up by fears that the commercial Unices were plotting to take their revenge on Linux, that maybe all that isn't super relevant anymore, and they can just start shipping ZoL.
- hiyer 8y ago
  Facebook uses btrfs extensively [1]. I was tempted to try it out again (burnt my fingers on it a couple of years back) but from the experiences mentioned here it looks like it's not really production-ready yet. 1. https://code.fb.com/open-source/linux/ https://code.fb.com/open-source/linux/
jchw 8y ago
I gave up on btrfs a long time ago. People were considering it 'production ready' when I experienced an issue that rendered a volume unusable, in the form of kernel panics on boot. This rubbed me the wrong way and I just moved back to ZFS-on-Linux which I've never had as much as a single issue with - not even once! Today I'm running ext4 because maintaining ZFS setups is exhausting since it is not upstreamed. Still... I hope there's a future in one of these volume-manager-meets-filesystem filesystems. Maybe bcachefs will end up being the answer some day?

cookiecaper 8y ago

Yes, this is similar to my fairly recent experience with btrfs. While it's competitive on paper and there are several meta-arguments in its favor (being in mainline, for example), the cold hard facts just don't stack up. btrfs fares far worse and requires much more tuning than a generic ZFS installation. A great example is that if you ever try to run a database or a large virtual machine on a btrfs mount, you'll quickly become acquainted with chattr +C for "nocow". Any write-intensive applications grinds the filesystem to a halt until you disable the copy-on-write functionality for that file specifically, making it so snapshots, etc., are ineffective and the FS stops tracking the changes that would make them possible. Whereas on ZFS, like any application that uses fixed block sizes, virtualization and databases _may_ require some alignment tuning to avoid a read-modify-write cycle, and ZFS works without fuss from that moment on. I know that Facebook is investing heavily in btrfs right now, but to be frank, I don't see the point. btrfs doesn't have its youth as an excuse anymore and the core issues that exist seem like they're pretty fundamental. It's a gargantuan waste of effort when people could just take the tack that Ubuntu has taken and decide all that FUD around the CDDL, kicked up by fears that the commercial Unices were plotting to take their revenge on Linux, that maybe all that isn't super relevant anymore, and they can just start shipping ZoL.

hiyer 8y ago
Facebook uses btrfs extensively [1]. I was tempted to try it out again (burnt my fingers on it a couple of years back) but from the experiences mentioned here it looks like it's not really production-ready yet. 1. https://code.fb.com/open-source/linux/ https://code.fb.com/open-source/linux/

jchw 8y ago

I gave up on btrfs a long time ago. People were considering it 'production ready' when I experienced an issue that rendered a volume unusable, in the form of kernel panics on boot. This rubbed me the wrong way and I just moved back to ZFS-on-Linux which I've never had as much as a single issue with - not even once! Today I'm running ext4 because maintaining ZFS setups is exhausting since it is not upstreamed. Still... I hope there's a future in one of these volume-manager-meets-filesystem filesystems. Maybe bcachefs will end up being the answer some day?