5 ms·

> I've used btrfs in the past, and between my negative experiences and the repeated anecdotes posted all over the internet to this day I'm hesitant to give it a

> I've used btrfs in the past, and between my negative experiences and the repeated anecdotes posted all over the internet to this day I'm hesitant to give it another go. What are those issues? I've not followed the conversation, but I've been running btrfs for over a decade on multiple machines with different usage patterns without any issues. I'm genuinely interested. Are those substantiated claims? Or just the general noisy rants you can find about essentially anything people have opinions about?

Filligree 3y ago
In my own case: Btrfs performance got absolutely terrible once I was up to a few thousand snapshots. Never had it lose data, but I also never ran it long enough to lose a disk.
- unqueued 3y ago
  This is an example of how btrfs is poorly handled by many userland tools by default. You should never ever need to have thousands of snapshots. I use btrbk to manage my snapshots. It keeps my snapshots pruned to 300, pruning intermediate snapshots, so I can still easily step back 52 weeks, or I can purge them if I want to improve performance and reclaim space. While at the same time, every single snapshot is sent to two compressed encrypted backup images over ssh. The diff algorithm is pretty good, but for some types of files that fragment frequently, it is best to put them on their own subvolumes. My places.sqlite files from my browser testing profiles were adding tens or hundreds of MiB to snapshots, so they went in a seperate subvolume. I don't think the zfs would perform any better. I don't blame people for having bad experiences with btrfs, it is very inconsistently documented. I think it is especially annoying how many userland tools keep every single snapshot as a subvolume of the root, so that you have tens of millions of read-only files stored in /.snapshots. Awful.
  - seanw444 3y ago
    My setup is that I only have subvolumes for root, and (for now) my Doom Emacs config. Whenever I do a system upgrade, or a Doom Emacs upgrade, I manually take a snapshot of the appropriate subvolume first, just in case anything goes wrong. And then I only keep a max of 7 or so snapshots before I go through and delete the old ones. Has worked well for me so far and saved my butt a few times, and since I don't have a bunch of subvolumes to manage, I'm not overwhelmed. It's nice and simple, and I need nothing other than the default `btrfs` CLI utility.
    - unqueued 3y ago
      That's what's great about btrfs. I started using it that way too. It was nice to be able to take instant snapshots of my fs. I just amended my existing backup scripts to act on temporary snapshots instead. Then, I started dividing things up into subvolumes. I suggest moving your filesystem root to its own subvolume (the convention is to call it @root), then optionally mounting the actual filesystem root as like /mnt/fsroot, and mounting it by passing subvol=/. That is where I keep my snapshots. You can still have /home be nested in @root. I have subvolumes for /var/cache, /var/lib/docker, etc. And more fragmented qemu images just go on a NoCOW subvolume with +C set, and I back up normally. I also highly suggest reflinks. Once you start using them you will use them all the time. I don't know if zfs supports reflinks yet, at the time it didn't. But combining reflinks with subvolumes solves a lot of problems. Since btrfs doesn't support recursive snapshotting, you can just do cp --reflink=always -rp @root /path/to/new-subvolume, and it will combine the contents of the subvolumes without consuming any more space. And of course, btrfs-send will (usually) not duplicate reflink data.
  - Filligree 3y ago
    I did keep snapshots pruned to ~300 per subvolume. But I also had around twenty subvolume I needed to snapshot in this manner.
yjftsjthsd-h 3y ago
N=1, though I suspect my case is going to sound a lot like the other complaints you might find. I thought btrfs looked great so I used it for the root and home filesystems on a laptop running OpenSUSE. (This is modestly significant, because SUSE specifically supports BTRFS and I figured it would give me a better experience.) Now the home filesystem never gave me any problems. The root filesystem, however, broke to the degree that I ended up reinstalling. Twice. I couldn't tell you the exact error anymore, unfortunately, but it looked like running out of space left the fs in an unrecoverable state.
- 5e92cb50239222b 3y ago
  Recovery from out of space errors is difficult on any copy-on-write filesystem since you need to write more data to disk to actually delete anything. You'll find similar stories about ZFS if you care to look for them. btrfs has had some protection against out-of-space issues for some time now (look for "global reserve" in `btrfs filesystem usage`). I think ZFS has something similar, but I don't care enough to check since I'm very careful to never let it get that far.
Volundr 3y ago
For my 2c, I haven't touch BTRFS since using it as the primary storage in a main storage server at work. We had a 6TB system reporting 3TB free (back when this was a lot of storage), yet every write operation (including deletes) was failing with an error indicating the disk was full. This was years ago, so I'm a little fuzzy on the details, but after some really painful downtime and troubleshooting, it turns out that BTRFS marks blocks as either metadata or data blocks. Once a block is marked, it won't be unmarked even if empty. So because this was a high churn system all of the blocks ended up marked as data and there was none available when more metadata was needed. The solution was running `balance` with some specific flags. My understanding is that this is partially resolved now (empty blocks are automatically freed now), but not completely (partially empty blocks are never compacted without a balance, so you can still have a bunch of half-empty data blocks and no room for metadata), so one is a lot less likely to run into similar issues, but it was scarring enough to me that I've never run a BTRFS system again.
bmurphy1976 3y ago
It's been a couple years since I last tried but I had file system corruption and some random nuisance issues that have long since slipped my mind. I never really got to the bottom of it, I had other things to do than fight my file system to get it working properly.

Filligree 3y ago

In my own case: Btrfs performance got absolutely terrible once I was up to a few thousand snapshots. Never had it lose data, but I also never ran it long enough to lose a disk.

unqueued 3y ago
This is an example of how btrfs is poorly handled by many userland tools by default. You should never ever need to have thousands of snapshots. I use btrbk to manage my snapshots. It keeps my snapshots pruned to 300, pruning intermediate snapshots, so I can still easily step back 52 weeks, or I can purge them if I want to improve performance and reclaim space. While at the same time, every single snapshot is sent to two compressed encrypted backup images over ssh. The diff algorithm is pretty good, but for some types of files that fragment frequently, it is best to put them on their own subvolumes. My places.sqlite files from my browser testing profiles were adding tens or hundreds of MiB to snapshots, so they went in a seperate subvolume. I don't think the zfs would perform any better. I don't blame people for having bad experiences with btrfs, it is very inconsistently documented. I think it is especially annoying how many userland tools keep every single snapshot as a subvolume of the root, so that you have tens of millions of read-only files stored in /.snapshots. Awful.
- seanw444 3y ago
  My setup is that I only have subvolumes for root, and (for now) my Doom Emacs config. Whenever I do a system upgrade, or a Doom Emacs upgrade, I manually take a snapshot of the appropriate subvolume first, just in case anything goes wrong. And then I only keep a max of 7 or so snapshots before I go through and delete the old ones. Has worked well for me so far and saved my butt a few times, and since I don't have a bunch of subvolumes to manage, I'm not overwhelmed. It's nice and simple, and I need nothing other than the default `btrfs` CLI utility.
  - unqueued 3y ago
    That's what's great about btrfs. I started using it that way too. It was nice to be able to take instant snapshots of my fs. I just amended my existing backup scripts to act on temporary snapshots instead. Then, I started dividing things up into subvolumes. I suggest moving your filesystem root to its own subvolume (the convention is to call it @root), then optionally mounting the actual filesystem root as like /mnt/fsroot, and mounting it by passing subvol=/. That is where I keep my snapshots. You can still have /home be nested in @root. I have subvolumes for /var/cache, /var/lib/docker, etc. And more fragmented qemu images just go on a NoCOW subvolume with +C set, and I back up normally. I also highly suggest reflinks. Once you start using them you will use them all the time. I don't know if zfs supports reflinks yet, at the time it didn't. But combining reflinks with subvolumes solves a lot of problems. Since btrfs doesn't support recursive snapshotting, you can just do cp --reflink=always -rp @root /path/to/new-subvolume, and it will combine the contents of the subvolumes without consuming any more space. And of course, btrfs-send will (usually) not duplicate reflink data.
- Filligree 3y ago
  I did keep snapshots pruned to ~300 per subvolume. But I also had around twenty subvolume I needed to snapshot in this manner.

yjftsjthsd-h 3y ago

N=1, though I suspect my case is going to sound a lot like the other complaints you might find. I thought btrfs looked great so I used it for the root and home filesystems on a laptop running OpenSUSE. (This is modestly significant, because SUSE specifically supports BTRFS and I figured it would give me a better experience.) Now the home filesystem never gave me any problems. The root filesystem, however, broke to the degree that I ended up reinstalling. Twice. I couldn't tell you the exact error anymore, unfortunately, but it looked like running out of space left the fs in an unrecoverable state.

5e92cb50239222b 3y ago
Recovery from out of space errors is difficult on any copy-on-write filesystem since you need to write more data to disk to actually delete anything. You'll find similar stories about ZFS if you care to look for them. btrfs has had some protection against out-of-space issues for some time now (look for "global reserve" in `btrfs filesystem usage`). I think ZFS has something similar, but I don't care enough to check since I'm very careful to never let it get that far.

Volundr 3y ago

For my 2c, I haven't touch BTRFS since using it as the primary storage in a main storage server at work. We had a 6TB system reporting 3TB free (back when this was a lot of storage), yet every write operation (including deletes) was failing with an error indicating the disk was full. This was years ago, so I'm a little fuzzy on the details, but after some really painful downtime and troubleshooting, it turns out that BTRFS marks blocks as either metadata or data blocks. Once a block is marked, it won't be unmarked even if empty. So because this was a high churn system all of the blocks ended up marked as data and there was none available when more metadata was needed. The solution was running `balance` with some specific flags. My understanding is that this is partially resolved now (empty blocks are automatically freed now), but not completely (partially empty blocks are never compacted without a balance, so you can still have a bunch of half-empty data blocks and no room for metadata), so one is a lot less likely to run into similar issues, but it was scarring enough to me that I've never run a BTRFS system again.

bmurphy1976 3y ago

It's been a couple years since I last tried but I had file system corruption and some random nuisance issues that have long since slipped my mind. I never really got to the bottom of it, I had other things to do than fight my file system to get it working properly.