3 ms·

After an experience where btrfs decided to break my combined filesystem I vow to never use it again.

by iikoolpp 7y ago

pas 7y ago
Could you explain/describe what happened and why and how to prevent it?
- iikoolpp 7y ago
  Sure, I was trying to remove a disk from my array (1TB, with 50GB used, the other disk was 4TB and had ~600GB used). I tried doing `btrfs remove /dev/disk1 /mnt` and it refused, claiming there was no free space. No amount of arcane commands worked. Eventually I just copied all the files to somewhere else and nuked the filesystem entirely.
  - cmurf 7y ago
    Due to the way btrfs allocates in "collections" of extents called block groups, it's possible all the space was allocated by mostly empty block groups, which could make enospc possible. But that's a rather old set of bugs that I haven't seen in a very long time and predates all the modern space handling code. It must have been pre-4.0 kernels. And I did run into it myself on purpose many times while trying to help improve the behavior. Non-obvious, but very straight forward way around such a wedged in file system, is to add a 3rd device. It could even be a USB stick, back then I was using small 4GiB sticks and it would work. That was enough to allocate a couple metadata only block groups to the stick, to write out the file system changes necessary to back out the second device. And once that completes, a brief filtered balance (e.g. btrfs balance start -dusage=10 is usually sufficien) allows enough free space on the 1st device, to back out the 3rd (the USB stick). The non-obvious thing about any COW file system is that deletion always requires free space. There is no such thing as deleting a file with COW unless the fs can write that deletion change to all the affected trees into free space. Once the entire set of metadata changes is committed, then the data and metadata extents for those deleted files can be freed. Anyway, a lot has changed even in one year in Btrfs, let alone the past five years. It's thousands of line changes per kernel release.

After an experience where btrfs decided to break my combined filesystem I vow to never use it again.

pas 7y ago

Could you explain/describe what happened and why and how to prevent it?

iikoolpp 7y ago
Sure, I was trying to remove a disk from my array (1TB, with 50GB used, the other disk was 4TB and had ~600GB used). I tried doing `btrfs remove /dev/disk1 /mnt` and it refused, claiming there was no free space. No amount of arcane commands worked. Eventually I just copied all the files to somewhere else and nuked the filesystem entirely.
- cmurf 7y ago
  Due to the way btrfs allocates in "collections" of extents called block groups, it's possible all the space was allocated by mostly empty block groups, which could make enospc possible. But that's a rather old set of bugs that I haven't seen in a very long time and predates all the modern space handling code. It must have been pre-4.0 kernels. And I did run into it myself on purpose many times while trying to help improve the behavior. Non-obvious, but very straight forward way around such a wedged in file system, is to add a 3rd device. It could even be a USB stick, back then I was using small 4GiB sticks and it would work. That was enough to allocate a couple metadata only block groups to the stick, to write out the file system changes necessary to back out the second device. And once that completes, a brief filtered balance (e.g. btrfs balance start -dusage=10 is usually sufficien) allows enough free space on the 1st device, to back out the 3rd (the USB stick). The non-obvious thing about any COW file system is that deletion always requires free space. There is no such thing as deleting a file with COW unless the fs can write that deletion change to all the affected trees into free space. Once the entire set of metadata changes is committed, then the data and metadata extents for those deleted files can be freed. Anyway, a lot has changed even in one year in Btrfs, let alone the past five years. It's thousands of line changes per kernel release.