4 ms·

I've been burned enough times to avoid it. My Linux laptop, running OpenSuse Tumbleweed, got its root fs into a state where it would hang when trying to mount

I've been burned enough times to avoid it. My Linux laptop, running OpenSuse Tumbleweed, got its root fs into a state where it would hang when trying to mount RW. I was able to boot off of a rescue image, mount it RO, and copy my data to another machine. None of the repair suggestions were able to get the fs into a mountable state, I had to reinstall it and copy my data back. Every VM at work that uses btrfs gets into a state where some maintenance process (rebalancing, I think - I'm not one of the Linux admins) will consume all CPU in the VM and prevent it from doing any work. The only thing that varies is how long it takes - it seems to be related to how much write activity there is on the fs, VMs with heavier write activity get into this state faster. This has led to outages and the Linux vendor was unable to resolve this issue. We've since removed all traces of btrfs from our data center.

cyphar 4y ago
If you used Docker on your machine, this was a bug in the remount code (they forgot to re-trigger the qgroup scan if it was interrupted when you did an rw-remount) which I fixed several years ago[1]. I don't know what that will do for your confidence in btrfs, but hopefully it'll give you some closure. [1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6c6b5a39c4bf3dbd8cf629c9f5450e983c19dbb9 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
- Mister_Snuggles 4y ago
  The incident with my Linux laptop was definitely not Docker-related, but the description of the issue sounds like it could still apply. We didn't use Docker at work, at least not on the VMs that I was affected by. This one was a gradual degradation over time - the maintenance process would run fine, but after a while it would start hanging the machine. It really felt like the fs was only good for some amount of writes before the maintenance process got overwhelmed and would hang the VM whenever it ran. I'm happy to see that things are getting better, but the reputational damage will take a long time to repair. All that being said, I still use btrfs on my Linux laptop, but I've also become more conscientious about backing it up.
  - cyphar 4y ago
    I was told a while ago that rebalancing could get into pathological cases fairly easily (on openSUSE we used to run a rebalance in a cron job and it turns out this would grind machines to a halt). I still use btrfs and haven't had this issue for several years so maybe the issue has been resolved? I'm not sure what the resolution might have been.

cyphar 4y ago

If you used Docker on your machine, this was a bug in the remount code (they forgot to re-trigger the qgroup scan if it was interrupted when you did an rw-remount) which I fixed several years ago[1]. I don't know what that will do for your confidence in btrfs, but hopefully it'll give you some closure. [1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6c6b5a39c4bf3dbd8cf629c9f5450e983c19dbb9 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

Mister_Snuggles 4y ago
The incident with my Linux laptop was definitely not Docker-related, but the description of the issue sounds like it could still apply. We didn't use Docker at work, at least not on the VMs that I was affected by. This one was a gradual degradation over time - the maintenance process would run fine, but after a while it would start hanging the machine. It really felt like the fs was only good for some amount of writes before the maintenance process got overwhelmed and would hang the VM whenever it ran. I'm happy to see that things are getting better, but the reputational damage will take a long time to repair. All that being said, I still use btrfs on my Linux laptop, but I've also become more conscientious about backing it up.
- cyphar 4y ago
  I was told a while ago that rebalancing could get into pathological cases fairly easily (on openSUSE we used to run a rebalance in a cron job and it turns out this would grind machines to a halt). I still use btrfs and haven't had this issue for several years so maybe the issue has been resolved? I'm not sure what the resolution might have been.