8 ms·

A data-point of one, and I only have a few TB spread over a handful of disks, but I lost an entire 1TB btrfs volume recently because of a single disk sector gon

A data-point of one, and I only have a few TB spread over a handful of disks, but I lost an entire 1TB btrfs volume recently because of a single disk sector gone bad (of course, I have backups, but it's a pain to restore 1TB, not to mention racking of the nerves (will the backup drive fail with sudden heavy use?) It's the only FS I've ever been unable to recover data from, at all, (I've had quite a few disks with errors before going back 25 years), and so it's no longer on my list of filesystems-I-trust.

andrewstuart2 9y ago
I've experienced data loss twice in 15 years of using Linux and trying different file systems. Both were on btrfs. I didn't use any advanced features, though I had plans to before this happened. But at that point, I cut my losses and converted all my disks back to ext4. The features are great, but data availability/integrity is the fundamental thing I want from a filesystem. If I write some bytes, they should be there tomorrow. Everything else is secondary.
- cyphar 9y ago
  If you're interested in a more featureful filesystem that isn't btrfs and has a very solid track record, I would recommend looking at XFS. It's a very old filesystem but has a lot of quite modern features (and it performs better than ext4 for several workloads).
  - kogepathic 9y ago
    > I would recommend looking at XFS. It's a very old filesystem but has a lot of quite modern features (and it performs better than ext4 for several workloads). I would second this. XFS is great. A few years ago I moved from JFS to XFS for bulk data storage because JFS is pretty abandoned these days. No issues at all with XFS, and never lost any data. Just make sure it's right for you before you use it (e.g. you cannot shrink XFS volumes as you can ext4)
  - arjie 9y ago
    Funnily enough, XFS is the only file system on Linux where I've lost data. Back in the day, the wisdom was to stay away from it if you were in an iffy power situation because it would serve zeroes if there was a write to a file near a power loss (I.e. you wouldn't get old file or new file but something else) Having had that happen to me I always used some extN and didn't lose any existing files. Of course that's a decade or more ago and I may be misremembering but a cursory Google looks like other people encountered something like it too.
    - crest 9y ago
      Afaik the problem is that XFS trusts the hardware to do the right thing during a power loss aka stop inflight requests before the brownout turns you disks and their controllers into heavily biased random number generators. Lots of x86(-64) hardware lacks a proper power loss interrupt triggered early enough to stop the all I/O in time. The ext3 journaling hides that problem to some extend by journaling more than just the minimal metadata required.
      - msla 9y ago
        Then XFS isn't suitable for consumer-grade hardware.
    - linsomniac 9y ago
      For maybe 3-6 months a decade ago I was running XFS on my laptop. My laptop had some sort of flakiness, ISTR it was graphics related, and it would require frequent reboots for a while. I remember one particular instance where after a power cycle a file I was working on an hour prior to it was trashed. That was a real "Oh FFS!" moment and I stopped using XFS. But, I will say I've had a lot of success using XFS to serve images, particularly tiles. Particularly map tiles that are generated once and then basically static. There it has a lot less overhead when formatted with size=2048, so lots of small files are handled better. Of course, reiserfs was even better, but part of that was with a job I worked on where they just blindly rendered map tiles for the whole earth, and so there were a lot of 50 byte solid blue tiles. Reiser did really well with that, though some deduplication or linking would have been a huge benefit.
  - yellowapple 9y ago
    XFS has been my FS of choice for a few years now. If I remember right, openSUSE even defaults to it for /home.
  - ArneBab 9y ago
    XFS is the system where I lost all data when we had a powerloss.
    - btmaybe 9y ago
      XFS has the same problem as btrfs: extreme sensitivity to metadata corruption. I'm not sure XFS has the RAID1-for-metadata (DUP) feature like btrfs has recently added.
  - pnutjam 9y ago
    I believe Suse uses XFS for larger data volumes. Their emphasis os on btrfs for the OS and system drives where it can help with administration tasks.
aljarry 9y ago
I've had similar experience around 5-6 years ago. One of the notebooks with ubuntu was force rebooted and btrfs got corrupted, no way to boot the system again.
vog 9y ago
Serious question: Why would anyone today use BTRFS, given that the battle-proof ZFS has finnally been ported to Linux for quite some time?
- josteink 9y ago
  > Why would anyone today use BTRFS, given that the battle-proof ZFS has finnally been ported to Linux for quite some time? It's only been available out of the box in a simple-to-use and supported fashion in 1 Distro for barely 1 year. You can hardly say it's been widely available, nor for a "quite some time".
  - zacmps 9y ago
    ZFS on Linux has official support for Arch, Debian, Fedora, Gentu, OpenSUSE, RHEL &CentOS and Ubuntu.
    - cyphar 9y ago
      It has "official support" from the ZoL folks. And yes, openSUSE has ZFS packages in OBS. But we sure as hell don't ship them by default, or in our official repos. The same applies for Arch, Debian, Fedora, Gentoo, RHEL and CentOS. Ubuntu is the only distribution that official supports ZoL, and actually ships it in it's official repositories (and by default). What that means is that Canonical is effectively saying "we trust there's no legal reason why we cannot do this." No other distribution has made that claim. EDIT: Actually NixOS also supports it, but the point stands.
      - Filligree 9y ago
        Well, also NixOS.
    - jononor 9y ago
      Supported by whom? Is it at the level one can expect from paid support from say Red Hat, SUSE or Canonical?
tqh 9y ago
Same here. I lost a disk to btrfs and did not manage to recover any data. No longer on my list of filesystems-I-trust. I guess Suse is the biggest contributor since the others stopped? It doesn't give me any confidence that it can be trusted.
RX14 9y ago
Which raid level were you using over those multiple disks? I've been using btrfs and never even experienced a hiccup despite continuous sector failures, continuous drive failures and even disk controllers disconnecting on the fly. And I've been running the whole thing on a single, fairly weak, server with second-hand disks attached over the cheapest USB3 sata adapters I've found. By all accounts I've been putting btrfs in a situation that is well outside the norm and I have 0 complaints. I also use it on my desktop (no raid) with similar experience (minus the shitty hardware). I am continuously amazed at the amount of people who have issues with btrfs. It's been absolutely rock solid over the time I've used it and I have 0 complaints (apart from the ext4 -> btrfs converter producing a corrupted btrfs filesystem, but the actual kernel btrfs code itself has been flawless).
- btmaybe 9y ago
  I too have lost an entire btrfs volume (single disk) because of a bad sector. It seems btrfs is very sensitive to bad sectors in metadata, compared to ext4. Since then, I've formatted my btrfs partition to do RAID1 on metadata and haven't had problems since. $ sudo btrfs fi df / Data, single System, DUP Metadata, DUP
  - Dylan16807 9y ago
    Isn't that the default?
    - sp332 9y ago
      "A single device filesystem will default [metadata] to DUP, unless a SSD is detected. Then it will default to single." https://btrfs.wiki.kernel.org/index.php/Manpage/mkfs.btrfs#OPTIONS https://btrfs.wiki.kernel.org/index.php/Manpage/mkfs.btrfs#O...
    - btmaybe 9y ago
      It's a relatively new feature (Jan-2016) and it's disabled by default for SSD devices (increased wear). Personally I prefer increased wear over unmountable volumes.

andrewstuart2 9y ago

I've experienced data loss twice in 15 years of using Linux and trying different file systems. Both were on btrfs. I didn't use any advanced features, though I had plans to before this happened. But at that point, I cut my losses and converted all my disks back to ext4. The features are great, but data availability/integrity is the fundamental thing I want from a filesystem. If I write some bytes, they should be there tomorrow. Everything else is secondary.

cyphar 9y ago
If you're interested in a more featureful filesystem that isn't btrfs and has a very solid track record, I would recommend looking at XFS. It's a very old filesystem but has a lot of quite modern features (and it performs better than ext4 for several workloads).
- kogepathic 9y ago
  > I would recommend looking at XFS. It's a very old filesystem but has a lot of quite modern features (and it performs better than ext4 for several workloads). I would second this. XFS is great. A few years ago I moved from JFS to XFS for bulk data storage because JFS is pretty abandoned these days. No issues at all with XFS, and never lost any data. Just make sure it's right for you before you use it (e.g. you cannot shrink XFS volumes as you can ext4)
- arjie 9y ago
  Funnily enough, XFS is the only file system on Linux where I've lost data. Back in the day, the wisdom was to stay away from it if you were in an iffy power situation because it would serve zeroes if there was a write to a file near a power loss (I.e. you wouldn't get old file or new file but something else) Having had that happen to me I always used some extN and didn't lose any existing files. Of course that's a decade or more ago and I may be misremembering but a cursory Google looks like other people encountered something like it too.
  - crest 9y ago
    Afaik the problem is that XFS trusts the hardware to do the right thing during a power loss aka stop inflight requests before the brownout turns you disks and their controllers into heavily biased random number generators. Lots of x86(-64) hardware lacks a proper power loss interrupt triggered early enough to stop the all I/O in time. The ext3 journaling hides that problem to some extend by journaling more than just the minimal metadata required.
    - msla 9y ago
      Then XFS isn't suitable for consumer-grade hardware.
  - linsomniac 9y ago
    For maybe 3-6 months a decade ago I was running XFS on my laptop. My laptop had some sort of flakiness, ISTR it was graphics related, and it would require frequent reboots for a while. I remember one particular instance where after a power cycle a file I was working on an hour prior to it was trashed. That was a real "Oh FFS!" moment and I stopped using XFS. But, I will say I've had a lot of success using XFS to serve images, particularly tiles. Particularly map tiles that are generated once and then basically static. There it has a lot less overhead when formatted with size=2048, so lots of small files are handled better. Of course, reiserfs was even better, but part of that was with a job I worked on where they just blindly rendered map tiles for the whole earth, and so there were a lot of 50 byte solid blue tiles. Reiser did really well with that, though some deduplication or linking would have been a huge benefit.
- yellowapple 9y ago
  XFS has been my FS of choice for a few years now. If I remember right, openSUSE even defaults to it for /home.
- ArneBab 9y ago
  XFS is the system where I lost all data when we had a powerloss.
  - btmaybe 9y ago
    XFS has the same problem as btrfs: extreme sensitivity to metadata corruption. I'm not sure XFS has the RAID1-for-metadata (DUP) feature like btrfs has recently added.
- pnutjam 9y ago
  I believe Suse uses XFS for larger data volumes. Their emphasis os on btrfs for the OS and system drives where it can help with administration tasks.

aljarry 9y ago

I've had similar experience around 5-6 years ago. One of the notebooks with ubuntu was force rebooted and btrfs got corrupted, no way to boot the system again.

vog 9y ago

Serious question: Why would anyone today use BTRFS, given that the battle-proof ZFS has finnally been ported to Linux for quite some time?

josteink 9y ago
> Why would anyone today use BTRFS, given that the battle-proof ZFS has finnally been ported to Linux for quite some time? It's only been available out of the box in a simple-to-use and supported fashion in 1 Distro for barely 1 year. You can hardly say it's been widely available, nor for a "quite some time".
- zacmps 9y ago
  ZFS on Linux has official support for Arch, Debian, Fedora, Gentu, OpenSUSE, RHEL &CentOS and Ubuntu.
  - cyphar 9y ago
    It has "official support" from the ZoL folks. And yes, openSUSE has ZFS packages in OBS. But we sure as hell don't ship them by default, or in our official repos. The same applies for Arch, Debian, Fedora, Gentoo, RHEL and CentOS. Ubuntu is the only distribution that official supports ZoL, and actually ships it in it's official repositories (and by default). What that means is that Canonical is effectively saying "we trust there's no legal reason why we cannot do this." No other distribution has made that claim. EDIT: Actually NixOS also supports it, but the point stands.
    - Filligree 9y ago
      Well, also NixOS.
  - jononor 9y ago
    Supported by whom? Is it at the level one can expect from paid support from say Red Hat, SUSE or Canonical?

tqh 9y ago

Same here. I lost a disk to btrfs and did not manage to recover any data. No longer on my list of filesystems-I-trust. I guess Suse is the biggest contributor since the others stopped? It doesn't give me any confidence that it can be trusted.

RX14 9y ago

Which raid level were you using over those multiple disks? I've been using btrfs and never even experienced a hiccup despite continuous sector failures, continuous drive failures and even disk controllers disconnecting on the fly. And I've been running the whole thing on a single, fairly weak, server with second-hand disks attached over the cheapest USB3 sata adapters I've found. By all accounts I've been putting btrfs in a situation that is well outside the norm and I have 0 complaints. I also use it on my desktop (no raid) with similar experience (minus the shitty hardware). I am continuously amazed at the amount of people who have issues with btrfs. It's been absolutely rock solid over the time I've used it and I have 0 complaints (apart from the ext4 -> btrfs converter producing a corrupted btrfs filesystem, but the actual kernel btrfs code itself has been flawless).

btmaybe 9y ago
I too have lost an entire btrfs volume (single disk) because of a bad sector. It seems btrfs is very sensitive to bad sectors in metadata, compared to ext4. Since then, I've formatted my btrfs partition to do RAID1 on metadata and haven't had problems since. $ sudo btrfs fi df / Data, single System, DUP Metadata, DUP
- Dylan16807 9y ago
  Isn't that the default?
  - sp332 9y ago
    "A single device filesystem will default [metadata] to DUP, unless a SSD is detected. Then it will default to single." https://btrfs.wiki.kernel.org/index.php/Manpage/mkfs.btrfs#OPTIONS https://btrfs.wiki.kernel.org/index.php/Manpage/mkfs.btrfs#O...
  - btmaybe 9y ago
    It's a relatively new feature (Jan-2016) and it's disabled by default for SSD devices (increased wear). Personally I prefer increased wear over unmountable volumes.