11 ms·

I used to be in the "why BTRFS" camp and religiously would only install ext4 without LVM on Fedora for my laptops, desktops, and servers. When I saw so many su

• regularfry 5y ago
  It certainly used to be the case that BTRFS had some nasty behaviour patterns when it started to run low on space. It could well be that it has not presented any problem for you yet. On the other hand, those days might be behind it. I haven't kept track.
  • cmurf 5y ago
    There are still edge cases but I can count on one hand the number of users who have run into it since Fedora switched to Btrfs by default (on desktops) two years ago (and Cloud edition in the most recent Fedora release late last year). The upstream maintainer of btrfs-progs also maintains btrfs maintenance https://github.com/kdave/btrfsmaintenance https://github.com/kdave/btrfsmaintenance which is a set of scripts for doing various tasks, one of which is a periodic filtered balance focusing on data block groups. I don't run it myself, because well (a) I haven't run into such a bug in years myself (b) I want to run into such a bug so I can report it and get it fixed. The maintenance script basically papers over any remaining issues, for those folks who are more interested in avoiding issues than bug reporting (a completely reasonable position). There's been a metric ton of work on ENOSPC bugs over the years, but a large pile were set free with the ticketed ENOSPC system quite a few years ago now (circa 2015? 2016?)
• pkulak 5y ago
  It's obviously not there as a NAS filesystem, ZFS drop-in replacement, etc. But if what you take away from that is that BTRFS is no good as a filesystem on a single drive system, you're missing out. Just a few weeks ago I used a snapshot to get myself out of some horrible rebase issue that lost half my changes. Could I have gone to the reflog and done other magic? Probably. But browsing my .snapshots directory was infinitely easier!
  • azinman2 5y ago
    Synology ships it for their NAS
    • doublepg23 5y ago
      Sort of. They use it on top of dm-raid and use dm-integrity for the checksum features. They claim BTRFS RAID is unstable. https://kb.synology.com/en-us/DSM/tutorial/What_was_the_RAID_implementation_for_Btrfs_File_System_on_SynologyNAS https://kb.synology.com/en-us/DSM/tutorial/What_was_the_RAID...
      • ziml77 5y ago
        Does BTRFS actually need to be augmented like that? I've been afraid of using it for a NAS because it doesn't sound like it's as trustworthy as ZFS when it comes to handling bitrot. But I don't know if that's actually true. When I tried to find info on it a couple weeks ago, a lot of people were trying to claim that bitrot isn't a thing if you have ECC RAM.
        • michaelmrose 5y ago
          It's raid 5/6 comes with a warning from the developers not to use it and RAID 1 is a weird arrangement that actually keeps 2 copies on however many disks and can lose data if a disk comes and goes for example with a bad cable. Bitrot still happens with ECC.
          • 5y ago
• bgorman 5y ago
  I have not used BTRFS for years, but I remember at some point a BTRFS regression prevented me from booting my system. It is hard to regain trust after such a meltdown from such a fundamental component. That said, I believe my Synology NAS uses btrfs and it has never had an issue.
  • F00Fbug 5y ago
    I've been in the same boat. Around 2012 or 2013 I put BTRFS on my DIY NAS/media server. For some reason, totally unprovoked the array just went belly up with no warning or logs. I tried and tried without success and couldn't recover it. Fortunately I had good, recent backups and restored to ext4+LVM and I'm still there 10 years later. BTRFS sounded cool with all it's new features, but the reality is that ext4+LVM does absolutely everything I need and it's never given me any issues. I'm sure BTRFS is much more robust these days, but I'm still gun shy!
    • stjohnswarts 5y ago
      I sort of had the same experience, dropped it for a decade, and came back around. It's a lot more robust these days. Also the btfsmaintenance tools take a lot of the load off of an admin. I just use the default settings and don't have any issues.
    • danudey 5y ago
      In 2019 I was setting up my new computer at my new job, and the Ubuntu installer had btrfs as an option. Figuring that it had been ages since the last time I'd heard about btrfs issues, I opted for that. A week later, the power failed in the office, and my filesystem got corrupted. In the middle of repairing, the power dipped again, and my entire fs was unrecoverable after that. I managed to get the data off by booting off a separate drive and running a command to extract everything, but it would never mount no matter what I did. I've never had an issue with ext4, xfs, or zfs no matter how much I've abused them over the past 10+ years, but if losing power twice can wipe out my filesystem then no thanks, I'm out. (Plus: non-recursive snapshots? No thanks.)
      • 5y ago
• badsectoracula 5y ago
  > deduplication, checksumming, snapshotting, and so many other features that modern FSes generally come with (e.g., ZFS) Does any other FS on Linux provide those?
  • throwawaylinux 5y ago
    That's traditionally the domain of logical volume managers.
    • coder543 5y ago
      Snapshots, sure. But... any of the other stuff? Checksums, deduplication, compression, etc. Citations, please. LVM VDO wasn't really a thing until a couple of years ago, and I've never actually heard anyone recommend using it. Definitely not within its "traditional domain". Perhaps you mean on other operating systems... when this discussion is all about a Linux filesystem and whether other Linux filesystems provided these features, including the comment you replied to?
      • throwawaylinux 5y ago
        > But... any of the other stuff? Checksums, deduplication, compression, etc. Citations, please. Yes. The Linux LVM logical volume manager for example can do those things. Traditional domain meaning this kind of data management traditionally came under the purview of LVMs. Not that an implementation has had particular features for a length of time, but that you might (also) look there rather than the filesystem for those features.
        • hnlmorg 5y ago
          Deduplication and compression have traditionally been file system features not volume management features: As fair as I know the only solution for deduplication in LVM is VDO and that was created after ZFS and isn't technically part of LVM (it's a kernel module that sits between the filesystem and LVM). Same is true for compression except in the case of compression that's been a file system feature since the days of NTFS on NT 3.5 (released 1995) -- I can't recall when UNIX file systems first saw compression but I'd wager it was before ZFS, which we've already acknowledge predates VDO. I also thing it makes sense that the above should be part of the file system domain because it's an integral part to how the file system organizes the data (much like how we don't argue that the journal isn't part of the file system domain). As for checksums, they have existed in all parts of the stack, from the application level (for example rsync), to the volume level (RAID controller cards) to the file system. I wouldn't say that's something that has ever had a "traditional domain". The things I'd say ZFS and its ilk do that isn't part of its traditional domain is RAIDing (ie management of the physical devices) and cache control. Both make a lot of sense being managed by the file system in modern file systems but I can also sympathize with those who think that's a step too far. In the case of RAIDing, you can still run ZFS on LVM or a hardware controller if you wish -- I wouldn't suggest you do so because it gives you additional complications for zero benefit but the option is still there if you want it. But with regards to cache, the only way to opt out of ZFS managing it's own cache is not to use ZFS.
• dotancohen 5y ago
  Are you using a UPS on the desktop? A recent HN thread highlighted BTRFS issues, especially with regards to dataloss on power loss. Also, there's a "write hole issue" on some RAID configurations, RAID 4 or 6 I think. That said, I'm thinking about leaving a BTRFS partition unmounted and mounting it only to perform backups, taking advantage of the snapshotting features.
  • waffleiron 5y ago
    It's RAID 5/6, there is a great status page: https://btrfs.wiki.kernel.org/index.php/Status https://btrfs.wiki.kernel.org/index.php/Status
  • michaelmrose 5y ago
    This would seem to suggest that ANY raid configuration is unacceptable. Device replace -------------- >Device replace and device delete insist on being able to read or reconstruct all data. If any read fails due to an IO error, the delete/replace operation is aborted and the administrator must remove or replace the damaged data before trying again. Device replace isn't something where "mostly ok" is a good enough status.
    • wtallis 5y ago
      This doesn't sound like a bug to me. It sounds like a refusal to fail silently.
      • withinboredom 5y ago
        If you’re removing a device, it shouldn’t matter if the data is unreadable (that’s probably why it’s being removed, no?)
        • wtallis 5y ago
          I interpreted "being able to read or reconstruct all data" as meaning that there must exist a good valid copy of each chunk of data somewhere in the array, but not necessarily on the device that you're trying to remove or replace. That interpretation matches my experience, which is that btrfs can correctly handle replacing a drive that is dead or dying. You should certainly expect errors if eg. both copies of a chunk in a RAID1 are lost/corrupted.
• 0xbadcafebee 5y ago
  > If the distributions are willing to take that maintenance on their shoulders, I'm willing to trust them and deal with the consequences – at least I know I'm not alone. But then they make changes that add an insane amount of complexity, and suddenly you're running into random errors and googling all the time to try to find the magical fixes to all the problems you didn't have before.
• alerighi 5y ago
  BTRFS works fine. I use it on my everyday laptop without problems. Compression can help on devices with not a lot of disk, and also copy on write. However, BTRFS has its drawbacks, for example it's tricky to have a swapfile on it (now it's possible with some special attributes). Also I wouldn't trust BTRFS for the purpose of data archiving, for the fact that ext4 is a proven (and simpler) filesystem, thus it's less likely to became corrupt, and it's more likely to being able to recover data from it if it will become corrupt (or the disk has some bad sectors and that sort of stuff).
  • cmurf 5y ago
    I'm not sure how you prove that ext4 is less likely to become corrupt. But it is easily shown that it's less likely to inform you that there's corruption. Quite a lot of the assumptions of earlier file systems is the hardware either returns correctness, or reports a problem e.g. uncorrectable read error or media error. That's been shown to be untrue even with enterprise class hardware, largely by the ZFS developers, hence why it exists. And also why ZFS has had quite a lot less "bad press" where Btrfs wasn't developed in a kind of skunkworks, it was developed out in the open where quite a lot of early users were using it with ordinary every day hardware. And as it turns out, we see most hardware by make/model doing mostly the right things, a small number of make/models, making up a significant minority of usage volume, don't do the right things. Hence, Btrfs has always had full checksumming of data and metadata. Both XFS and ext4 were running into the same kinds of problems Btrfs (and ZFS before it) revealed - torn writes, misdirected writes, bit rot, memory bit flips, and even SSD's exhibit prefail behavior by returning either zeros or garbage instead of data (or metadata). XFS and ext4 subsequently added metadata checksums, which further reinforced the understanding that devices sometimes do the wrong thing and also lie about it. It is true that overwriting filesystems have a better chance of repairing metadata inconsistencies. A big reason why is locality. They have fixed locations on disk for different kinds of metadata, thus a lot of correct assumptions can be made about what should be in that location. Btrfs doesn't have that at all, it has very few fixed locations for metadata (pretty much just the super blocks). Since no assumptions can be made about what's been found in metadata areas, it's harder to fix. So the strategy is different with Btrfs (and probably ZFS too since it has a fairly nascent fsck even compared to Btrfs's) - cheap and fast replication of data via snapshots and send/receive, which requires no deep traversal of either the source or destination. And equally cheap and fast restore (backwards replication) using the same method. Conversely, conventional backup and restore are meaningfully different when reversing, so you have to test both the backup and restore to really understand if your backup method is reliable. That's going to be your disaster go to rather than trying to fix them. Fixing is almost certainly going to take much longer than restoring. If you don't have current backups, at least Btrfs now has various rescue mount options to make the file system more tolerant of broken file systems, but as a consequence you also have to mount read-only. Pretty good chance you can still get your data out, even if it's inconvenient to have to wipe the file system and create a new one. It'll still be faster than mucking with repair. Also, Btrfs since kernel 5.3 has both read time and write time tree checkers, that verify certain trees for consistency, not just blindly accepting checksums. Various problems are exposed and stopped before they can cause worse problems, and even helps find memory bitflips and btrfs bugs. Btrfs doesn't just complain about hardware related issues, it'll rat itself out if it's to blame for the problem - which at this point isn't happening any more often than ext4 or XFS in very large deployments (millions of instances).
• stjohnswarts 5y ago
  I was an early adopter and some bad experiences early on made it a bitter pill. I swore it off for a decade, and about year and half came back around. It's MUCH MUCH better now. With automated "sensible" settings for btrfsmaintenance tools it's actually just fine now.
  • hnlmorg 5y ago
    I honestly don't see the point of Btrfs. ZFS is more mature, more stable, better defaults, etc. The only reason Btrfs existed was because ZFS wasn't GPL. But now we have a way of running ZFS on Linux Btrfs seems utterly redundant. I mean I'm all for choice on Linux but when it comes to file systems I'd rather have fewer choices but those choices be absolutely rock solid. Btrfs might have gotten to that stage now however it's eaten enough peoples data (including mine) over the years that I can't help wondering why people bothered to persist with using it when a better option was available.
• atoav 5y ago
  One of the a little weird things about btrfs is that some Software (e.g. OBS) seems to have a hard time to get the free space on disk. Maybe because they assume the usual ways of getting free space works (which doesn't on btrfs)
• nix23 5y ago
  DONT DO THAT: Try to fill your root-filesystem with dd, then remove the file and sync, reboot and enjoy a non booting OS ;) It's like they don't test it at all.
• pronik 5y ago
  Anecdata: Once upon a time I installed a SuSE with their default choice of ReiserFS (v3) as root partition. Couple of months later that filesystem was dead beyond repair. I don't know whether I did something wrong, but I've been very wary of "defaults" ever since. That said, that was a different time and I tend to see a ZFS or a BTRFS in my near future.