8 ms·

Lots of people seem to turn up their nose at btrfs. Is there a reason for that? Was it perhaps launched before it was really ready and people still remember ear

by api 5y ago

Lots of people seem to turn up their nose at btrfs. Is there a reason for that? Was it perhaps launched before it was really ready and people still remember early versions?

j16sdiz 5y ago
In their earlier days, the ENOSPCE bug corrupts the filesystem. If you do a heavy random write workload, it fills up the disk pretty quickly and require a re-balance _before_ ran out of space. Of cause you can do nocow on those files, but than it lost all the checksuming/snapshotting features.
pnutjam 5y ago
Yes, the only real problems are edge cases. I use it all the time.
- gh02t 5y ago
  I agree, I love BTRFS and have used it for ages, including some small scale production systems. But I know it still has some edge cases as you mention, which made me wonder: what is the impediment to having those cases fixed? BTRFS has been around long enough and even has some decent commercial support from a few vendors, so it seems like we can't just discount it as "it's open source and nobody is motivated to fix those long tail problems." Is there some kind of design issue that makes them hard?
  - tinco 5y ago
    edit: sorry, cheap shot at Facebook. I have no idea why BTRFS edge cases are not being fixed. What I do know is that ZFS recently released a feature specifically for the hobbyist/frugal community. The feature allows you to grow an existing RAID array, something a financially sound business would never do. So no customer of anyone supporting ZFS would ever use this, and it took significant effort of ZFS developers to implement this. Not to mention that introducing feature potentially introduces weird behaviour in ZFS that might endanger its (reputation of) stability. I'm super happy with it, (as my company was not in fact financially sound when we invested in our on-premise storage hardware), but if I was CEO of ZFS I'm not sure I'd sign off on it.
    - alophawen 5y ago
      Sarcastic comment adding nothing to the discussion. How rare.
    - Brian_K_White 5y ago
      I could already grow mdraid and reiserfs forever ago.
      - tinco 5y ago
        Yes.. but you couldn't grow ZFS. I don't understand what your point is.
        cestith 5y ago
        Unless Im'm terribly mistaken one could always grow ZFS, but it was via adding more pools.
        tecleandor 5y ago
        To be exact, either: adding more vdevs to your pool, or substituting ALL your drives in a vdev with a bigger size.
        Brian_K_White 5y ago
        Point was the post is about losing reiserfs, and people are presenting virtues of zfs and btrfs as reasons why we don't need reiserfs any more, and this virtue of zfs is unremarkable.
- kzrdude 5y ago
  Is ENOSPC still included as one of those edge cases?
- duskwuff 5y ago
  All problems are edge cases, to some degree or another. The only real question is how far out those edges are, and whether users are likely to bump into them.
- craftkiller 5y ago
  Edge cases like Raid5/6 which had the write hole issue approximately a decade after btrfs was released. At some point you say "This filesystem has lost so much of my data that I will never return to it."
  - pnutjam 5y ago
    That's pretty old news. It's been problem free for a long time and it's very well documented where you might have issues.
    - bastardoperator 5y ago
      Burn me once, shame on you, burn me twice, shame on me. If you purchased a new ford and that car fell apart a week later, would you ever buy a ford again? Some will, most wont.
      - tinus_hn 5y ago
        A better analogy would be if the car got you in an accident. I don’t care if something breaks quickly as long as that means it can be returned or replaced.
- cestith 5y ago
  What are the equivalent edge cases in XFS, ZFS, ext4, Reiser4, Reiser5, bcachefs, or f2fs that make btrfs worth considering on a level playing field?
  - pnutjam 5y ago
    The wiki has everything you need. https://btrfs.wiki.kernel.org/index.php/Status https://btrfs.wiki.kernel.org/index.php/Status https://btrfs.wiki.kernel.org/index.php/Gotchas https://btrfs.wiki.kernel.org/index.php/Gotchas
    - cestith 5y ago
      That is very informative about the edge cases for btrfs. My question was what are the edge cases in the other filesystems which put them on a level playing field with btrfs considersing it still has so many.
- user-the-name 5y ago
  I really do not want to use a file system that has problems at edge cases. A file system needs to be incredibly stable.
adastra22 5y ago
Lots of us got burnt with data loss and aren’t willing to give it a chance again. Maybe it’s better now? I don’t have a reason to give it a second chance when there are plenty of stable alternatives that have saved my ass I’m the past instead of telling me I’m SOL.
Syonyk 5y ago
> Is there a reason for that? I can give you mine. I was working with a Raspberry Pi 3, and using a USB SSD. It's a USB2 link, so a bit choked, and I figured, hey, filesystem compression can help here, btrfs supports it, great! And it helped - you could get "real world" disk reads a good bit faster than the USB2 bus speed. Until one day, I rebooted, and it didn't come back up. Analysis on another system was that the btrfs filesystem was just... toast. I've no idea what happened, I found some stuff that said "Oh, uh... don't use btrfs over USB, it kinda breaks in some cases...", the recovery tools couldn't even decide that the filesystem was a btrfs filesystem, and, nope. I put data on the filesystem, I expect it to come back. btrfs broke that guarantee with a Pi full of data (nothing too important, they're just scratch systems and light desktops), so... I now stick to the boring things like ext4 that have been exceedingly well proven. Is it the best filesystem out there in terms of features? Certainly not. Am I pretty darn sure that I'm not going to trip some edge case and totally scramble the filesystem? Yes, and that's what I care about.
- deleted 5y ago
  [deleted]
ak217 5y ago
That's exactly it. I've used btrfs in production since Ubuntu 10.04, at scale since 12.04, and had nothing but great experiences with it - especially with the seed volume functionality, which allowed me to build the foundation for a major container-as-a-service platform before Docker was a thing. btrfs never lost our data, but I've also seen way too many btrfs kernel panics that were clearly related to insufficiently mature filesystem code, and I can understand people who did lose data, got burned and never want to trust btrfs again.
__david__ 5y ago
For me, when I tried btrfs (which was about 10 years ago now) I discovered it was extremely slow. And not like 50% slower—when I switched to ext4 or xfs on the same disk with the same data I was getting a 10x or so speedup.
TheCondor 5y ago
It is a complex beast. It needs some maintenance and performance will degrade without it. I've never lost data to it, I've never tried the soft RAID modes it has though, but I've experienced it making a system almost unusably slow. SUSE out of the box with it automates a lot of it and it's pretty remarkable. Transactional mode if you want it seems like a game changer for some servers and the snapper stuff has saved my bacon a couple times. It's getting there but like I said, it needs some maintenance and just formatting a partition with it is likely the wrong way to experience it.
patrakov 5y ago
For me, it was https://bugzilla.kernel.org/show_bug.cgi?id=85581 https://bugzilla.kernel.org/show_bug.cgi?id=85581. Yes this endless-write loop is long fixed, but, given that something with 99% similar symptoms has surfaced in kernel 5.16 (or was this original bug not fixed properly?), I would say no.
MrDOS 5y ago
AFAIK it's not so bad in single-device use-cases. I think most of the more recent failures I've heard about have all had to do with Btrfs RAID. The prevailing wisdom still seems to be that if you want to use RAID, use an md soft-RAID device or LVM under your single-device Btrfs filesystem.
- cestith 5y ago
  RAID, especially 5 or 6 was my main concern, yes. If I'm using hardware RAID or a soft RAID under the FS, much of the promised benefit of btrfs is gone anyway. I can add to storage pools with ZFS or expand an LVM set, too, but what does using btrfs on top of anything buy me that ZFS, bcachefs, or something like f2fs does not?
  - MrDOS 5y ago
    > but what does using btrfs on top of anything buy me that ZFS, bcachefs, or something like f2fs does not? Well, inclusion in mainline kernels is the big one over ZFS and bcachefs, I guess. I haven't seen F2FS before, so I'm commenting on the basis of 30 seconds of Googling, here, but it doesn't look like it supports either copy-on-write or snapshots, which are the big selling points I've heard for continuing to use Btrfs on top of a device manager.
nwmcsween 5y ago
BTRFS was marked stable in 2012 yet it still has abysmal performance compared to zfs, ext4, xfs, etc.

by api 5y ago

Lots of people seem to turn up their nose at btrfs. Is there a reason for that? Was it perhaps launched before it was really ready and people still remember early versions?

j16sdiz 5y ago

In their earlier days, the ENOSPCE bug corrupts the filesystem. If you do a heavy random write workload, it fills up the disk pretty quickly and require a re-balance _before_ ran out of space. Of cause you can do nocow on those files, but than it lost all the checksuming/snapshotting features.

pnutjam 5y ago

Yes, the only real problems are edge cases. I use it all the time.

gh02t 5y ago
I agree, I love BTRFS and have used it for ages, including some small scale production systems. But I know it still has some edge cases as you mention, which made me wonder: what is the impediment to having those cases fixed? BTRFS has been around long enough and even has some decent commercial support from a few vendors, so it seems like we can't just discount it as "it's open source and nobody is motivated to fix those long tail problems." Is there some kind of design issue that makes them hard?
- tinco 5y ago
  edit: sorry, cheap shot at Facebook. I have no idea why BTRFS edge cases are not being fixed. What I do know is that ZFS recently released a feature specifically for the hobbyist/frugal community. The feature allows you to grow an existing RAID array, something a financially sound business would never do. So no customer of anyone supporting ZFS would ever use this, and it took significant effort of ZFS developers to implement this. Not to mention that introducing feature potentially introduces weird behaviour in ZFS that might endanger its (reputation of) stability. I'm super happy with it, (as my company was not in fact financially sound when we invested in our on-premise storage hardware), but if I was CEO of ZFS I'm not sure I'd sign off on it.
  - alophawen 5y ago
    Sarcastic comment adding nothing to the discussion. How rare.
  - Brian_K_White 5y ago
    I could already grow mdraid and reiserfs forever ago.
    - tinco 5y ago
      Yes.. but you couldn't grow ZFS. I don't understand what your point is.
      - cestith 5y ago
        Unless Im'm terribly mistaken one could always grow ZFS, but it was via adding more pools.
        tecleandor 5y ago
        To be exact, either: adding more vdevs to your pool, or substituting ALL your drives in a vdev with a bigger size.
      - Brian_K_White 5y ago
        Point was the post is about losing reiserfs, and people are presenting virtues of zfs and btrfs as reasons why we don't need reiserfs any more, and this virtue of zfs is unremarkable.
kzrdude 5y ago
Is ENOSPC still included as one of those edge cases?
duskwuff 5y ago
All problems are edge cases, to some degree or another. The only real question is how far out those edges are, and whether users are likely to bump into them.
craftkiller 5y ago
Edge cases like Raid5/6 which had the write hole issue approximately a decade after btrfs was released. At some point you say "This filesystem has lost so much of my data that I will never return to it."
- pnutjam 5y ago
  That's pretty old news. It's been problem free for a long time and it's very well documented where you might have issues.
  - bastardoperator 5y ago
    Burn me once, shame on you, burn me twice, shame on me. If you purchased a new ford and that car fell apart a week later, would you ever buy a ford again? Some will, most wont.
    - tinus_hn 5y ago
      A better analogy would be if the car got you in an accident. I don’t care if something breaks quickly as long as that means it can be returned or replaced.
cestith 5y ago
What are the equivalent edge cases in XFS, ZFS, ext4, Reiser4, Reiser5, bcachefs, or f2fs that make btrfs worth considering on a level playing field?
- pnutjam 5y ago
  The wiki has everything you need. https://btrfs.wiki.kernel.org/index.php/Status https://btrfs.wiki.kernel.org/index.php/Status https://btrfs.wiki.kernel.org/index.php/Gotchas https://btrfs.wiki.kernel.org/index.php/Gotchas
  - cestith 5y ago
    That is very informative about the edge cases for btrfs. My question was what are the edge cases in the other filesystems which put them on a level playing field with btrfs considersing it still has so many.
user-the-name 5y ago
I really do not want to use a file system that has problems at edge cases. A file system needs to be incredibly stable.

adastra22 5y ago

Lots of us got burnt with data loss and aren’t willing to give it a chance again. Maybe it’s better now? I don’t have a reason to give it a second chance when there are plenty of stable alternatives that have saved my ass I’m the past instead of telling me I’m SOL.

Syonyk 5y ago

> Is there a reason for that? I can give you mine. I was working with a Raspberry Pi 3, and using a USB SSD. It's a USB2 link, so a bit choked, and I figured, hey, filesystem compression can help here, btrfs supports it, great! And it helped - you could get "real world" disk reads a good bit faster than the USB2 bus speed. Until one day, I rebooted, and it didn't come back up. Analysis on another system was that the btrfs filesystem was just... toast. I've no idea what happened, I found some stuff that said "Oh, uh... don't use btrfs over USB, it kinda breaks in some cases...", the recovery tools couldn't even decide that the filesystem was a btrfs filesystem, and, nope. I put data on the filesystem, I expect it to come back. btrfs broke that guarantee with a Pi full of data (nothing too important, they're just scratch systems and light desktops), so... I now stick to the boring things like ext4 that have been exceedingly well proven. Is it the best filesystem out there in terms of features? Certainly not. Am I pretty darn sure that I'm not going to trip some edge case and totally scramble the filesystem? Yes, and that's what I care about.

deleted 5y ago
[deleted]

ak217 5y ago

That's exactly it. I've used btrfs in production since Ubuntu 10.04, at scale since 12.04, and had nothing but great experiences with it - especially with the seed volume functionality, which allowed me to build the foundation for a major container-as-a-service platform before Docker was a thing. btrfs never lost our data, but I've also seen way too many btrfs kernel panics that were clearly related to insufficiently mature filesystem code, and I can understand people who did lose data, got burned and never want to trust btrfs again.

__david__ 5y ago

For me, when I tried btrfs (which was about 10 years ago now) I discovered it was extremely slow. And not like 50% slower—when I switched to ext4 or xfs on the same disk with the same data I was getting a 10x or so speedup.

TheCondor 5y ago

It is a complex beast. It needs some maintenance and performance will degrade without it. I've never lost data to it, I've never tried the soft RAID modes it has though, but I've experienced it making a system almost unusably slow. SUSE out of the box with it automates a lot of it and it's pretty remarkable. Transactional mode if you want it seems like a game changer for some servers and the snapper stuff has saved my bacon a couple times. It's getting there but like I said, it needs some maintenance and just formatting a partition with it is likely the wrong way to experience it.

patrakov 5y ago

For me, it was https://bugzilla.kernel.org/show_bug.cgi?id=85581 https://bugzilla.kernel.org/show_bug.cgi?id=85581. Yes this endless-write loop is long fixed, but, given that something with 99% similar symptoms has surfaced in kernel 5.16 (or was this original bug not fixed properly?), I would say no.

MrDOS 5y ago

AFAIK it's not so bad in single-device use-cases. I think most of the more recent failures I've heard about have all had to do with Btrfs RAID. The prevailing wisdom still seems to be that if you want to use RAID, use an md soft-RAID device or LVM under your single-device Btrfs filesystem.

cestith 5y ago
RAID, especially 5 or 6 was my main concern, yes. If I'm using hardware RAID or a soft RAID under the FS, much of the promised benefit of btrfs is gone anyway. I can add to storage pools with ZFS or expand an LVM set, too, but what does using btrfs on top of anything buy me that ZFS, bcachefs, or something like f2fs does not?
- MrDOS 5y ago
  > but what does using btrfs on top of anything buy me that ZFS, bcachefs, or something like f2fs does not? Well, inclusion in mainline kernels is the big one over ZFS and bcachefs, I guess. I haven't seen F2FS before, so I'm commenting on the basis of 30 seconds of Googling, here, but it doesn't look like it supports either copy-on-write or snapshots, which are the big selling points I've heard for continuing to use Btrfs on top of a device manager.

nwmcsween 5y ago

BTRFS was marked stable in 2012 yet it still has abysmal performance compared to zfs, ext4, xfs, etc.