11 ms·

Bcachefs File-System Plans to Try Again to Land in Linux 6.6

113 points by mikece 3y ago

insanitybit 3y ago
Anyone know how the compression features of bcachefs work? At what level is the compression happening?
- throwaway29303 3y ago
  There's plenty of information about bcachefs here: https://bcachefs.org/bcachefs-principles-of-operation.pdf https://bcachefs.org/bcachefs-principles-of-operation.pdf
  - insanitybit 3y ago
    Thanks! This looks perfect.
- tyingq 3y ago
  Reading the peer comment document, the unique thing about compression with bcachefs is that it compresses extents, rather than blocks. Where a compressed and checksummed extent is 64Kb, and a block is typically 2/4/8 Kb. So better compression at the cost of slightly reduced performance for random reads. Edit: I was paraphrasing the text in the peer linked document: "As with data checksumming, we compress entire extents, not individual disk blocks - this gives us better compression ratios than other filesystems, at the cost of reduced small random read performance."
  - cwillu 3y ago
    That's not unique to bcachefs; it's how transparent compression is implemented in every filesystem that implements it that I'm aware of (for example, I just checked NTFS, BTRFS and ZFS).
    - mastax 3y ago
      ZFS compresses records (blocks) independently.
      - someplaceguy 3y ago
        A ZFS block is 128 KB by default (smaller if the file is < 128 KB, but can also be larger if you manually configure it). So a ZFS block is even larger than a bcachefs "extent".
        mastax 3y ago
        Well yes, my point is that (counter to GP) ZFS doesn't combine together multiple blocks into a compression extent, rather it uses larger blocks by default.
        someplaceguy 3y ago
        ZFS's larger blocks you are referring to are the logical block size, which is the size unit for compression among other things. I don't know much about bcachefs, but as I understand it, a logical block in ZFS is equivalent to a bcachefs extent (in terms of compression, at least) and a ZFS physical block is equivalent to a (possibly empty) set of bcachefs blocks. As an example, in terms of actual on-disk storage, a 128 KB logical block in ZFS can be compressed down to (for example) a 3.5 KB "physical" block and it will only actually consume 3.5 KB of disk space (surprisingly, the block size of a ZFS physical block doesn't have to be a power of 2!). That said, if you have 4 KB-sector disks then you should probably use a physical block size granularity of 4 KB for your ZFS pool rather than 512 bytes. So, regardless of the file size, an on-disk ZFS block can actually occupy anywhere from 0 bytes up to 1 MB, with a granularity of up to 512 bytes, depending on how much it is compressed, what the physical block size granularity of the ZFS pool is and what you've selected as the recordsize property (i.e. the logical block size) for the corresponding filesystem in the pool.
        rincebrain 3y ago
  - magicalhippo 3y ago
    This is similar to how ZFS does it, and probably others, so hardly unique. In ZFS they call it "records" or "blocks" rather than "extents", and the default value record size is 128KiB.
j16sdiz 3y ago
Don't hold your breath. This is lkml business as usual. Patches as large as this usually take a few tries to get things into shape.
- simcop2387 3y ago
  That said, this isn't the first time bcachefs has been through this process. Though previous runs were mostly just an RFC for finding issues and getting input on how things are structured and seeing if there are any difficult to understand/reason about areas. I'm not sure I expect it to end up in 6.6 just because reviews like this do take a long time anyway, but the 6.6 merge window is still a decently long time away from happening. If it doesn't make 6.6 and no major issues are found I could definitely see 6.7 being the one it's in. That'd likely mean it'd be sometime in the late fall early winter (northern hemisphere) which is still great news. I've been looking forward to bcachefs coming in as an alternative to XFS/ext4 on the lower end of features and as an alternative for ZFS for when you don't need or want all of the stuff it has either. It looks like it should shape up to being a very capable middle ground with a lot of good stuff in it.
  - danudey 3y ago
    It sounds like one of the biggest issues with merging bcachefs is that the patch is huge and touches systems outside of bcachefs. I suspect if they had taken those external patches and submitted them separately they could have had incremental acceptance (or at least discussion) of those patches and we might be further along, or at least know how far along we actually are.
    - simcop2387 3y ago
      That's my understanding too, and I believe that's what came out of the previous RFC discussions it had before making it to this merge discussion. The parts that go outside the FS itself I believe have been split out better so that it's easier to review and I think can be taken in parts to verify that it doesn't cause issues in those external places.
- 3y ago
mustache_kimono 3y ago
Read maybe three comments deep in the actual PR[0], and you'll get the impression this is definitely not going to ever be merged. I've said it 1000x -- OpenZFS is already free and open source. Linux kernel developers should just stop punching themselves in face. The ZFS hair shirt is self imposed by semi-religious Linux wackadoos. [0]: https://lore.kernel.org/lkml/20230626214656.hcp4puionmtoloat@moria.home.lan/ https://lore.kernel.org/lkml/20230626214656.hcp4puionmtoloat...
- dralley 3y ago
  I've read several such LKML threads about bcachefs and while the discussions have been contentious (and Kent could stand to chill out a bit), everyone has been in agreement that they'd like to see bcachefs succeed and be merged. The major technical blockers have been addressed or are being addressed. Bcachefs also has plenty of features which ZFS does not, so I don't see why you're so critical. Oracle could relicense ZFS at any time, or make an explicit statement about its legal status. They have not done so after 10 years.
  - gigatexal 3y ago
    yeah i love love love ZFS and use it everywhere and am very excited to see Bcachefs get merged. More legit competition in this space is good. Btrfs doesn't apply, it's not even in the same league.
  - mustache_kimono 3y ago
    >I've read several such LKML threads about bcachefs and while the discussions have been contentious (and Kent could stand to chill out a bit), everyone has been in agreement that they'd like to see bcachefs succeed and be merged. Bcachefs also has plenty of features which ZFS does not, so I don't see why you're so critical. Only because I've read this story more than a few times before? See: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=%22bcachefs%22%20merge&sort=byPopularity&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... >Oracle could relicense ZFS at any time, or make an explicit statement about its legal status. They have not done so after 10 years. Any legal difficulty remains from the GPL side, and no suit has been brought in the years upon years of the GPL hair shirts screaming about non-compliance. Unfortunately, the GPL hair shirts don't have much (any) credibility on the subject of what the GPL means. Bring the suit or STFU. I have no patience for armchair legal opinions from people who won't put their money where their mouth is. Stop spreading legal FUD!
Geezus_42 3y ago
ATM machine
- simcop2387 3y ago
  Kind of sort of mostly? The big difference here is that Bcachefs is a filesystem based originally on the design of Bcache which is a block device caching layer, so it'd be very easy to miss the fs on the end and confuse Bcachefs for Bcache. Because of this, calling it Bcache filesystem also doesn't really work because it's not Bcache anymore either.
bmurphy1976 3y ago
Question: I currently run ZFS via Ubuntu on my home devices. I've had a very positive experience with it. My only (minor) complaint is the extra 30 seconds it adds to my desktop's boot time. It's been fantastic for all my other devices, particularly the snapshotting features and ease of which I can ship the snapshots between the devices. I've read some comments on various forums that seem to hint Ubuntu is moving on from ZFS (see https://discourse.ubuntu.com/t/future-of-zfs-on-ubuntu-desktop/33001 https://discourse.ubuntu.com/t/future-of-zfs-on-ubuntu-deskt... for one example). This is obviously concerning and the lack of support across the whole Linux ecosystem continues to be a nuisance. Realistically how well could bcachefs serve as a long term ZFS replacement? I know very little about it. I've used btrfs in the past, and between my negative experiences and the repeated anecdotes posted all over the internet to this day I'm hesitant to give it another go.
- Filligree 3y ago
  I tried precisely that, and had it corrupt my data… twice. Bcachefs isn’t ready for prime time. I’m reasonably sure it’s better designed than Btrfs, and it’ll get there, but don’t do it yet. If Ubuntu becomes a problem, NixOS has 100% support for ZFS.
  - koverstreet 3y ago
    Were you able to report this? If you can get me enough information (e.g. via bcachefs dump), I'd like to track this down.
    - Filligree 3y ago
      We talked on IRC. :-) I'm planning to do that, but it'll have to be after my vacation; don't want to be running bcachefs on a remote server I can't fix. For those following along, this sort of response is why I'm optimistic overall. And bcachefs was dramatically higher performance than ZFS, although the lack of an ARC is a particular pain point. === It's going to be an interesting bug, I'm pretty sure. It affected two sets of files: The Steam Linux runtime (Solder), and everything in /nix/store, but absolutely nothing else. Actually, I think it's probably a bug in the fsck routine; everything worked perfectly until I rebooted.
snvzz 3y ago
Another option is Dragonfly's HAMMER2[0]. 0. https://www.dragonflybsd.org/hammer/ https://www.dragonflybsd.org/hammer/
- viraptor 3y ago
  Not on Linux. There weren't any serious attempts to port it as far as I know.
  - londons_explore 3y ago
    It's a bit of a shame that filesystems are so OS-specific. It's kinda a shame that there isn't at least some FUSE layer to let you run a filesystem driver from windows/mac/bsd/whatever on linux.
    - yjftsjthsd-h 3y ago
      FWIW FUSE kind of is that layer; a filesystem targeting FUSE should at least be trivially portable across the FOSS unix-likes.
- 1MachineElf 3y ago
  I really wish HAMMER2 had more traction. It's BSD licensing means it has the potential to be supported on Linux, Windows, macOS, etc. (in addition to other BSDs.)
pengaru 3y ago
Nice to see even the graybeards on LKML are as susceptible to bikeshedding as anyone else. Note how much heated debate is happening around how Kent is behaving, rather than the substance of his patchset. The former requires far less effort and knowledge to criticize and attack...
- alphazard 3y ago
  The real feedback has already happened. Last year Kent sent out the initial patches, and the main feedback from Linus was about the Git history and locking. All the important stuff is sorted out, it's in good shape to go into the kernel. The issue now is there are lots of maintainers who don't like that someone else did something impressive that they couldn't have done, and they are dragging their feet getting the various patches outside `fs/bcachefs` integrated. The design and implementation is basically just Kent, there are others onboarding to help with maintenance. There is a "Wireguard" level of craftsmanship in this project, it's a big deal. A younger Linus, in another time, might have told everyone to cut the bullshit--if you can't comment on the code, shut up, you're probably incompetent--but that's not the way things work anymore.
demizer 3y ago
I've been using it for the last two years and I think it has potential over btrfs, and zfs specifically. I used zfs on my hobby data array for 10 years prior but decided to downsize everything and use bcachefs in the hopes of it one day being in the kernel (unlike zfs). Device management is super easy, but is still missing knobs and dials for the most part. Multi device fault tolerance also needs work as I have had to recreate a 16tb array twice now due to journal corruption on power outages. I'll keep using it and reporting bugs, but I do feel the author is letting some bugs slide to focus on getting the fs in the kernel so that it gets more capable users that can provide patches. I am a sophisticated user, but definitely not a kernel developer.
- londons_explore 3y ago
  > as I have had to recreate a 16tb array twice now due to journal corruption on power outages. This isn't a good sign... Journalling should never, if bug-free, lead to data corruption or loss even if there is a power outage. And, even if there was some bug, one would expect a robust filesystem to say "ah, there is some data corruption here, so we're gonna run an fsck and recover every single file on the disk except perhaps the one or two that the bug clobbered."
  - cmurf 3y ago
    ZFS and Btrfs have demonstrated devices have a variety of transient failures including maintaining write order implied by fsync or fua. This will thwart any filesystem. SSDs do not reliably report UNC read errors when data can't be retrieved. Garbage or zeros are returned instead. There's a reason why ext4 and XFS added journal and metadata checksumming. Storage devices just aren't as reliable at informing the kernel when it suspects the data returned is bad.
    - Dylan16807 3y ago
      Incorrect write order shouldn't thwart a CoW filesystem. It can check at mount time whether the last few commits are fully there.
londons_explore 3y ago
I want a multi-device filesystem, offering (typically) the performance of the best device, yet an upper bound on data loss if one device fails. Eg. I could have an SSD and a slow hard disk. Data is written to the SSD first, and eventually mirrored to the hard disk. If the SSD fails, the hard disk will still be fully consistent up to the point the mirroring process got to. That mirroring would never be allowed to get more than say 5 minutes behind (writes would be throttled if that ever happened). This lets you have performance and resilience while paying less money. The scheme can be extended with erasure codes to say 32 drives for a huge database so that you could for example sustain any 3 device failures not losing any fsynced bytes of data, or up to 6 failures but you might lose up to 30 seconds of data. I think many multi-device filesystems could support this mode, but I don't think any do.
- dymk 3y ago
  What's are the usecases of a filesystem like that? I can't think of a situation where I care that my data is mirrored, but I'm okay with losing 5 minutes of it.
  - londons_explore 3y ago
    Your home desktop PC? I'd be perfectly fine losing 5 minutes of data in case of a disk failure. After all - I can probably recreate it in about 5 minutes. Anyone who does daily backups is effectively saying "I care about my data, but I'm happy to lose up to a days worth if there is a failure".
    - dymk 3y ago
      I suppose I would just throw another SSD in there and mirror with RAID1 (or equivalent), and save myself the headache. SSDs are cheap - A Samsung 980 1TB is $50. Edit: I think you can achieve what you want with current tools, or something pretty close at least, with ZFS. Create a snapshot of your SSD pool every 5 minutes, then stream the delta to an HDD pool in an automated fashion. Delete snapshots more than X hours / days old.
vorticalbox 3y ago
I have never really paid much attention to filesystems on Linux just installed which ever the installer is set to use, with that in mind do we need another filesystem?
- deleted 3y ago
  [deleted]
- danudey 3y ago
  bcachefs supports a lot of features that would be useful for even home user desktop systems. For example, the filesystem supports snapshots. This would let you do something like have apt (in Ubuntu) create an automatic snapshot of the filesystem before running an upgrade, so that if the upgrade fails or causes problems you can roll it back trivially. It also supports native compression, saving users tons of disk space. It supports combining multiple devices together for redundancy, and it can checksum all of your data so that you can know if a file was corrupted without it introducing transparent failures into your system. It supports copy-on-write, so that you can make instantaneous "copies" of files or directories which take no extra space unless you change them. So yeah, there are a lot of functionalities that we can enable with newer filesystems. ext4 is pretty great all in all, but there are a lot of modern features that modern hardware and concepts enable.
- 5e92cb50239222b 3y ago
  If you go by features, there aren't many alternatives. Unless I set up a database server, I consider checksums + snapshots + compression + reflinks to be table stakes these days. There is only one mainline filesystem that supports all of that, and it has its share of issues that are unlikely to ever be fixed, and is (probably permanently) tarnished in many users' eyes because it was pushed too early. So the question is, do we really need a second filesystem? (Yes?)
- bityard 3y ago
  Different file systems are optimized for different workloads and use cases. If you're just installing Linux on a laptop for medium-duty use, then you don't need anything special and can go with whatever general-purpose FS the distribution recommends, usually ext4. (Although I am partial to XFS myself.) You only need to consider more specialized filesystems if you have unique needs. For example, if you are setting up a file server, you might want a filesystem that is good at disk redundancy and data durability. Or you might need a filesystem that can handle billions of small files. Or you might have a huge database that needs random access to large amounts of flash storage. All of these are very different scenarios that require different (competing) things from the filesystems. In this case, bcachefs _seems_ to be promising many of the features of ZFS but with a better design and less overhead, potentially bringing these features to general-purpose workloads as well. (ZFS is fairly resource-intensive: in order to make full use of its features, you need a dedicated box for it, or really beef up the hardware on the application server that it will run on.)
deleted 3y ago
[deleted]
dodox 3y ago
Bcachefs does not yet implement scrubs. Corrupted data can be detected, reading instead from a copy (if available). But the problems get neither reported nor fixed. So does not seem production ready yet... https://github.com/koverstreet/bcachefs/issues/146 https://github.com/koverstreet/bcachefs/issues/146
Roark66 3y ago
Could someone explain what is the expected main benefit over btrfs? I've been running btrfs on all my "fast&busy" disks for quite a while, including hourly snapshots etc and I've been very happy with it.

113 points by mikece 3y ago

insanitybit 3y ago

Anyone know how the compression features of bcachefs work? At what level is the compression happening?

throwaway29303 3y ago
There's plenty of information about bcachefs here: https://bcachefs.org/bcachefs-principles-of-operation.pdf https://bcachefs.org/bcachefs-principles-of-operation.pdf
- insanitybit 3y ago
  Thanks! This looks perfect.
tyingq 3y ago
Reading the peer comment document, the unique thing about compression with bcachefs is that it compresses extents, rather than blocks. Where a compressed and checksummed extent is 64Kb, and a block is typically 2/4/8 Kb. So better compression at the cost of slightly reduced performance for random reads. Edit: I was paraphrasing the text in the peer linked document: "As with data checksumming, we compress entire extents, not individual disk blocks - this gives us better compression ratios than other filesystems, at the cost of reduced small random read performance."
- cwillu 3y ago
  That's not unique to bcachefs; it's how transparent compression is implemented in every filesystem that implements it that I'm aware of (for example, I just checked NTFS, BTRFS and ZFS).
  - mastax 3y ago
    ZFS compresses records (blocks) independently.
    - someplaceguy 3y ago
      A ZFS block is 128 KB by default (smaller if the file is < 128 KB, but can also be larger if you manually configure it). So a ZFS block is even larger than a bcachefs "extent".
      - mastax 3y ago
        Well yes, my point is that (counter to GP) ZFS doesn't combine together multiple blocks into a compression extent, rather it uses larger blocks by default.
        someplaceguy 3y ago
        ZFS's larger blocks you are referring to are the logical block size, which is the size unit for compression among other things. I don't know much about bcachefs, but as I understand it, a logical block in ZFS is equivalent to a bcachefs extent (in terms of compression, at least) and a ZFS physical block is equivalent to a (possibly empty) set of bcachefs blocks. As an example, in terms of actual on-disk storage, a 128 KB logical block in ZFS can be compressed down to (for example) a 3.5 KB "physical" block and it will only actually consume 3.5 KB of disk space (surprisingly, the block size of a ZFS physical block doesn't have to be a power of 2!). That said, if you have 4 KB-sector disks then you should probably use a physical block size granularity of 4 KB for your ZFS pool rather than 512 bytes. So, regardless of the file size, an on-disk ZFS block can actually occupy anywhere from 0 bytes up to 1 MB, with a granularity of up to 512 bytes, depending on how much it is compressed, what the physical block size granularity of the ZFS pool is and what you've selected as the recordsize property (i.e. the logical block size) for the corresponding filesystem in the pool.
        rincebrain 3y ago
- magicalhippo 3y ago
  This is similar to how ZFS does it, and probably others, so hardly unique. In ZFS they call it "records" or "blocks" rather than "extents", and the default value record size is 128KiB.

j16sdiz 3y ago

Don't hold your breath. This is lkml business as usual. Patches as large as this usually take a few tries to get things into shape.

simcop2387 3y ago
That said, this isn't the first time bcachefs has been through this process. Though previous runs were mostly just an RFC for finding issues and getting input on how things are structured and seeing if there are any difficult to understand/reason about areas. I'm not sure I expect it to end up in 6.6 just because reviews like this do take a long time anyway, but the 6.6 merge window is still a decently long time away from happening. If it doesn't make 6.6 and no major issues are found I could definitely see 6.7 being the one it's in. That'd likely mean it'd be sometime in the late fall early winter (northern hemisphere) which is still great news. I've been looking forward to bcachefs coming in as an alternative to XFS/ext4 on the lower end of features and as an alternative for ZFS for when you don't need or want all of the stuff it has either. It looks like it should shape up to being a very capable middle ground with a lot of good stuff in it.
- danudey 3y ago
  It sounds like one of the biggest issues with merging bcachefs is that the patch is huge and touches systems outside of bcachefs. I suspect if they had taken those external patches and submitted them separately they could have had incremental acceptance (or at least discussion) of those patches and we might be further along, or at least know how far along we actually are.
  - simcop2387 3y ago
    That's my understanding too, and I believe that's what came out of the previous RFC discussions it had before making it to this merge discussion. The parts that go outside the FS itself I believe have been split out better so that it's easier to review and I think can be taken in parts to verify that it doesn't cause issues in those external places.
3y ago

mustache_kimono 3y ago

Read maybe three comments deep in the actual PR[0], and you'll get the impression this is definitely not going to ever be merged. I've said it 1000x -- OpenZFS is already free and open source. Linux kernel developers should just stop punching themselves in face. The ZFS hair shirt is self imposed by semi-religious Linux wackadoos. [0]: https://lore.kernel.org/lkml/20230626214656.hcp4puionmtoloat@moria.home.lan/ https://lore.kernel.org/lkml/20230626214656.hcp4puionmtoloat...

dralley 3y ago
I've read several such LKML threads about bcachefs and while the discussions have been contentious (and Kent could stand to chill out a bit), everyone has been in agreement that they'd like to see bcachefs succeed and be merged. The major technical blockers have been addressed or are being addressed. Bcachefs also has plenty of features which ZFS does not, so I don't see why you're so critical. Oracle could relicense ZFS at any time, or make an explicit statement about its legal status. They have not done so after 10 years.
- gigatexal 3y ago
  yeah i love love love ZFS and use it everywhere and am very excited to see Bcachefs get merged. More legit competition in this space is good. Btrfs doesn't apply, it's not even in the same league.
- mustache_kimono 3y ago
  >I've read several such LKML threads about bcachefs and while the discussions have been contentious (and Kent could stand to chill out a bit), everyone has been in agreement that they'd like to see bcachefs succeed and be merged. Bcachefs also has plenty of features which ZFS does not, so I don't see why you're so critical. Only because I've read this story more than a few times before? See: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=%22bcachefs%22%20merge&sort=byPopularity&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... >Oracle could relicense ZFS at any time, or make an explicit statement about its legal status. They have not done so after 10 years. Any legal difficulty remains from the GPL side, and no suit has been brought in the years upon years of the GPL hair shirts screaming about non-compliance. Unfortunately, the GPL hair shirts don't have much (any) credibility on the subject of what the GPL means. Bring the suit or STFU. I have no patience for armchair legal opinions from people who won't put their money where their mouth is. Stop spreading legal FUD!

Geezus_42 3y ago

ATM machine

simcop2387 3y ago
Kind of sort of mostly? The big difference here is that Bcachefs is a filesystem based originally on the design of Bcache which is a block device caching layer, so it'd be very easy to miss the fs on the end and confuse Bcachefs for Bcache. Because of this, calling it Bcache filesystem also doesn't really work because it's not Bcache anymore either.

bmurphy1976 3y ago

Question: I currently run ZFS via Ubuntu on my home devices. I've had a very positive experience with it. My only (minor) complaint is the extra 30 seconds it adds to my desktop's boot time. It's been fantastic for all my other devices, particularly the snapshotting features and ease of which I can ship the snapshots between the devices. I've read some comments on various forums that seem to hint Ubuntu is moving on from ZFS (see https://discourse.ubuntu.com/t/future-of-zfs-on-ubuntu-desktop/33001 https://discourse.ubuntu.com/t/future-of-zfs-on-ubuntu-deskt... for one example). This is obviously concerning and the lack of support across the whole Linux ecosystem continues to be a nuisance. Realistically how well could bcachefs serve as a long term ZFS replacement? I know very little about it. I've used btrfs in the past, and between my negative experiences and the repeated anecdotes posted all over the internet to this day I'm hesitant to give it another go.

Filligree 3y ago
I tried precisely that, and had it corrupt my data… twice. Bcachefs isn’t ready for prime time. I’m reasonably sure it’s better designed than Btrfs, and it’ll get there, but don’t do it yet. If Ubuntu becomes a problem, NixOS has 100% support for ZFS.
- koverstreet 3y ago
  Were you able to report this? If you can get me enough information (e.g. via bcachefs dump), I'd like to track this down.
  - Filligree 3y ago
    We talked on IRC. :-) I'm planning to do that, but it'll have to be after my vacation; don't want to be running bcachefs on a remote server I can't fix. For those following along, this sort of response is why I'm optimistic overall. And bcachefs was dramatically higher performance than ZFS, although the lack of an ARC is a particular pain point. === It's going to be an interesting bug, I'm pretty sure. It affected two sets of files: The Steam Linux runtime (Solder), and everything in /nix/store, but absolutely nothing else. Actually, I think it's probably a bug in the fsck routine; everything worked perfectly until I rebooted.

snvzz 3y ago

Another option is Dragonfly's HAMMER2[0]. 0. https://www.dragonflybsd.org/hammer/ https://www.dragonflybsd.org/hammer/

viraptor 3y ago
Not on Linux. There weren't any serious attempts to port it as far as I know.
- londons_explore 3y ago
  It's a bit of a shame that filesystems are so OS-specific. It's kinda a shame that there isn't at least some FUSE layer to let you run a filesystem driver from windows/mac/bsd/whatever on linux.
  - yjftsjthsd-h 3y ago
    FWIW FUSE kind of is that layer; a filesystem targeting FUSE should at least be trivially portable across the FOSS unix-likes.
1MachineElf 3y ago
I really wish HAMMER2 had more traction. It's BSD licensing means it has the potential to be supported on Linux, Windows, macOS, etc. (in addition to other BSDs.)

pengaru 3y ago

Nice to see even the graybeards on LKML are as susceptible to bikeshedding as anyone else. Note how much heated debate is happening around how Kent is behaving, rather than the substance of his patchset. The former requires far less effort and knowledge to criticize and attack...

alphazard 3y ago
The real feedback has already happened. Last year Kent sent out the initial patches, and the main feedback from Linus was about the Git history and locking. All the important stuff is sorted out, it's in good shape to go into the kernel. The issue now is there are lots of maintainers who don't like that someone else did something impressive that they couldn't have done, and they are dragging their feet getting the various patches outside `fs/bcachefs` integrated. The design and implementation is basically just Kent, there are others onboarding to help with maintenance. There is a "Wireguard" level of craftsmanship in this project, it's a big deal. A younger Linus, in another time, might have told everyone to cut the bullshit--if you can't comment on the code, shut up, you're probably incompetent--but that's not the way things work anymore.

demizer 3y ago

I've been using it for the last two years and I think it has potential over btrfs, and zfs specifically. I used zfs on my hobby data array for 10 years prior but decided to downsize everything and use bcachefs in the hopes of it one day being in the kernel (unlike zfs). Device management is super easy, but is still missing knobs and dials for the most part. Multi device fault tolerance also needs work as I have had to recreate a 16tb array twice now due to journal corruption on power outages. I'll keep using it and reporting bugs, but I do feel the author is letting some bugs slide to focus on getting the fs in the kernel so that it gets more capable users that can provide patches. I am a sophisticated user, but definitely not a kernel developer.

londons_explore 3y ago
> as I have had to recreate a 16tb array twice now due to journal corruption on power outages. This isn't a good sign... Journalling should never, if bug-free, lead to data corruption or loss even if there is a power outage. And, even if there was some bug, one would expect a robust filesystem to say "ah, there is some data corruption here, so we're gonna run an fsck and recover every single file on the disk except perhaps the one or two that the bug clobbered."
- cmurf 3y ago
  ZFS and Btrfs have demonstrated devices have a variety of transient failures including maintaining write order implied by fsync or fua. This will thwart any filesystem. SSDs do not reliably report UNC read errors when data can't be retrieved. Garbage or zeros are returned instead. There's a reason why ext4 and XFS added journal and metadata checksumming. Storage devices just aren't as reliable at informing the kernel when it suspects the data returned is bad.
  - Dylan16807 3y ago
    Incorrect write order shouldn't thwart a CoW filesystem. It can check at mount time whether the last few commits are fully there.

londons_explore 3y ago

I want a multi-device filesystem, offering (typically) the performance of the best device, yet an upper bound on data loss if one device fails. Eg. I could have an SSD and a slow hard disk. Data is written to the SSD first, and eventually mirrored to the hard disk. If the SSD fails, the hard disk will still be fully consistent up to the point the mirroring process got to. That mirroring would never be allowed to get more than say 5 minutes behind (writes would be throttled if that ever happened). This lets you have performance and resilience while paying less money. The scheme can be extended with erasure codes to say 32 drives for a huge database so that you could for example sustain any 3 device failures not losing any fsynced bytes of data, or up to 6 failures but you might lose up to 30 seconds of data. I think many multi-device filesystems could support this mode, but I don't think any do.

dymk 3y ago
What's are the usecases of a filesystem like that? I can't think of a situation where I care that my data is mirrored, but I'm okay with losing 5 minutes of it.
- londons_explore 3y ago
  Your home desktop PC? I'd be perfectly fine losing 5 minutes of data in case of a disk failure. After all - I can probably recreate it in about 5 minutes. Anyone who does daily backups is effectively saying "I care about my data, but I'm happy to lose up to a days worth if there is a failure".
  - dymk 3y ago
    I suppose I would just throw another SSD in there and mirror with RAID1 (or equivalent), and save myself the headache. SSDs are cheap - A Samsung 980 1TB is $50. Edit: I think you can achieve what you want with current tools, or something pretty close at least, with ZFS. Create a snapshot of your SSD pool every 5 minutes, then stream the delta to an HDD pool in an automated fashion. Delete snapshots more than X hours / days old.

vorticalbox 3y ago

I have never really paid much attention to filesystems on Linux just installed which ever the installer is set to use, with that in mind do we need another filesystem?

deleted 3y ago
[deleted]
danudey 3y ago
bcachefs supports a lot of features that would be useful for even home user desktop systems. For example, the filesystem supports snapshots. This would let you do something like have apt (in Ubuntu) create an automatic snapshot of the filesystem before running an upgrade, so that if the upgrade fails or causes problems you can roll it back trivially. It also supports native compression, saving users tons of disk space. It supports combining multiple devices together for redundancy, and it can checksum all of your data so that you can know if a file was corrupted without it introducing transparent failures into your system. It supports copy-on-write, so that you can make instantaneous "copies" of files or directories which take no extra space unless you change them. So yeah, there are a lot of functionalities that we can enable with newer filesystems. ext4 is pretty great all in all, but there are a lot of modern features that modern hardware and concepts enable.
5e92cb50239222b 3y ago
If you go by features, there aren't many alternatives. Unless I set up a database server, I consider checksums + snapshots + compression + reflinks to be table stakes these days. There is only one mainline filesystem that supports all of that, and it has its share of issues that are unlikely to ever be fixed, and is (probably permanently) tarnished in many users' eyes because it was pushed too early. So the question is, do we really need a second filesystem? (Yes?)
bityard 3y ago
Different file systems are optimized for different workloads and use cases. If you're just installing Linux on a laptop for medium-duty use, then you don't need anything special and can go with whatever general-purpose FS the distribution recommends, usually ext4. (Although I am partial to XFS myself.) You only need to consider more specialized filesystems if you have unique needs. For example, if you are setting up a file server, you might want a filesystem that is good at disk redundancy and data durability. Or you might need a filesystem that can handle billions of small files. Or you might have a huge database that needs random access to large amounts of flash storage. All of these are very different scenarios that require different (competing) things from the filesystems. In this case, bcachefs _seems_ to be promising many of the features of ZFS but with a better design and less overhead, potentially bringing these features to general-purpose workloads as well. (ZFS is fairly resource-intensive: in order to make full use of its features, you need a dedicated box for it, or really beef up the hardware on the application server that it will run on.)

deleted 3y ago

[deleted]

dodox 3y ago

Bcachefs does not yet implement scrubs. Corrupted data can be detected, reading instead from a copy (if available). But the problems get neither reported nor fixed. So does not seem production ready yet... https://github.com/koverstreet/bcachefs/issues/146 https://github.com/koverstreet/bcachefs/issues/146

Roark66 3y ago

Could someone explain what is the expected main benefit over btrfs? I've been running btrfs on all my "fast&busy" disks for quite a while, including hourly snapshots etc and I've been very happy with it.