3 ms·

> Technically speaking, bcachefs has been merged into the Linux Kernel - that makes your initial assertion wrong. bcachefs doesn't implement its erasure coding

> Technically speaking, bcachefs has been merged into the Linux Kernel - that makes your initial assertion wrong. bcachefs doesn't implement its erasure coding/RAID yet? Doesn't implement send/receive. Doesn't implement scrub/fsck. See: https://bcachefs.org/Roadmap https://bcachefs.org/Roadmap, https://bcachefs.org/Wishlist/ https://bcachefs.org/Wishlist/ btrfs is still more of a legit competitor to ZFS these days and it isn't close to touching ZFS where it matters. If the perpetually half-finished bcachefs and btrfs are the "answer" to ZFS that seems like too little, too late to me.

deleted 2y ago
[deleted]
koverstreet 2y ago
Erasure coding is almost done; all that's missing is some of the device evacuate and reconstruct paths, and people have been testing it and giving positive feedback (especially w.r.t. performance). It most definitely does have fsck and has since the beginning, and it's a much more robust and dependable fsck than btrfs's. Scrub isn't quite done - I actually was going to have it ready for this upcoming merge window except for a nasty bout of salmonella :) Send/recv is a long ways off, there might be some low level database improvements needed before that lands. Short term (next year or two) priorities are finishing off online fsck, more scalability work (upcoming version for this merge window will do 50PB, but now we need to up the limit on number of drives), and quashing bugs.
- ryao 2y ago
  Hearing that it is missing some code for reconstruction makes it sound like it is missing something fairly important. The original purpose of parity RAID is to support reconstruction.
  - koverstreet 2y ago
    We can do reconstruct reads, what's missing is the code to rewrite missing blocks in a stripe after a drive dies. In general, due to the scope of the project, I've been prioritizing the functionality that's needed to validate the design and the parts that are needed for getting the relationships between different components correct. e.g. recently I've been doing a bunch of work on backpointers scalability, and that plus scrub are leading to more back and forth iteration on minor interactions with erasure coding. So: erasure coding is complete enough to know that it works and for people to torture test it, but yes you shouldn't be running it in production yet (and it's explicitly marked as such). What's remaining is trivial but slightly tedious stuff that's outside the critical path of the rest of the design. Some of the code I've been writing for scrub is turning out to also be what we want for reconstruct, so maybe we'll get there sooner rather than later...
- BSDobelix 2y ago
  >except for a nasty bout of salmonella Did the Linux Foundation send you some "free" sushi? ;) However keep the good work rolling, super happy about a good, usable and modern Filesystem native to Linux.
- pdimitar 2y ago
  FYI: the main reason I gave up on bcachefs is that I can't use devices with native 16K blocks. Hope that's coming this year. I have a bunch of old HDDs and SSDs and I could very easily assemble a spare storage server with about 4TB capacity. Already tested bcachefs with most of the drives and it performed very well. Also lack of ability to reconstruct seems like another worrying omission.
  - koverstreet 2y ago
    I wasn't aware there were actual users needing bs > ps yet. Cool :) That should be a completely trivial for bcachefs to support, it'll mostly just be a matter of finding or writing the tests.
    - pdimitar 2y ago
      Seriously? But... NVMe drives! I stopped testing because I only have one spare NVMe and couldn't use it with bcachefs. If you or others can get it done I'm absolutely starting to use bcachefs the month after. I do need fast storage servers in my home office.
      - ryao 2y ago
        You can do this on ZFS today with `zpool create -o ashift=14 ...`.
        pdimitar 2y ago
        Yeah I know, thanks. But ZFS still mostly requires drives with the same sizes. My main NAS is like that but I can't expand it even though I want to, with drives of different sizes I have lying around, and I am not keen on spending for new HDDs right now. So I thought I'll make a secondary NAS with bcachefs and all the spare drives I have. As for ZFS, I'll be buying some extra drives later this year and will make use of direct_io so I can use another NVMe spare for faster access.
        2y ago
- mafuy 2y ago
  Thank you, looking forward to it!

deleted 2y ago

[deleted]

koverstreet 2y ago

Erasure coding is almost done; all that's missing is some of the device evacuate and reconstruct paths, and people have been testing it and giving positive feedback (especially w.r.t. performance). It most definitely does have fsck and has since the beginning, and it's a much more robust and dependable fsck than btrfs's. Scrub isn't quite done - I actually was going to have it ready for this upcoming merge window except for a nasty bout of salmonella :) Send/recv is a long ways off, there might be some low level database improvements needed before that lands. Short term (next year or two) priorities are finishing off online fsck, more scalability work (upcoming version for this merge window will do 50PB, but now we need to up the limit on number of drives), and quashing bugs.

ryao 2y ago
Hearing that it is missing some code for reconstruction makes it sound like it is missing something fairly important. The original purpose of parity RAID is to support reconstruction.
- koverstreet 2y ago
  We can do reconstruct reads, what's missing is the code to rewrite missing blocks in a stripe after a drive dies. In general, due to the scope of the project, I've been prioritizing the functionality that's needed to validate the design and the parts that are needed for getting the relationships between different components correct. e.g. recently I've been doing a bunch of work on backpointers scalability, and that plus scrub are leading to more back and forth iteration on minor interactions with erasure coding. So: erasure coding is complete enough to know that it works and for people to torture test it, but yes you shouldn't be running it in production yet (and it's explicitly marked as such). What's remaining is trivial but slightly tedious stuff that's outside the critical path of the rest of the design. Some of the code I've been writing for scrub is turning out to also be what we want for reconstruct, so maybe we'll get there sooner rather than later...
BSDobelix 2y ago
>except for a nasty bout of salmonella Did the Linux Foundation send you some "free" sushi? ;) However keep the good work rolling, super happy about a good, usable and modern Filesystem native to Linux.
pdimitar 2y ago
FYI: the main reason I gave up on bcachefs is that I can't use devices with native 16K blocks. Hope that's coming this year. I have a bunch of old HDDs and SSDs and I could very easily assemble a spare storage server with about 4TB capacity. Already tested bcachefs with most of the drives and it performed very well. Also lack of ability to reconstruct seems like another worrying omission.
- koverstreet 2y ago
  I wasn't aware there were actual users needing bs > ps yet. Cool :) That should be a completely trivial for bcachefs to support, it'll mostly just be a matter of finding or writing the tests.
  - pdimitar 2y ago
    Seriously? But... NVMe drives! I stopped testing because I only have one spare NVMe and couldn't use it with bcachefs. If you or others can get it done I'm absolutely starting to use bcachefs the month after. I do need fast storage servers in my home office.
    - ryao 2y ago
      You can do this on ZFS today with `zpool create -o ashift=14 ...`.
      - pdimitar 2y ago
        Yeah I know, thanks. But ZFS still mostly requires drives with the same sizes. My main NAS is like that but I can't expand it even though I want to, with drives of different sizes I have lying around, and I am not keen on spending for new HDDs right now. So I thought I'll make a secondary NAS with bcachefs and all the spare drives I have. As for ZFS, I'll be buying some extra drives later this year and will make use of direct_io so I can use another NVMe spare for faster access.
        2y ago
mafuy 2y ago
Thank you, looking forward to it!