3 ms·

Bcachefs is experimental, and last I heard, the authors hope to be able to declare it not experimental in ~6 months. To me, that's 'try on a cache server/build

by ajb 2y ago

Bcachefs is experimental, and last I heard, the authors hope to be able to declare it not experimental in ~6 months. To me, that's 'try on a cache server/build server' territory, not on anything where you even think about backing it up. Kent Overstreet does have a problem working with people; I can well believe that interactions around bugs were painful. He likely he should try to hire someone else to deal with users and distributions and use his limited patience to deal with the kernel maintainers. But it sounds like the OP was a bit naive about what it means to run an experimental FS.

AceJohnny2 2y ago
> last I heard, the authors hope to be able to declare it not experimental in ~6 months How long have they been saying that? I feel like that's been the case for years.
- ajb 2y ago
  It's only been in the upstream kernel 6 months or so? I know the FS has actually been around longer, but I've not heard claims about it's stability before that. It's only now that's it's getting a lot more testing
  - pixelesque 2y ago
    It was merged in 6.7, 12 months ago...
  - aragilar 2y ago
    Searching bcachefs on LWN (https://lwn.net/Search/DoTextSearch https://lwn.net/Search/DoTextSearch) does make for a somewhat interesting read e.g. https://lwn.net/Articles/717379/ https://lwn.net/Articles/717379/ (from 2017) which suggests the disc format would have one last change (c.f. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=2622f2904170 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... which is for the upcoming 6.14 release, which notes the format will have at least one more changes after the one included), and the numerous articles on its upstreaming (e.g. https://lwn.net/Articles/755276/ https://lwn.net/Articles/755276/ from 2018), and the stability claims do seem a bit, well, optimistic?
    - ajb 2y ago
      Given that it's a DB under the hood, we really need some new terminology to understand what's happening when there's a disc format change. I think that some of these format changes are in fact schema changes, which makes them a bit less risky (but not entirely risk free). I get the impression that after it is considered stable there may still be schema changes.
      - koverstreet 2y ago
        Yes, the low level format for btree nodes/btree keys isn't changing, and the fact that it's a database does ease a lot of backwards and forwards compatibility issues - for example, most key types can have new fields appended and old versions will just ignore the new field. But schema changes can still be pretty impactful. The big on disk format changes were: - accounting rewrite in 6.11, which switched from a high performance, but complex and difficult to extend mechanism where accounting was stored in memory with percpu counters and then bundled up in each journal entry, to storing them in btree keys and journalling them as deltas. The runtime code was so vastly different that it just wasn't possible to keep writing out the old accounting without retaining all of the old code side by side with the new code - it would have been really gross. - backpointers change in 6.14, for fsck scalability improvements This one would have been possible to do without rewriting all the backpointers, but that still would've had severe downsides. The main option would've meant permanently inflating the size of the backpointers btree by ~20%, IIRC, and backpointers are almost half of all metadata on a typical filesystem. There were ways to get around that, but they would've added a lot of complexity that we would've had to carry around forever. But those are exactly the sorts of things the experimental phase is for - getting it on more and bigger systems and seeing what shakes out and finding out what we still need.
        Momi_V 2y ago
        Honestly the ability to do those major changes in place are pretty exciting and might be a draw towards BcacheFs for me. As long as it doesn't happen too often (with the experimental phase as an exception) it's really nice to be able to benefit from major features and improvements. I really dislike the attitude some FS devs have that if you want some improvement made after you setup a system you should just reformat and restore from backup. Sure a 5 hour metadata rewrite is disruptive, but it's nothing compared to a 5 day backup, reprovision, restore dance. The former can run over night on a weekend, the latter takes basically the entire work week. Therefore it won't get done until all the infrastructure is EOL, and replaced with something else 3-7 years down the line.
gkmcd 2y ago
FWIW, I've done a few bcachefs bug reports and thought Kent's responses were great.
- ajb 2y ago
  I should not have implied that every bug report would be a bad interaction - and to be fair where I've seen it degenerate in the mailing list, it's often the other person wasn't exactly civil to start with (which unfortunately includes Linus Torvalds). It's just that Kent seems to feel the need to fight back.
  - jpgvm 2y ago
    A file system is an emotional investment of a decade+ so I can understand Kent's desire to defend it the way he does. That said the kernel is the bigger picture, if he wants it to be successful and remain in mainline he needs to adapt to how Linus expects maintainers to operate. That probably means delaying getting fixes to mainline users until he can show they have been adequately tested. Until that trust is built bcachefs is always a few bad cycles away from being dropped from mainline.
    - koverstreet 2y ago
      Linus can be a bit of a dick sometimes (and hey, so can I), but he's not prone to being rash or a petty tyrant. He just rides people a bit hard. Following process to the letter isn't what we should be doing right now; this isn't the time for slowing down to cross every t and dot every i, this is the time for getting fixes pushed out quickly so we can move on to the next thing. But things have also been stabilizing dramatically, it's already looking like I'll have a lot less out of merge window stuff to send this cycle (there wasn't much that I would've sent last cycle, either). So things should be pretty well calmed down.

by ajb 2y ago

AceJohnny2 2y ago

> last I heard, the authors hope to be able to declare it not experimental in ~6 months How long have they been saying that? I feel like that's been the case for years.

ajb 2y ago
It's only been in the upstream kernel 6 months or so? I know the FS has actually been around longer, but I've not heard claims about it's stability before that. It's only now that's it's getting a lot more testing
- pixelesque 2y ago
  It was merged in 6.7, 12 months ago...
- aragilar 2y ago
  Searching bcachefs on LWN (https://lwn.net/Search/DoTextSearch https://lwn.net/Search/DoTextSearch) does make for a somewhat interesting read e.g. https://lwn.net/Articles/717379/ https://lwn.net/Articles/717379/ (from 2017) which suggests the disc format would have one last change (c.f. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=2622f2904170 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... which is for the upcoming 6.14 release, which notes the format will have at least one more changes after the one included), and the numerous articles on its upstreaming (e.g. https://lwn.net/Articles/755276/ https://lwn.net/Articles/755276/ from 2018), and the stability claims do seem a bit, well, optimistic?
  - ajb 2y ago
    Given that it's a DB under the hood, we really need some new terminology to understand what's happening when there's a disc format change. I think that some of these format changes are in fact schema changes, which makes them a bit less risky (but not entirely risk free). I get the impression that after it is considered stable there may still be schema changes.
    - koverstreet 2y ago
      Yes, the low level format for btree nodes/btree keys isn't changing, and the fact that it's a database does ease a lot of backwards and forwards compatibility issues - for example, most key types can have new fields appended and old versions will just ignore the new field. But schema changes can still be pretty impactful. The big on disk format changes were: - accounting rewrite in 6.11, which switched from a high performance, but complex and difficult to extend mechanism where accounting was stored in memory with percpu counters and then bundled up in each journal entry, to storing them in btree keys and journalling them as deltas. The runtime code was so vastly different that it just wasn't possible to keep writing out the old accounting without retaining all of the old code side by side with the new code - it would have been really gross. - backpointers change in 6.14, for fsck scalability improvements This one would have been possible to do without rewriting all the backpointers, but that still would've had severe downsides. The main option would've meant permanently inflating the size of the backpointers btree by ~20%, IIRC, and backpointers are almost half of all metadata on a typical filesystem. There were ways to get around that, but they would've added a lot of complexity that we would've had to carry around forever. But those are exactly the sorts of things the experimental phase is for - getting it on more and bigger systems and seeing what shakes out and finding out what we still need.
      - Momi_V 2y ago
        Honestly the ability to do those major changes in place are pretty exciting and might be a draw towards BcacheFs for me. As long as it doesn't happen too often (with the experimental phase as an exception) it's really nice to be able to benefit from major features and improvements. I really dislike the attitude some FS devs have that if you want some improvement made after you setup a system you should just reformat and restore from backup. Sure a 5 hour metadata rewrite is disruptive, but it's nothing compared to a 5 day backup, reprovision, restore dance. The former can run over night on a weekend, the latter takes basically the entire work week. Therefore it won't get done until all the infrastructure is EOL, and replaced with something else 3-7 years down the line.

gkmcd 2y ago

FWIW, I've done a few bcachefs bug reports and thought Kent's responses were great.

ajb 2y ago
I should not have implied that every bug report would be a bad interaction - and to be fair where I've seen it degenerate in the mailing list, it's often the other person wasn't exactly civil to start with (which unfortunately includes Linus Torvalds). It's just that Kent seems to feel the need to fight back.
- jpgvm 2y ago
  A file system is an emotional investment of a decade+ so I can understand Kent's desire to defend it the way he does. That said the kernel is the bigger picture, if he wants it to be successful and remain in mainline he needs to adapt to how Linus expects maintainers to operate. That probably means delaying getting fixes to mainline users until he can show they have been adequately tested. Until that trust is built bcachefs is always a few bad cycles away from being dropped from mainline.
  - koverstreet 2y ago
    Linus can be a bit of a dick sometimes (and hey, so can I), but he's not prone to being rash or a petty tyrant. He just rides people a bit hard. Following process to the letter isn't what we should be doing right now; this isn't the time for slowing down to cross every t and dot every i, this is the time for getting fixes pushed out quickly so we can move on to the next thing. But things have also been stabilizing dramatically, it's already looking like I'll have a lot less out of merge window stuff to send this cycle (there wasn't much that I would've sent last cycle, either). So things should be pretty well calmed down.