6 ms·

>not even considering some hostile emails that I recently received from the upstream developer or his public rants on lkml and reddit It feels like whenever th

>not even considering some hostile emails that I recently received from the upstream developer or his public rants on lkml and reddit It feels like whenever the author of bcachefs comes up, it's always because of some drama. Just the other day he clashed with Linus Torvalds: https://lore.kernel.org/lkml/CAHk-=wj1Oo9-g-yuwWuHQZU8v=VAsBceWCRLhWxy7_-QnSa1Ng@mail.gmail.com/ https://lore.kernel.org/lkml/CAHk-=wj1Oo9-g-yuwWuHQZU8v=VAsB... My reading is that he's very passionate, so he wants to "move fast and break things" and doesn't get why the others aren't necessarily very happy about it.

sp1rit 2y ago
[flagged]
jt2190 2y ago
Is bcachefs-tools going into the mainline distros or into something that’s meant to be less stable and experimental? Linus makes it sound like there’s a more appropriate place for this work. Edit: Reading through the thread, it seems like there is a claim of rigorous but effectively private testing. Without the ability to audit those results easily it’s causing a lot of worry.
- ralferoo 2y ago
  I don't think Linus is particularly concerned about bcachefs-tools, and whether a particular distributions ships with it or not isn't a concern for the kernel. Presumably though, distributions that don't ship the tools may also want to disable it in the kernel, although I'd imagine they'd leave it alone if it was previously supported. Linus' complaint was about adding feature work into a supposed minor bug fix, especially because (going from Linus' comments) they were essentially refactors of major systems that impacted other areas of the kernel.
ralferoo 2y ago
It's very clear from that thread that he doesn't understand the purpose of the stable branch. It doesn't mean "stable" as in "the best possible experience", it means it as in "this code has been tested for a long period of time with no serious defects found" so that when the stable branch is promoted to release, everything has undergone a long testing period by a broad user base. If there is a defect found, the change to a stable branch should literally be the minimal code change to fix the reported issue. Ideally, if it's a newly introduced issue (i.e. since being on the stable branch), the problematic code reverted and a different fix to the original defect applied instead (or left if it's deemed less of an issue than taking another speculative fix). Anything that requires a re-organisation of code, by definition, isn't a minimal fix. Maybe it's the correct long-term solution, but that can be done on the unstable branch, but for the stable branch, the best fix is the simplest work around. If there isn't a simple work around, the best fix is to revert everything back to the previous stable version and keep iterating on the unstable branch. The guy even admits it as well with his repeated "please don't actually use this in production" style messages - it's hard to give a greater indication than this that the code isn't yet ready for stable. I can understand why from his perspective he wants his changes in the hands of users as soon as possible - it's something he's poured his heart and soul and he strongly believes it will improve his users' experience. It's also the case that he is happy running the very latest and probably has more confidence in it that an older version. The rational choice from his perspective is to always use the latest code. But, discounting the extremely unlikely situation that his code is entirely bug free, that just means he hasn't yet found the next serious bug. If a big code change is rushed out into the stable branch, it just increases the likelihood that any serious bug won't have the time it needs in testing to have the confidence that's the branch is suitable for promotion to release.
- _ph_ 2y ago
  It seems to be a difficult situation: he has bug fixes against the version in the stable kernel for bugs which haven't been reported. I can see both perspectives: on stable you don't want to do development, but also you want all bugfixes you can get. I can also see the point of Linus, who wants just to add bug fixes and to minimize the risk of introducing new bugs. Considering that Kent himself warns against general use right now, I don't quite see the urgency to get the bug fixes out - in my understanding Linus would happily merge them in the next development kernel. And whoever is set to to run bcachefs right now, might also be happy to run a dev kernel.
  - ralferoo 2y ago
    I agree. If the author himself is telling people it's not ready for production, it doesn't really matter what bugs this code has, unless it affects other subsystems or is a dangerous regression from the previous stable release. If the bug it's fixing was already in the current release branch, wasn't noticed before and has only shown up now late in the stable branch lifetime, then it definitely doesn't seem like something that needs an urgent fix.
- Arch-TK 2y ago
  He is not submitting changes for stable. He is submitting non-regression fixes after the merge window. It's clear he understands the rules and the reasons for them but feels like his own internal development process is equivalent at reducing the chance of major regressions introduced in such a PR such that he can simply persuade Linus to let things go through anyway. Whether this internal process gives him a pass for getting his non-regression fixes in after the merge window is at the end of the day for Linus to decide. And Linus is finally erring on the side of "Please just do what everyone else is doing" rather than "Okay, fine Kent, just this once". I would say it's ironic to start a comment saying: "It's very clear from that thread that he doesn't understand the purpose of the stable branch" when it's "very clear" from your opening paragraph that you don't understand the thread.
  - ralferoo 2y ago
ants_everywhere 2y ago
Hey at least it's not the worst behavior we've seen from a Linux file system creator... I thought Carl Thompson's response was very good and constructive: https://lore.kernel.org/lkml/1816164937.417.1724473375169@mail.carlthompson.net/ https://lore.kernel.org/lkml/1816164937.417.1724473375169@ma... What I don't understand is that IIUC Kent has his development git history well broken up into small tight commits. But he seems to be sending the Linux maintainers patches that are much larger than they want. I don't get why he doesn't take the feedback and work with them to send smaller patches. EDIT: The culture at Google (where Kent used to work) was small patches, although that did vary by team. At Google you have fleet-wide control and can roll back changes that looked good in testing but worked out poorly in production. You can't do that across all organizations or people who have installed bcachefs. Carl pointed out that Kent seemed to be missing some social aspects, but I feel like he's also not fully appreciating the technical aspects behind why the process is the way it is.
- koverstreet 2y ago
  Honesty, I think I just presented that pull request badly. I included the rcu_pending and vfs inode rhashtable conversion because I was getting user reports that it fixed issues that were seriously affecting system usability, and because they were algorithmically simple and well tested. Back in the day, on multiple occasions Linus and others were rewriting core mm code in RC kernels; bcachefs is still experimental, so stuff like this should still be somewhat expected.
  - ants_everywhere 2y ago
    Yeah I see where you're coming from. By the way, I only heard of bcachefs yesterday and I watched a great video where you were presenting about it. I'm excited about the file system and it's super cool to hear from you!
    - koverstreet 2y ago
      Likewise!
  - 2y ago
ajb 2y ago
> he wants to "move fast and break things" That's not how I read that thread. This is just about where the diligence happens, not that it can be avoided; and exactly how small a fix is mergable in the fixes phase of kernel development. I don't see that thread as being particularly angry either. There have been ones where both of them have definitely lost their cool; here they are having a (for them) calm disagreement. Linus is just not going to merge this one until the next development phase, which is fine. There have been arguments involving this developer that do raise questions; I just don't see this as one of them.
htpart 2y ago
That's a normal LKML conversation. Nowhere do I see actual bugs pointed out, in fact the only testimony by Carl said that bcachefs has been quite stable so far. This is just about following procedure. There are people who follow procedure and introduce many bugs, and people who don't but write perfect software. The bcachefs author cautiously marks bcachefs as unstable, which is normal for file systems. The only issue here is that the patch touched other areas, but in the kernel development model Linus is free not to pull it.
rubiquity 2y ago
Let’s not misrepresent Kent over a single incident of sending too much after a merge window. He’s extremely helpful and nice in every interaction I’ve ever read.
qalmakka 2y ago
My 2 cents: these are the types of people that actually get the job done. All good software in my experience starts thanks to overachieving human representations of Pareto's law - people that can do alone in months what a team of average-skilled developers do in years. In this industry it's very, very easy to run in circle, keeping doing stuff over stuff without realising you are in a bikeshedding loop, you're overengineering or simply wasting time. We need people that want to just push forward and assume the responsibility of anything that breaks, otherwise I'm sure that in 30 years we'd all still be using the same stuff we've always used because it's just human nature to stick with what we know works, quirks and all.
ris 2y ago
Great and now the top thread on the HN discussion is about that drama only tangentially referenced in the article.

sp1rit 2y ago

[flagged]

jt2190 2y ago

Is bcachefs-tools going into the mainline distros or into something that’s meant to be less stable and experimental? Linus makes it sound like there’s a more appropriate place for this work. Edit: Reading through the thread, it seems like there is a claim of rigorous but effectively private testing. Without the ability to audit those results easily it’s causing a lot of worry.

ralferoo 2y ago
I don't think Linus is particularly concerned about bcachefs-tools, and whether a particular distributions ships with it or not isn't a concern for the kernel. Presumably though, distributions that don't ship the tools may also want to disable it in the kernel, although I'd imagine they'd leave it alone if it was previously supported. Linus' complaint was about adding feature work into a supposed minor bug fix, especially because (going from Linus' comments) they were essentially refactors of major systems that impacted other areas of the kernel.

ralferoo 2y ago

It's very clear from that thread that he doesn't understand the purpose of the stable branch. It doesn't mean "stable" as in "the best possible experience", it means it as in "this code has been tested for a long period of time with no serious defects found" so that when the stable branch is promoted to release, everything has undergone a long testing period by a broad user base. If there is a defect found, the change to a stable branch should literally be the minimal code change to fix the reported issue. Ideally, if it's a newly introduced issue (i.e. since being on the stable branch), the problematic code reverted and a different fix to the original defect applied instead (or left if it's deemed less of an issue than taking another speculative fix). Anything that requires a re-organisation of code, by definition, isn't a minimal fix. Maybe it's the correct long-term solution, but that can be done on the unstable branch, but for the stable branch, the best fix is the simplest work around. If there isn't a simple work around, the best fix is to revert everything back to the previous stable version and keep iterating on the unstable branch. The guy even admits it as well with his repeated "please don't actually use this in production" style messages - it's hard to give a greater indication than this that the code isn't yet ready for stable. I can understand why from his perspective he wants his changes in the hands of users as soon as possible - it's something he's poured his heart and soul and he strongly believes it will improve his users' experience. It's also the case that he is happy running the very latest and probably has more confidence in it that an older version. The rational choice from his perspective is to always use the latest code. But, discounting the extremely unlikely situation that his code is entirely bug free, that just means he hasn't yet found the next serious bug. If a big code change is rushed out into the stable branch, it just increases the likelihood that any serious bug won't have the time it needs in testing to have the confidence that's the branch is suitable for promotion to release.

_ph_ 2y ago
It seems to be a difficult situation: he has bug fixes against the version in the stable kernel for bugs which haven't been reported. I can see both perspectives: on stable you don't want to do development, but also you want all bugfixes you can get. I can also see the point of Linus, who wants just to add bug fixes and to minimize the risk of introducing new bugs. Considering that Kent himself warns against general use right now, I don't quite see the urgency to get the bug fixes out - in my understanding Linus would happily merge them in the next development kernel. And whoever is set to to run bcachefs right now, might also be happy to run a dev kernel.
- ralferoo 2y ago
  I agree. If the author himself is telling people it's not ready for production, it doesn't really matter what bugs this code has, unless it affects other subsystems or is a dangerous regression from the previous stable release. If the bug it's fixing was already in the current release branch, wasn't noticed before and has only shown up now late in the stable branch lifetime, then it definitely doesn't seem like something that needs an urgent fix.
Arch-TK 2y ago
He is not submitting changes for stable. He is submitting non-regression fixes after the merge window. It's clear he understands the rules and the reasons for them but feels like his own internal development process is equivalent at reducing the chance of major regressions introduced in such a PR such that he can simply persuade Linus to let things go through anyway. Whether this internal process gives him a pass for getting his non-regression fixes in after the merge window is at the end of the day for Linus to decide. And Linus is finally erring on the side of "Please just do what everyone else is doing" rather than "Okay, fine Kent, just this once". I would say it's ironic to start a comment saying: "It's very clear from that thread that he doesn't understand the purpose of the stable branch" when it's "very clear" from your opening paragraph that you don't understand the thread.
- ralferoo 2y ago

ants_everywhere 2y ago

Hey at least it's not the worst behavior we've seen from a Linux file system creator... I thought Carl Thompson's response was very good and constructive: https://lore.kernel.org/lkml/1816164937.417.1724473375169@mail.carlthompson.net/ https://lore.kernel.org/lkml/1816164937.417.1724473375169@ma... What I don't understand is that IIUC Kent has his development git history well broken up into small tight commits. But he seems to be sending the Linux maintainers patches that are much larger than they want. I don't get why he doesn't take the feedback and work with them to send smaller patches. EDIT: The culture at Google (where Kent used to work) was small patches, although that did vary by team. At Google you have fleet-wide control and can roll back changes that looked good in testing but worked out poorly in production. You can't do that across all organizations or people who have installed bcachefs. Carl pointed out that Kent seemed to be missing some social aspects, but I feel like he's also not fully appreciating the technical aspects behind why the process is the way it is.

koverstreet 2y ago
Honesty, I think I just presented that pull request badly. I included the rcu_pending and vfs inode rhashtable conversion because I was getting user reports that it fixed issues that were seriously affecting system usability, and because they were algorithmically simple and well tested. Back in the day, on multiple occasions Linus and others were rewriting core mm code in RC kernels; bcachefs is still experimental, so stuff like this should still be somewhat expected.
- ants_everywhere 2y ago
  Yeah I see where you're coming from. By the way, I only heard of bcachefs yesterday and I watched a great video where you were presenting about it. I'm excited about the file system and it's super cool to hear from you!
  - koverstreet 2y ago
    Likewise!
- 2y ago

ajb 2y ago

> he wants to "move fast and break things" That's not how I read that thread. This is just about where the diligence happens, not that it can be avoided; and exactly how small a fix is mergable in the fixes phase of kernel development. I don't see that thread as being particularly angry either. There have been ones where both of them have definitely lost their cool; here they are having a (for them) calm disagreement. Linus is just not going to merge this one until the next development phase, which is fine. There have been arguments involving this developer that do raise questions; I just don't see this as one of them.

htpart 2y ago

That's a normal LKML conversation. Nowhere do I see actual bugs pointed out, in fact the only testimony by Carl said that bcachefs has been quite stable so far. This is just about following procedure. There are people who follow procedure and introduce many bugs, and people who don't but write perfect software. The bcachefs author cautiously marks bcachefs as unstable, which is normal for file systems. The only issue here is that the patch touched other areas, but in the kernel development model Linus is free not to pull it.

rubiquity 2y ago

Let’s not misrepresent Kent over a single incident of sending too much after a merge window. He’s extremely helpful and nice in every interaction I’ve ever read.

qalmakka 2y ago

My 2 cents: these are the types of people that actually get the job done. All good software in my experience starts thanks to overachieving human representations of Pareto's law - people that can do alone in months what a team of average-skilled developers do in years. In this industry it's very, very easy to run in circle, keeping doing stuff over stuff without realising you are in a bikeshedding loop, you're overengineering or simply wasting time. We need people that want to just push forward and assume the responsibility of anything that breaks, otherwise I'm sure that in 30 years we'd all still be using the same stuff we've always used because it's just human nature to stick with what we know works, quirks and all.

ris 2y ago

Great and now the top thread on the HN discussion is about that drama only tangentially referenced in the article.