6 ms·

So if I understand correctly, Linus & co say no maintainer is forced to write or accept Rust code in their little piece of the kernel. But maintainers claim th

• braiamp 2y ago
  No, Linus is saying that to have a say, you have to stake something. You can't veto something and at the same time not engaging with it in any other constructive way. You either ignore it totally, or you don't and become part of the process. In this specific case, you do not get to decide who the client of your code/subsystem is, and the only way you have a say about it is by offering actual feedback about it.
  • IshKebab 2y ago
    I don't think that's true though, given the Linux development model where all internal APIs are private and unstable and can be changed at any time. It means there is a two-way dependency between API providers and users - because it isn't a public API you are allowed to change it, but you have to fix all the places it is used too. If those places now include Rust then it would require you to learn Rust (or depend on other people). Really they just need to make a decision at this point. Either Rust is allowed and maintainers are going to have to learn it, or there's no point and the Rust people are going to have to fork Linux. I guess there is a third option: a stable driver ABI. But they've avoided that at all costs for years so I don't see it happening. And besides, Rust is going to be used for more than just drivers eventually.
    • db48x 2y ago
      > Really they just need to make a decision at this point. They already _have_ made this decision. Why do people keep misunderstanding this. While ordinarily if you change the API you are required to change all of the callers of that API to match, in this one special case the maintainer of a module can declare that they don’t want to touch the Rust code and that if they change the API then they are not going to change the Rust code that calls it. In that case the Rust for Linux team will step in and make whatever changes to the Rust code are needed.
      • rincebrain 2y ago
        People keep misunderstanding it because the people who are shouting about how Rust is a terrible thing to have in the kernel keep claiming the situation is not that way. I don't do mainline Linux dev regularly, I can't say whether the claim that Christoph made that the process doesn't actually work that way in practice is true or not without doing a lot of archaeology, but that's why there's confusion around it - because some people keep insisting that it doesn't work that way in reality. (At a 500 mile view, it seems like Linus going around making very explicit statements shutting down claims of that uncertainty a bunch would do wonders for clarity, but that assumes good faith on a lot of people's parts, and I also think Linus has learned not to be too visible in his proclamations about anything if he can avoid it...)
        • db48x 2y ago
          > […] it seems like Linus going around making very explicit statements shutting down claims of that uncertainty a bunch would do wonders for clarity […] Did you read the message? This is Linus making a very explicit statement shutting down the claim that there is uncertainty.
          • rincebrain 2y ago
            I did, yes. My remark is that the complaint that kept coming up from both proponents and detractors of the R4L effort is that Linus said one thing in writing and the actual outcomes appear to be substantially different, and at the point where several people who have contributed for years to Linux have left over the friction involved, I don't think "he should have intervened sooner" is a novel take.
            • db48x 2y ago
              I agree that he should have stepped in immediately, but that is not at all the content of your prior messages.
      • adrian_b 2y ago
        I do not believe that it will be possible for "the Rust for Linux team to step in and make whatever changes to the Rust code are needed", because that is much easier said than done. With this rule that the Rust developers must maintain their own code, they are in the same situation like the maintainers of out-of-kernel device drivers. I have maintained out-of-kernel Linux device drivers, so I know that this is a huge PITA. Almost every new Linux kernel minor version was breaking the device drivers, due to interface changes like: moving definitions from one Linux kernel header to another, adding or deleting parameters to functions, adding or deleting members to structures. It is not enough to say that the Rust developers must maintain their own code, but it is also necessary for the responsibilities of the maintainers of the kernel subsystems written in C to be changed. Whoever changes an interface should also have the obligation to write a concise migration guide in which to explain what must be done to convert the invocations of the old interfaces into invocations of the new interfaces. Only in this case the Rust developers would be able to maintain their code. Otherwise, they would have to bombard the C developers with questions after each change, which I assume is what Christoph Hellwig fears. Such a requirement will increase the workload for the C developers, so I am not surprised that they are reluctant to accept users who will maintain their own code. I have not checked recently the kernel documentation, but I doubt that anything has changed. When I was maintaining device drivers, every time when they became broken the only thing that could be done was to scan the kernel E-mail lists, to find who and why has changed the interface. This provided some context about the purpose of the changes, but there has never been any information about how to migrate the existing code to use the new interfaces. When function arguments or structure members were deleted, it was simpler, one could delete them also in the device drivers hoping that they will continue to work. That did not work always, because sometimes additional new interfaces had to be invoked, to compensate for the simplification of the old interfaces. Much worse was when function arguments or structure members were added, because only seldom it was possible to guess what values should be put in them. In many cases the only solution was to waste a lot of time with reading the source for the entire subsystem whose interface had changed, in order to understand which were the effects of the interface changes. All this was simple, but very tedious. What could have been done in a few minutes, had a migration guide existed, could require many hours of work in the absence of adequate documentation. Unless the requirements for the documentation of the internal interfaces change, this is exactly what the Rust developers will also have to do, with the additional difficulty that they may be not very proficient at reading the kernel C language sources. If there would be a clear requirement for the C maintainers to write a migration guide whenever they change interfaces, that would help not only the maintainers of Rust Linux device drivers, but also the maintainers of any out-of-kernel Linux device drivers. However, there have been some Linux developers who not only did not make easy the maintaining of out-of-kernel device drivers, but who have made efforts to make as difficult as possible the maintaining of out-of-kernel device drivers, apparently due to some misguided belief that this will hurt some evil corporations. In my experience, most out-of-kernel device drivers are for obsolete devices or for devices with very few users, whose maintenance is not sponsored by any corporation. So this behavior of actively making difficult the maintenance of out-of-kernel device drivers hurts mostly hobbyists, or at most small businesses, because they do not have the resources to push device drivers into the official tree.
    • rincebrain 2y ago
      The argument being made was that the nominal rule is that the Rust side has to fix their things if the C side breaks them, but in practice it doesn't work that way, allegedly because Linus has NAKed patchsets that didn't have fixes for Rust being broken in them. I would imagine the hopefully straightforward path would be to make sure the Rust code gets build tested, and then if that fails, have a route to reach out to the people who want to do R4L work and tell them "this is breaking because of what I changed, can you help when you get a chance", for the maintainers who don't want to learn Rust. ...of course, one reason the maintainers loathe this idea, is that the personality archetype who works on Linux generally is the kind of person who would go fix it themselves because they're impatient, so with that framing, it's pretty clear why they think "wait on other people to fix it" is a nonstarter de facto...
      • braiamp 2y ago
        > allegedly because Linus has NAKed patchsets that didn't have fixes for Rust being broken in them He NAKed one patch. One single patch. And the reasoning for that patch was that everyone was in the holidays. Otherwise he was willing to ship a broken rust kernel.
      • Vilian 2y ago
        That's standard specially when a subsystem break other, you Cc them
        • rincebrain 2y ago
          Sure, sorry, I should have described that better. I understand that's standard if you know you broke them, I was imagining a solution to the problem "this is broken and needs to be fixed, preferably quickly, to unblock me, but I don't want to go down the Rust rabbit hole", since by nature, I would not expect most of the R4L maintainers to want to volunteer to do that kind of turnaround time on a consistent basis. (This might be an unnecessary optimization if they're already planning for that, but speaking personally, I would think a lot of the fixes involved would probably be somewhat mechanical on the R4L side for that kind of breakage, and I also think it would provide something of a "happy path" for maintainers, in terms of perception, to know that they have an "in case of Rust do X" ramp even if in practice it rarely if ever gets exercised...)
    • steveklabnik 2y ago
      > I don't think that's true though, given the Linux development model where all internal APIs are private and unstable and can be changed at any time Others have pointed out that this is mistaken, but also, it's in the link itself: > So when you change the C interfaces, the Rust people will have to deal with the fallout, and will have to fix the Rust bindings. That's kind of the promise here: there's that "wall of protection" around C developers that don't want to deal with Rust issues in the promise that they don't have to deal with Rust.
      • IshKebab 2y ago
        > So when you change the C interfaces, the Rust people will have to deal with the fallout, and will have to fix the Rust bindings. I wasn't mistaken; this is exactly what I thought. The problem is that Rust developers aren't magical genies who you can summon to fix Rust code. In the real world they are volunteers who might be busy, on holiday, no longer interested, dead, etc. What do C maintainers do when the Rust genies don't magically appear and fix their code? Just break the Rust code? That's obviously not a good long term solution. So while I totally disagree with the C developers - they should just learn Rust! - I don't think you can get around this point by just hand-waving "don't worry about it; someone else will fix the Rust code". Maybe in the very short term, but not in the medium term.
        • braiamp 2y ago
          > The problem is that Rust developers aren't magical genies who you can summon to fix Rust code Linus will release a broken Rust kernel. Literally that has happened before in an indirect way because it was the holidays and a patch was not merged, but was explicitly told that the only reason why it wasn't merged was because the holidays made tracking the issue harder. Also, every new piece of code has to enter linux-next.
        • steveklabnik 2y ago
          There are at least six people whose full time employment is working on Rust for Linux, across a few companies. That scenario won’t happen.
• GolDDranks 2y ago
  The maintainer of the DMA subsystem, Christoph Hellwig seems to vehemently oppose having an mixed-language codebase, and is using the confusion whether he as a maintainer has a say against the consumers of his APIs, to further his cause. There was indeed some confusion about this, because the normal rule of kernel development is that if you change an API, you also fix the downstream users. (A standard-ish monorepo rule.) However, because R4L is an experiment, the rule seems to be different: it's up to the R4L team to fix the Rust side if things break, so the upstream C devs don't need to care about Rust if they don't want to. (And conversely, they can't then weaponize their reluctance against Rust. That blade cuts both ways!) However, apparently there wasn't sufficient clarity about this rule. Well, now there is.
  • woodrowbarlow 2y ago
    logistically, how would this work in a monorepo? if you merge an API change and don't fix downstream users, isn't the build broken for everyone? or is there a deprecation process where the old API remains functional for a period?
    • bryanlarsen 2y ago
      Rust is behind a flag that defaults off, so it's only broken for people who flip the flag.
      • wtallis 2y ago
        More generally, the Linux kernel and its build system are pretty modular in spite of being maintained in a single repo, so a bug somewhere usually isn't a showstopper for everyone, everywhere. Code that won't compile only affects configurations that try to compile that code. No real-world configs could enable compiling every optional piece of code in the kernel tree.
• tuetuopay 2y ago
  In the end, for kernel users, yes. However, what he did is much larger IMHO: he addressed the utter blockade performed by the maintainer, that basically said no without any margin of negotiation. In the end, he explicited that not wanting to touch Rust goes both ways: you don't want to touch Rust? Fine. But you lose all rights to complain about what others are doing with Rust. In other words: don't be a jerk, and don't have selective involvement.
• rincebrain 2y ago
  Specifically, the reason Linus is lashing out here is that devs are claiming they want nothing to do with Rust, and then suggesting that that means they get to say Rust can't call their code. e.g. what they really are doing, as Christoph explicitly admitted in the thread, is that they are attempting to relitigate whether you can use Rust at all in Linux, and convince people that it's so hard to work on that they just give up and go home.
• hu3 2y ago
  > So when you change the C interfaces, the Rust people will have to deal with the fallout, and will have to fix the Rust bindings. That's kind of the promise here: there's that "wall of protection" around C developers that don't want to deal with Rust issues in the promise that they don't have to deal with Rust. - Linus