5 ms·

>Changing C interfaces will often have implications for the Rust code and may break it; somebody will the have to fix the problems. Torvalds said that, for now,

>Changing C interfaces will often have implications for the Rust code and may break it; somebody will the have to fix the problems. Torvalds said that, for now, breaking the Rust code is permissible, but that will change at some point in the future. I think this is the main technical change needed from the Linux kernel. It needs a layer of quasi-stable well documented subsystem APIs, which ideally would be "inherently safe" or at least have clear safe usage contracts. And it's fine for these interfaces to have relaxed stability guarantees in the early (pre-1.0, if you will) experimental stages. Changing them would involve more work and synchronization (C maintainers would not be able to quickly "refactor" these parts), but it's a familiar problem for many large projects. It's the only reasonable point from the infamous tantrum by Ted Ts'o during the Rust for filesystems talk, everything else, to put it mildly, was a really disappointing behavior from a Linux subsystem maintainer.

BD103 2y ago
For those curious, this is the link[0] to the filesystems talk with the relevant timestamp. A bit more was discussed in this[1] article as well about Wedson Almeida Filho leaving. [0]: https://youtu.be/WiPp9YEBV0Q?t=1529 https://youtu.be/WiPp9YEBV0Q?t=1529 [1]: https://lwn.net/Articles/987635/ https://lwn.net/Articles/987635/
josephcsible 2y ago
Why do we need quasi-stable anything within the kernel? Wouldn't a much better long-term solution be changing the rule to "break whatever APIs you want, but you have to fix all of the in-tree Rust uses too"?
- newpavlov 2y ago
  The problem is that some high-profile people who want to preserve their right to "break whatever", do not want to shoulder responsibility of fixing Rust code which depends on the broken stuff. Even worse, they even do not want to explain and document semantics of existing APIs (sic)! See the video linked in the sibling comment. Speaking more broadly, freely breaking stuff in large projects is dangerous, since fixing stuff may require a more specialized expertise and knowledge, e.g. being well-versed in Rust safety rules, knowing some obscure information about hardware behavior, or being aware about some tricky invariant which must be preserved by the code. This is why changes in API boundaries often require synchronization between different teams.
  - vlovich123 2y ago
    At least Ted Tso has clarified his position greatly from what initially read as “I refuse to learn rust”. Instead yes asking for a guide to help him understand the Rust: > There is a need for documentation and tutorials on how to write filesystem code in idiomatic Rust. He said that he has a lot to learn; he is willing to do that, but needs help on what to learn.
    - steveklabnik 2y ago
      This is heartening to read for sure.
  - pas 2y ago
    > since fixing stuff may require a more specialized expertise and knowledge That's why sign-offs from reviewers are "required", no? It seems the correct mentality to encourage people to do hack on the kernel, show that things are possible, they are "working on my machine", and then ... yeah, they will need to go through the usually length process of getting it merged. But starting with "synchronization between different teams" usually gets shrugs and "let's get back to it soon"-s.
  - arp242 2y ago
    One of the advantage of big monorepos (and Linux really is just one big "monorepo") is that you can change $whatever as long as you also fix everything that depends on that. Sometimes this is just simple stuff, like adding a function argument, changing a type, or something like that. "Break whatever" is a bit of a crude way of putting it, but yeah, it's definitely an advantage IMHO. Not just for the final patch, but also prototyping, experimenting, showing ideas, etc.
w10-1 2y ago
> It needs a layer of quasi-stable well documented subsystem APIs I think the Rust developers weren't even asking for that. They just want the C developers to sign up to some semantics. But the C developers know the function interface has evolved to be merely functional: it works, but with few invariants, riddled with caveats and few cross-function guarantees. It can't be hoisted into meaningful semantics, no less a type system, particularly across 10 filesystem API's. Rust developers should focus on drivers in subsystems with stable API's, instead of trying to stabilize what decades of work has failed to.
- rowanG077 2y ago
  Of course it has semantics. Whether someone knows all of them is a different matter. But whether you use rust or c you have to know the contract to be able to write correct code. The only reason the kernel devs got away with such sloppiness is because c is the ultimate 'I do what you tell me to boss" language.
  - deleted 2y ago
    [deleted]
  - rc00 2y ago
    And yet, Rust is the exact opposite. Why try to make the two meet? The argument for Rust in the kernel over memory safety can be remediated by better memory safety tools for C can't it? (Not that the Linux kernel project doesn't have any already.) Why not devote the time to writing a better memory safety tool for the Linux kernel (or C in general) rather than keep trying to force two disparate cultures and ideologies to meet in some fantasy middle? ThePrimeagen actually had an interesting opinion on this recently too, worth a watch: https://www.youtube.com/watch?v=62oTu9hjxL4 https://www.youtube.com/watch?v=62oTu9hjxL4
    - steveklabnik 2y ago
      Rust has the very strong pro of already existing. If those tools existed, I’m sure the conversation would be quite different. It’s also not just about memory safety. Greg in particular has recognized how Rust’s type system will be helpful for preventing other kinds of bugs, for example.

BD103 2y ago

For those curious, this is the link[0] to the filesystems talk with the relevant timestamp. A bit more was discussed in this[1] article as well about Wedson Almeida Filho leaving. [0]: https://youtu.be/WiPp9YEBV0Q?t=1529 https://youtu.be/WiPp9YEBV0Q?t=1529 [1]: https://lwn.net/Articles/987635/ https://lwn.net/Articles/987635/

josephcsible 2y ago

Why do we need quasi-stable anything within the kernel? Wouldn't a much better long-term solution be changing the rule to "break whatever APIs you want, but you have to fix all of the in-tree Rust uses too"?

newpavlov 2y ago
The problem is that some high-profile people who want to preserve their right to "break whatever", do not want to shoulder responsibility of fixing Rust code which depends on the broken stuff. Even worse, they even do not want to explain and document semantics of existing APIs (sic)! See the video linked in the sibling comment. Speaking more broadly, freely breaking stuff in large projects is dangerous, since fixing stuff may require a more specialized expertise and knowledge, e.g. being well-versed in Rust safety rules, knowing some obscure information about hardware behavior, or being aware about some tricky invariant which must be preserved by the code. This is why changes in API boundaries often require synchronization between different teams.
- vlovich123 2y ago
  At least Ted Tso has clarified his position greatly from what initially read as “I refuse to learn rust”. Instead yes asking for a guide to help him understand the Rust: > There is a need for documentation and tutorials on how to write filesystem code in idiomatic Rust. He said that he has a lot to learn; he is willing to do that, but needs help on what to learn.
  - steveklabnik 2y ago
    This is heartening to read for sure.
- pas 2y ago
  > since fixing stuff may require a more specialized expertise and knowledge That's why sign-offs from reviewers are "required", no? It seems the correct mentality to encourage people to do hack on the kernel, show that things are possible, they are "working on my machine", and then ... yeah, they will need to go through the usually length process of getting it merged. But starting with "synchronization between different teams" usually gets shrugs and "let's get back to it soon"-s.
- arp242 2y ago
  One of the advantage of big monorepos (and Linux really is just one big "monorepo") is that you can change $whatever as long as you also fix everything that depends on that. Sometimes this is just simple stuff, like adding a function argument, changing a type, or something like that. "Break whatever" is a bit of a crude way of putting it, but yeah, it's definitely an advantage IMHO. Not just for the final patch, but also prototyping, experimenting, showing ideas, etc.

w10-1 2y ago

> It needs a layer of quasi-stable well documented subsystem APIs I think the Rust developers weren't even asking for that. They just want the C developers to sign up to some semantics. But the C developers know the function interface has evolved to be merely functional: it works, but with few invariants, riddled with caveats and few cross-function guarantees. It can't be hoisted into meaningful semantics, no less a type system, particularly across 10 filesystem API's. Rust developers should focus on drivers in subsystems with stable API's, instead of trying to stabilize what decades of work has failed to.

rowanG077 2y ago
Of course it has semantics. Whether someone knows all of them is a different matter. But whether you use rust or c you have to know the contract to be able to write correct code. The only reason the kernel devs got away with such sloppiness is because c is the ultimate 'I do what you tell me to boss" language.
- deleted 2y ago
  [deleted]
- rc00 2y ago
  And yet, Rust is the exact opposite. Why try to make the two meet? The argument for Rust in the kernel over memory safety can be remediated by better memory safety tools for C can't it? (Not that the Linux kernel project doesn't have any already.) Why not devote the time to writing a better memory safety tool for the Linux kernel (or C in general) rather than keep trying to force two disparate cultures and ideologies to meet in some fantasy middle? ThePrimeagen actually had an interesting opinion on this recently too, worth a watch: https://www.youtube.com/watch?v=62oTu9hjxL4 https://www.youtube.com/watch?v=62oTu9hjxL4
  - steveklabnik 2y ago
    Rust has the very strong pro of already existing. If those tools existed, I’m sure the conversation would be quite different. It’s also not just about memory safety. Greg in particular has recognized how Rust’s type system will be helpful for preventing other kinds of bugs, for example.