6 ms·

"Torvalds said that it is not necessary to understand Rust to let it into a subsystem; after all, he said, nobody understands the memory-management subsystem, b

"Torvalds said that it is not necessary to understand Rust to let it into a subsystem; after all, he said, nobody understands the memory-management subsystem, but everybody is able to work with it." Chuckled a bit at this line, anyone have context on how true this is?

syndicatedjelly 2y ago
It’s an opinion, but it sounds very good from the perspective of treating the relationships between system and subsystems as an interface to be managed.
j16sdiz 2y ago
For most driver or subsystem, maybe you don't need to know how mm works. Rust is different. The kernel Rust teams are trying to encode some safety invariant. If any of those mismatch with the C side, it breaks. Those invariant need some non trivial knowledge of rust to understand
- nine_k 2y ago
  Is there an example of what you're describing?
  - steveklabnik 2y ago
    There’s a recent drama where the Rust folks asked some people to clarify some of the semantics of some of the filesystem APIs, and this request wasn’t taken well. There’s been a bunch of hn threads about it.
    - asne11 2y ago
      like which?
      - steveklabnik 2y ago
        I apologize, I am on my phone, so rather than curating links, check out https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=rust%20linux&sort=byDate&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu..., most of which are about this situation.
      - xgstation 2y ago
        not sure if this is what the op referred but like this one https://news.ycombinator.com/item?id=41450347 https://news.ycombinator.com/item?id=41450347 didn't find threads that regarding "clarify APIs semantics", but kernel docs are indeed not in a very good condition. Since C does not provide same level of soundness that Rust does, there are many hidden traps. asahi developer had a good discuss about this https://threadreaderapp.com/thread/1829852697107055047.html https://threadreaderapp.com/thread/1829852697107055047.html
        steveklabnik 2y ago
        This overall situation is, yes. And the stuff from Lina is related, thanks for also pointing that out.
  - pornel 2y ago
    About invariants generally: Rust wants to know when memory behind each pointer is immutable, or mutable by Rust only, or could be mutated by something else while Rust has a pointer to it. Rust also encodes which types can't be moved to another thread, and which pointers own their memory and need to be freed by Rust. These are part of the type system, so they need to be defined precisely. The answer to these questions can't be just "it depends". If there are run-time or config-dependent conditions when the same data is owned or not or immutable or not, Rust has enums, unions and unsafe escape hatches to guard access to it.
- dangitman 2y ago
  [dead]
  - vlovich123 2y ago
    By definition the undefined memory behavior is only in the undefined parts of the spec modulo bugs. The spec is written against an abstract virtual machine and C was one of the first to pioneer such a concept and why it was so successful at getting ported everywhere.
    - dangitman 2y ago
      [dead]
      - steveklabnik 2y ago
        > I was under the impression most of how c interacts with memory is part of the undefined part of the spec For a long time, the memory model, formally speaking, was underspecified. Both C and C++ agreed on and added a memory model in C11 and C++11. > fencing You can add a fence via this API: https://en.cppreference.com/w/c/atomic/atomic_thread_fence https://en.cppreference.com/w/c/atomic/atomic_thread_fence > This varies per arch Right, so what assembly this API will emit depends on the underlying architecture details. Notably, the Linux kernel does not use the standard C memory model, it uses its own.
        dangitman 2y ago
        [dead]
        vlovich123 2y ago
        To be clear, that’s the definition of the memory model under concurrent execution. I’m pretty sure the single threaded version was well defined
        steveklabnik 2y ago
klysm 2y ago
It's true in the sense that nobody understands it well enough to avoid writing memory safety bugs.
- AlotOfReading 2y ago
  A lot of kernel resources are managed through infrastructure like devres: https://docs.kernel.org/6.0/driver-api/driver-model/devres.html https://docs.kernel.org/6.0/driver-api/driver-model/devres.h... These days it's entirely possible to write a decent driver with only the foggiest idea of how memory management happens in the kernel.
j16sdiz 2y ago
"Torvalds said that, for now, breaking the Rust code is permissible, but that will change at some point in the future. Kroah-Hartman said that the Rust developers can take responsibility for the maintenance of the abstractions they add." This need some very good expectation management.
raggi 2y ago
It's very probably true in the totality of "as expressed in a real build for all configurations and architectures", too much variation of behavior to have the whole map in mind at once. You can work through it potentially, and I'm sure a few come close, but others will have things top of mind that experts don't.

syndicatedjelly 2y ago

It’s an opinion, but it sounds very good from the perspective of treating the relationships between system and subsystems as an interface to be managed.

j16sdiz 2y ago

For most driver or subsystem, maybe you don't need to know how mm works. Rust is different. The kernel Rust teams are trying to encode some safety invariant. If any of those mismatch with the C side, it breaks. Those invariant need some non trivial knowledge of rust to understand

nine_k 2y ago
Is there an example of what you're describing?
- steveklabnik 2y ago
  There’s a recent drama where the Rust folks asked some people to clarify some of the semantics of some of the filesystem APIs, and this request wasn’t taken well. There’s been a bunch of hn threads about it.
  - asne11 2y ago
    like which?
    - steveklabnik 2y ago
      I apologize, I am on my phone, so rather than curating links, check out https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=rust%20linux&sort=byDate&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu..., most of which are about this situation.
    - xgstation 2y ago
      not sure if this is what the op referred but like this one https://news.ycombinator.com/item?id=41450347 https://news.ycombinator.com/item?id=41450347 didn't find threads that regarding "clarify APIs semantics", but kernel docs are indeed not in a very good condition. Since C does not provide same level of soundness that Rust does, there are many hidden traps. asahi developer had a good discuss about this https://threadreaderapp.com/thread/1829852697107055047.html https://threadreaderapp.com/thread/1829852697107055047.html
      - steveklabnik 2y ago
        This overall situation is, yes. And the stuff from Lina is related, thanks for also pointing that out.
- pornel 2y ago
  About invariants generally: Rust wants to know when memory behind each pointer is immutable, or mutable by Rust only, or could be mutated by something else while Rust has a pointer to it. Rust also encodes which types can't be moved to another thread, and which pointers own their memory and need to be freed by Rust. These are part of the type system, so they need to be defined precisely. The answer to these questions can't be just "it depends". If there are run-time or config-dependent conditions when the same data is owned or not or immutable or not, Rust has enums, unions and unsafe escape hatches to guard access to it.
dangitman 2y ago
[dead]
- vlovich123 2y ago
  By definition the undefined memory behavior is only in the undefined parts of the spec modulo bugs. The spec is written against an abstract virtual machine and C was one of the first to pioneer such a concept and why it was so successful at getting ported everywhere.
  - dangitman 2y ago
    [dead]
    - steveklabnik 2y ago
      > I was under the impression most of how c interacts with memory is part of the undefined part of the spec For a long time, the memory model, formally speaking, was underspecified. Both C and C++ agreed on and added a memory model in C11 and C++11. > fencing You can add a fence via this API: https://en.cppreference.com/w/c/atomic/atomic_thread_fence https://en.cppreference.com/w/c/atomic/atomic_thread_fence > This varies per arch Right, so what assembly this API will emit depends on the underlying architecture details. Notably, the Linux kernel does not use the standard C memory model, it uses its own.
      - dangitman 2y ago
        [dead]
      - vlovich123 2y ago
        To be clear, that’s the definition of the memory model under concurrent execution. I’m pretty sure the single threaded version was well defined
        steveklabnik 2y ago

klysm 2y ago

It's true in the sense that nobody understands it well enough to avoid writing memory safety bugs.

AlotOfReading 2y ago
A lot of kernel resources are managed through infrastructure like devres: https://docs.kernel.org/6.0/driver-api/driver-model/devres.html https://docs.kernel.org/6.0/driver-api/driver-model/devres.h... These days it's entirely possible to write a decent driver with only the foggiest idea of how memory management happens in the kernel.

j16sdiz 2y ago

"Torvalds said that, for now, breaking the Rust code is permissible, but that will change at some point in the future. Kroah-Hartman said that the Rust developers can take responsibility for the maintenance of the abstractions they add." This need some very good expectation management.

raggi 2y ago

It's very probably true in the totality of "as expressed in a real build for all configurations and architectures", too much variation of behavior to have the whole map in mind at once. You can work through it potentially, and I'm sure a few come close, but others will have things top of mind that experts don't.