3 ms·

Linux is a unix-like kernel written in C, with no stable internal APIs. Introducing another programming language was never going to work. Not without ample sup

by snvzz 2y ago

Linux is a unix-like kernel written in C, with no stable internal APIs. Introducing another programming language was never going to work. Not without ample support within the existing developer base. The kind thing would have been to reject Rust from the start. We're too late now. The next best option would be to remove it from the kernel entirely. Remarkably, nothing stops Rust developers from forking the kernel or, even better, writing their own system from scratch.

eschaton 2y ago
They could also revise how their C interop works such that the kernel internal interfaces can continue to be specified in C and just used easily from Rust. This is the Swift approach: Consistent, predictable interop that doesn’t insist or presume that the new way is the preferred way.
- deleted 2y ago
  [deleted]
- SkiFire13 2y ago
  The kernel internal interfaces have no information about the expected semantics of the pointers passed in (i.e. for how much are they valid, who's responsible for "free"ing them, with what, etc etc). You have to specify those informations somewhere, and this is currently done when writing Rust bindings. If you want to avoid doing that then you'll just end up writing C in Rust with unsafe everywhere.
  - eschaton 2y ago
    Yes, and that’s just what has to be done. Welcome to interfacing with C, it’s been like that for decades. There are ways these semantics can be annotated via the attribute mechanism in many compilers; this, like nullability, should be added just to improve the ability to statically analyze the existing code and avoid bugs. It can also happen to improve importability of C APIs in other languages, but the primary reason is and should be to make the C code better. So you write unsafe-to-safe wrapper code for anything you need to interact with—once—so as to encode the interface semantics by hand. Eventually it’s no longer needed because you’ve added annotations to the headers instead. And the kernel gets _better_ because these things can be statically checked and thus have bugs avoided, _regardless of_ the use of Rust.
    - pornel 2y ago
      C doesn't have syntax/typesystem to express ownership, lifetimes and thread safety, but Rust does. Personally I'd love if C was extended to be able define APIs more precisely, but that's not a realistic option at the moment. There isn't even any vendor-specific solution to this in C (the hodgepodge of attributes barely scratches the surface). Linux would have to invent its own interface definition language and make C and Rust adopt it. That means more changes for the C maintainers. The C standard moves very slowly, and avoids making non-trivial changes. I'm still waiting for slices (pointer+length type). Rust is here today, and won't stop for a dream of C that may or may not happen in 2050.
      - eschaton 2y ago
        C has been extended in *exactly* that way using the `__((attribute))__` mechanism in GCC and clang. One does not need to wait for a standards body to do this. This is one of the ways in which the static analyzer in clang works and they’re very open to further such extensions, which can be easily wrapped in macros so as to not affect compilers that don’t support them. Do not let the perfect be the enemy of the good. It’s absolutely a realistic option, and it’s one that should be pursued if kernel code quality checking via static analysis is important.
        pornel 2y ago
        Nothing in GCC or clang that exists today is anywhere near the level of expressiveness that Rust uses in its APIs (e.g. lifetimebound in [1]). The current problem is not technical, it's maintainers not wanting to even think about another language in their C codebase. Adding non-standard foreign language's semantics on top of C, not supported natively by C compilers, and forced to have an even uglier syntax than Rust's own, would not make C maintainers happy. You could add an __((attribute))__ that's equivalent to a code comment containing a Rust type, but that also would be strictly worse: less useful to Rust devs, still annoying to C maintainers who wouldn't want to maintain it nor worry about the attributes breaking another compiler for a different language. I don't think it's feasible to use attributes to add functionality that actually does something meaningful, and doesn't look comically bad at the same time. Look at the zoo of annotations that clang had to add to work around lack of a built-in slice type in C, and that's just a simple thing that's written `[T]` in Rust: https://clang.llvm.org/docs/BoundsSafety.html https://clang.llvm.org/docs/BoundsSafety.html Lifetime annotations are annoyingly noisy and viral even in Rust itself, which has a first-class syntax for them, and generics and type inference to hide the noise. In C they'd be even noisier, and had to be added in more places. You can't really take safety annotations that are for Rust and reuse them for static analysis of C code itself. They're for a boundary between languages, not for the C implementation. Rust itself is essentially a static analyzer, with everything in the language designed from the ground up for the static analyzer. C wasn't designed for that. If you transplant enough of Rust's requirements to C, you'll make C maintainers write Rust, but using a botched C syntax and a C compiler that doesn't understand the actual language they're writing. Static analysis is fundamentally limited by C's semantics (limited by undecidability, not mere engineering challenges of implementing a Sufficiently Smart static analyzer). It's not something that can be easily solved with an attribute here and there, because that will be brittle and incomplete[2][3]. Static analysis at that level requires removing a lot of flexibility from C and adding new formalisms, which again would not make C maintainers happy who already resist even tiniest changes of their code to align better with non-C's restrictions. [1]: https://docs.google.com/document/d/e/2PACX-1vRZr-HJcYmf2Y76DhewaiJOhRNpjGHCxliAQTBhFxzv1QTae9o8mhBmDl32CRIuaWZLt5kVeH9e9jXv/pub https://docs.google.com/document/d/e/2PACX-1vRZr-HJcYmf2Y76D... [2]: https://www.circle-lang.org/draft-profiles.html https://www.circle-lang.org/draft-profiles.html [3]: https://safecpp.org/draft-lifetimes.html https://safecpp.org/draft-lifetimes.html
simonask 2y ago
So far, there's no proof that it "was never going to work", and ample circumstantial evidence that it does, in fact, work. There have been no serious technical obstacles, and no actual incidents where maintenance has become harder because of it. This looks 100% like personal resistance, rather than technical.
- snvzz 2y ago
  >This looks 100% like personal resistance, rather than technical. >there's no proof that it "was never going to work" Did you really just skim the post you replied to? It said: >Not without ample support within the existing developer base. Should be clear enough, but I will expand: This is about pre-existing Linux developers and maintainers not willing to commit to the new language, and thus it has nothing to do with anything technical at all. It won't work because it absolutely requires ample support within Linux's developer base to succeed, and such support is not there. Nearly nobody, even within the core (which support is needed the most!), is willing to put any effort into this. I will re-state what I already said: The best and kindest path Linus could have taken would have been to firmly reject the new language from the start. As that is no longer possible, the second best still is: To declare the experiment a failure and conclusively remove the new language from the kernel. The resources freed in Linux's and Rust's communities could then be put to better use, and after a while everybody would be a little less miserable.
  - simonask 2y ago
    So far, the only actual support that has been required is to not actively sabotage it. That’s not a high bar. There does seem to be very substantial interest in RfL in the kernel, with the exception of a few maintainers generating a lot of personal friction. Frankly, it’s exactly the kind of behavior you would be very worried to see from key people maintaining the most important operating system on the planet. My understanding is that part of the reason Linus approved the project is that the current state of Linux is unsustainable. I agree with him. Linux will evolve, or die. Gradually moving on from C will probably be part of that journey, sooner or later.

by snvzz 2y ago

eschaton 2y ago

They could also revise how their C interop works such that the kernel internal interfaces can continue to be specified in C and just used easily from Rust. This is the Swift approach: Consistent, predictable interop that doesn’t insist or presume that the new way is the preferred way.

deleted 2y ago
[deleted]
SkiFire13 2y ago
The kernel internal interfaces have no information about the expected semantics of the pointers passed in (i.e. for how much are they valid, who's responsible for "free"ing them, with what, etc etc). You have to specify those informations somewhere, and this is currently done when writing Rust bindings. If you want to avoid doing that then you'll just end up writing C in Rust with unsafe everywhere.
- eschaton 2y ago
  Yes, and that’s just what has to be done. Welcome to interfacing with C, it’s been like that for decades. There are ways these semantics can be annotated via the attribute mechanism in many compilers; this, like nullability, should be added just to improve the ability to statically analyze the existing code and avoid bugs. It can also happen to improve importability of C APIs in other languages, but the primary reason is and should be to make the C code better. So you write unsafe-to-safe wrapper code for anything you need to interact with—once—so as to encode the interface semantics by hand. Eventually it’s no longer needed because you’ve added annotations to the headers instead. And the kernel gets _better_ because these things can be statically checked and thus have bugs avoided, _regardless of_ the use of Rust.
  - pornel 2y ago
    C doesn't have syntax/typesystem to express ownership, lifetimes and thread safety, but Rust does. Personally I'd love if C was extended to be able define APIs more precisely, but that's not a realistic option at the moment. There isn't even any vendor-specific solution to this in C (the hodgepodge of attributes barely scratches the surface). Linux would have to invent its own interface definition language and make C and Rust adopt it. That means more changes for the C maintainers. The C standard moves very slowly, and avoids making non-trivial changes. I'm still waiting for slices (pointer+length type). Rust is here today, and won't stop for a dream of C that may or may not happen in 2050.
    - eschaton 2y ago
      C has been extended in *exactly* that way using the `__((attribute))__` mechanism in GCC and clang. One does not need to wait for a standards body to do this. This is one of the ways in which the static analyzer in clang works and they’re very open to further such extensions, which can be easily wrapped in macros so as to not affect compilers that don’t support them. Do not let the perfect be the enemy of the good. It’s absolutely a realistic option, and it’s one that should be pursued if kernel code quality checking via static analysis is important.
      - pornel 2y ago
        Nothing in GCC or clang that exists today is anywhere near the level of expressiveness that Rust uses in its APIs (e.g. lifetimebound in [1]). The current problem is not technical, it's maintainers not wanting to even think about another language in their C codebase. Adding non-standard foreign language's semantics on top of C, not supported natively by C compilers, and forced to have an even uglier syntax than Rust's own, would not make C maintainers happy. You could add an __((attribute))__ that's equivalent to a code comment containing a Rust type, but that also would be strictly worse: less useful to Rust devs, still annoying to C maintainers who wouldn't want to maintain it nor worry about the attributes breaking another compiler for a different language. I don't think it's feasible to use attributes to add functionality that actually does something meaningful, and doesn't look comically bad at the same time. Look at the zoo of annotations that clang had to add to work around lack of a built-in slice type in C, and that's just a simple thing that's written `[T]` in Rust: https://clang.llvm.org/docs/BoundsSafety.html https://clang.llvm.org/docs/BoundsSafety.html Lifetime annotations are annoyingly noisy and viral even in Rust itself, which has a first-class syntax for them, and generics and type inference to hide the noise. In C they'd be even noisier, and had to be added in more places. You can't really take safety annotations that are for Rust and reuse them for static analysis of C code itself. They're for a boundary between languages, not for the C implementation. Rust itself is essentially a static analyzer, with everything in the language designed from the ground up for the static analyzer. C wasn't designed for that. If you transplant enough of Rust's requirements to C, you'll make C maintainers write Rust, but using a botched C syntax and a C compiler that doesn't understand the actual language they're writing. Static analysis is fundamentally limited by C's semantics (limited by undecidability, not mere engineering challenges of implementing a Sufficiently Smart static analyzer). It's not something that can be easily solved with an attribute here and there, because that will be brittle and incomplete[2][3]. Static analysis at that level requires removing a lot of flexibility from C and adding new formalisms, which again would not make C maintainers happy who already resist even tiniest changes of their code to align better with non-C's restrictions. [1]: https://docs.google.com/document/d/e/2PACX-1vRZr-HJcYmf2Y76DhewaiJOhRNpjGHCxliAQTBhFxzv1QTae9o8mhBmDl32CRIuaWZLt5kVeH9e9jXv/pub https://docs.google.com/document/d/e/2PACX-1vRZr-HJcYmf2Y76D... [2]: https://www.circle-lang.org/draft-profiles.html https://www.circle-lang.org/draft-profiles.html [3]: https://safecpp.org/draft-lifetimes.html https://safecpp.org/draft-lifetimes.html

simonask 2y ago

So far, there's no proof that it "was never going to work", and ample circumstantial evidence that it does, in fact, work. There have been no serious technical obstacles, and no actual incidents where maintenance has become harder because of it. This looks 100% like personal resistance, rather than technical.

snvzz 2y ago
>This looks 100% like personal resistance, rather than technical. >there's no proof that it "was never going to work" Did you really just skim the post you replied to? It said: >Not without ample support within the existing developer base. Should be clear enough, but I will expand: This is about pre-existing Linux developers and maintainers not willing to commit to the new language, and thus it has nothing to do with anything technical at all. It won't work because it absolutely requires ample support within Linux's developer base to succeed, and such support is not there. Nearly nobody, even within the core (which support is needed the most!), is willing to put any effort into this. I will re-state what I already said: The best and kindest path Linus could have taken would have been to firmly reject the new language from the start. As that is no longer possible, the second best still is: To declare the experiment a failure and conclusively remove the new language from the kernel. The resources freed in Linux's and Rust's communities could then be put to better use, and after a while everybody would be a little less miserable.
- simonask 2y ago
  So far, the only actual support that has been required is to not actively sabotage it. That’s not a high bar. There does seem to be very substantial interest in RfL in the kernel, with the exception of a few maintainers generating a lot of personal friction. Frankly, it’s exactly the kind of behavior you would be very worried to see from key people maintaining the most important operating system on the planet. My understanding is that part of the reason Linus approved the project is that the current state of Linux is unsustainable. I agree with him. Linux will evolve, or die. Gradually moving on from C will probably be part of that journey, sooner or later.