5 ms·

Can someone with more knowledge than me explain what benefit Rust presents to a low level codebase such as a kernel? I am a C programmer by trade who has playe

Can someone with more knowledge than me explain what benefit Rust presents to a low level codebase such as a kernel? I am a C programmer by trade who has played with Rust. It is a great language but there is very little you can do safely. In higher level projects you can mostly push the un-safety into the standard library. But when you have to do everything yourself, all of the "buggy" code has to be marked unsafe anyway. Allocating and accessing(!!) memory is unsafe. Interfacing with C or asm is unsafe. Your buffer/vector implementation is unsafe. Basically, all the common culprits for memory bugs in C would be the code marked unsafe if rewritten in rust. So what is the benefit?

filmor 6y ago
You'd write the allocator or low-level memory access logic once with unsafe code and afterwards all code using the functionality can be written as safe code. That limits the amount of code that can have data races.
zanellato19 6y ago
Accessing memory is not unsafe, wdym?
- Asraelite 6y ago
  I guess dereferencing a raw pointer?
- mcguire 6y ago
  Building memory management/allocation/malloc systems is.
- megous 6y ago
  In a multi-core system, it may be, since you're not really accessing main memory directly, but you're going through several levels of all kinds of caches, some shared and some not (L1 (instruction, data), L2, L3, TLB, ...), and sometimes competing with devices that access the memory "directly" via DMA.
steveklabnik 6y ago
I work on firmware and kernel-level code, in pure Rust, at my job. We don't do malloc at all. One of the key features Rust enables is that you can build safe abstractions on top of unsafe things. Another key feature is that those abstractions often mean compile-time, but not run-time, cost. If you build the right abstractions, you can minimize the amount of unsafe code pretty dramatically. I don't have numbers handy for our codebase (and it's not open source yet, but will be eventually), but as an example, the Redox documentation (a Rust-based kernel that is, uh... linked... below) says: https://doc.redox-os.org/book/ch01-07-why-rust.html#unsafes https://doc.redox-os.org/book/ch01-07-why-rust.html#unsafes > A quick grep gives us some stats: the kernel has about 300 invocations of unsafe in about 16,000 lines of code overall. Every one of these is carefully audited to ensure correctness. The advantage here is that, if and when there are bugs in this code, you only have to look at the modules containing those unsafe invocations, rather than the entire codebase. There are also other advantages to the Rust language that make it worthwhile too, regardless of safety percentage, but it is true that this is often people's first concerns. The code that landed in linux-next has a lot of unsafe in it, because it is specifically trying to build out the proper abstractions here. The idea is that drivers built on top of this code will not need nearly as much, generally.
- dahfizz 6y ago
  > The advantage here is that, if and when there are bugs in this code, you only have to look at the modules containing those unsafe invocations, rather than the entire codebase. That is a brilliant feature I had not thought of. Thanks!
  - steveklabnik 6y ago
    No problem! I think the skepticism is very warranted. We know that these systems can be created, but we don't have examples of this working at the same scale as projects like Linux yet. That takes time and effort, exactly as Linus and Greg Kroah-Hartman say. We'll see how it works out in practice. I also think the "de novo system" and "adding Rust into a large existing system" are two related, but different, problems, with their own unique challenges.
sfvisser 6y ago
Ideally, you write a small subset of unsafe primitives and encapsulate those in a safe interface and try to use those to implement your actual kernel logic.
habitue 6y ago
unsafe isn't like a taint, where things that use unsafe code must propagate their unsafety outward. It's more like saying "there are invariants that this function expects, but does not enforce, and the caller may violate those invariants". Obviously, we want to minimize code like that. The best way to do that is to build abstractions around the unsafe code that only interact with the unsafe code in a way that respects those invariants. Often, (though not always) that means giving up some amount of flexibility to the user of your safe abstraction. If you design it right, the caller may never miss that flexibility. A recent example of using private constructors and zero-sized compile time invariant enforcement is an embedded blog here: https://www.ecorax.net/macro-bunker-1/ https://www.ecorax.net/macro-bunker-1/ The embedded rust community uses this typestate pattern to build safe abstractions around pin interactions. Very similar things can be done to build safe abstractions around the volatile memory accesses etc that the kernel does.
- bestouff 6y ago
  I see "unsafe" more like "this is code that the compiler can't prove it holds its invariants, but in fact it does and I hope the programmer added a comment to explain why".
dralley 6y ago
>It is a great language but there is very little you can do safely. Even extremely performance-oriented and low level projects such as the standard library or the Redox kernel are less than 5% unsafe code. There's lots you can do safely.
danhau 6y ago
I'm not a pro but here are my two cents. The benefit, and really the point of unsafe, is to encapsulate the unsafe stuff behind safe APIs and to use Rust's other features, like the borrow checker for example, to enforce correct usage of such API. Ex.: I was using nanovg (vector graphics library in C) to build a GUI app some day. I was using Rust and a bare-bones FFI library that provided a 1:1 mapping to the C API. On top of this unsafe C-API, I built a wrapper library in safe Rust that doesn't allow me to use nanovg incorrectly. And not in the use-after-free sense of incorrect (that too), but instead in the "you can't call this function between these other two, that's undefined / will produce unexpected graphical results". So the power of unsafe is to encapsulate "unsafe" behaviour and to enrich it with compile-time information, to build guarantees around it.
- adamnemecek 6y ago
  If you are looking for a Rust nanovg port, look no further https://github.com/femtovg/femtovg https://github.com/femtovg/femtovg. We are in the process of adding a wgpu backend and we are definitely looking for contributors so contact me if you want to get involved.
greydius 6y ago
> all the common culprits for memory bugs in C would be the code marked unsafe if rewritten in rust You've answered your own question.

filmor 6y ago

You'd write the allocator or low-level memory access logic once with unsafe code and afterwards all code using the functionality can be written as safe code. That limits the amount of code that can have data races.

zanellato19 6y ago

Accessing memory is not unsafe, wdym?

Asraelite 6y ago
I guess dereferencing a raw pointer?
mcguire 6y ago
Building memory management/allocation/malloc systems is.
megous 6y ago
In a multi-core system, it may be, since you're not really accessing main memory directly, but you're going through several levels of all kinds of caches, some shared and some not (L1 (instruction, data), L2, L3, TLB, ...), and sometimes competing with devices that access the memory "directly" via DMA.

steveklabnik 6y ago

I work on firmware and kernel-level code, in pure Rust, at my job. We don't do malloc at all. One of the key features Rust enables is that you can build safe abstractions on top of unsafe things. Another key feature is that those abstractions often mean compile-time, but not run-time, cost. If you build the right abstractions, you can minimize the amount of unsafe code pretty dramatically. I don't have numbers handy for our codebase (and it's not open source yet, but will be eventually), but as an example, the Redox documentation (a Rust-based kernel that is, uh... linked... below) says: https://doc.redox-os.org/book/ch01-07-why-rust.html#unsafes https://doc.redox-os.org/book/ch01-07-why-rust.html#unsafes > A quick grep gives us some stats: the kernel has about 300 invocations of unsafe in about 16,000 lines of code overall. Every one of these is carefully audited to ensure correctness. The advantage here is that, if and when there are bugs in this code, you only have to look at the modules containing those unsafe invocations, rather than the entire codebase. There are also other advantages to the Rust language that make it worthwhile too, regardless of safety percentage, but it is true that this is often people's first concerns. The code that landed in linux-next has a lot of unsafe in it, because it is specifically trying to build out the proper abstractions here. The idea is that drivers built on top of this code will not need nearly as much, generally.

dahfizz 6y ago
> The advantage here is that, if and when there are bugs in this code, you only have to look at the modules containing those unsafe invocations, rather than the entire codebase. That is a brilliant feature I had not thought of. Thanks!
- steveklabnik 6y ago
  No problem! I think the skepticism is very warranted. We know that these systems can be created, but we don't have examples of this working at the same scale as projects like Linux yet. That takes time and effort, exactly as Linus and Greg Kroah-Hartman say. We'll see how it works out in practice. I also think the "de novo system" and "adding Rust into a large existing system" are two related, but different, problems, with their own unique challenges.

sfvisser 6y ago

Ideally, you write a small subset of unsafe primitives and encapsulate those in a safe interface and try to use those to implement your actual kernel logic.

habitue 6y ago

unsafe isn't like a taint, where things that use unsafe code must propagate their unsafety outward. It's more like saying "there are invariants that this function expects, but does not enforce, and the caller may violate those invariants". Obviously, we want to minimize code like that. The best way to do that is to build abstractions around the unsafe code that only interact with the unsafe code in a way that respects those invariants. Often, (though not always) that means giving up some amount of flexibility to the user of your safe abstraction. If you design it right, the caller may never miss that flexibility. A recent example of using private constructors and zero-sized compile time invariant enforcement is an embedded blog here: https://www.ecorax.net/macro-bunker-1/ https://www.ecorax.net/macro-bunker-1/ The embedded rust community uses this typestate pattern to build safe abstractions around pin interactions. Very similar things can be done to build safe abstractions around the volatile memory accesses etc that the kernel does.

bestouff 6y ago
I see "unsafe" more like "this is code that the compiler can't prove it holds its invariants, but in fact it does and I hope the programmer added a comment to explain why".

dralley 6y ago

>It is a great language but there is very little you can do safely. Even extremely performance-oriented and low level projects such as the standard library or the Redox kernel are less than 5% unsafe code. There's lots you can do safely.

danhau 6y ago

I'm not a pro but here are my two cents. The benefit, and really the point of unsafe, is to encapsulate the unsafe stuff behind safe APIs and to use Rust's other features, like the borrow checker for example, to enforce correct usage of such API. Ex.: I was using nanovg (vector graphics library in C) to build a GUI app some day. I was using Rust and a bare-bones FFI library that provided a 1:1 mapping to the C API. On top of this unsafe C-API, I built a wrapper library in safe Rust that doesn't allow me to use nanovg incorrectly. And not in the use-after-free sense of incorrect (that too), but instead in the "you can't call this function between these other two, that's undefined / will produce unexpected graphical results". So the power of unsafe is to encapsulate "unsafe" behaviour and to enrich it with compile-time information, to build guarantees around it.

adamnemecek 6y ago
If you are looking for a Rust nanovg port, look no further https://github.com/femtovg/femtovg https://github.com/femtovg/femtovg. We are in the process of adding a wgpu backend and we are definitely looking for contributors so contact me if you want to get involved.

greydius 6y ago

> all the common culprits for memory bugs in C would be the code marked unsafe if rewritten in rust You've answered your own question.