8 ms·
"Rust is great for kernels and the embedded space" I'm no expert, but from a layperson's perspective, the rust usage in these areas depends on a lot of "unsafe
by tyingq 9y ago
"Rust is great for kernels and the embedded space"
I'm no expert, but from a layperson's perspective, the rust usage in these areas depends on a lot of "unsafe" blocks, fighting the borrow checker, etc.
Does that ratchet it down from "great" for this space to something like "ok" or "usable"?
Or is this just the best we can do in terms of a memory safe language for kernel and/or bare metal embedded?
- geofft 9y agoThe point of unsafe blocks is to contain unsafety. In a language like C, the natural pattern is to expose raw pointers through the entire codebase and document how to use them correctly, when they can be mutated, who's responsible for freeing them, etc. In Rust, the natural pattern is to implement wrapper types and other abstractions that themselves make heavy use of unsafe blocks but whose APIs are safe. This isn't unique to kernels. The standard heap-allocated vector type in Rust (on which the standard string type is based) is full of unsafe blocks: https://doc.rust-lang.org/src/alloc/vec.rs.html https://doc.rust-lang.org/src/alloc/vec.rs.html The problem of implementing a heap-allocated vector, with some elements uninitialized but available for use and with the ability to reallocate the entire vector and copy the members, is not a problem that lends itself well to the borrow checker's view of the world, so raw pointers are a fine way to solve it. Use of vectors themselves, their iterators, references to items in the vectors, etc. does match the borrow checker's view of the world, so the average Rust program running in userspace doesn't have to care that the standard library itself isn't implemented in safe Rust. So copious usage of the "unsafe" keyword in a codebase can be a sign that the code as a whole is more safe: instead of having large portions of the code be unsafe (or instead of having the code pretend it's containing unsafety when it's not), only the parts that need raw memory access use it, and they maintain invariants used by the rest of the code. It is certainly possible to do better - there are no machine-checked proofs that unsafe code does what it claims and upholds the invariants it claims to uphold. There could be, and I'm really interested to see what people do with languages that lend themselves better to proofs (usually dependently-typed, Turing-incomplete-but-very-powerful languages). But right now the state of the art is that such languages are a pain to use and writing proofs is even more of a pain. I think that's where the compromise is right now. The borrow checker is, arguably, a system for proving things about your code; it just only knows how to prove specific things. (Way more than C's type system proves, but way less than you'd prove in a perfect world.)
- tyingq 9y agoTerrific, in-depth, answer...and I appreciate it.
- javajosh 9y agoA really nice answer. It's too bad, I suppose, that the same abstraction can't be used "all the way down". I suppose then we'd be stuck with some high-level encoding of the chip's ISA, perhaps generalized to allow the programmer the ability to define new instructions, and an increased number of registers, etc.
- kzrdude 9y agoThis is a teaser / preview and not necessarily what is going to come. But Rust could evolve to providing checks like this: https://i.imgur.com/oqtEDoV.png https://i.imgur.com/oqtEDoV.png From miri, page 27 in their slide deck. The miri project has changed a lot since the slides were written — and it is now close to being merged into rustc. miri: https://github.com/solson/miri https://github.com/solson/miri
- Animats 9y agoThe usual problem in C is ambiguity over "how big is it", the cause of most buffer overflows. Rust tries to contain that, but doesn't solve the problem at the language level. Part of the trouble is that there's no way to talk about uninitialized memory in Rust. If you implement something like a growable array (rust calls this a "Vector", a poorly chosen name), it has space in use and space that's uninitialized and unused. There's no way to express that concept within the language. If you had a way in Rust to say that only elements 0..N are initialized, and only they can be accessed, you could express that situation. Initializing another slot means more elements are now initialized. This is run-time checkable with some overhead. Once you can say that, you can prove it. The proof techniques are well known. The first step is to be able to talk about it within the language. A predicate like initialized(A,i,j) meaning that array A is initialized from elements i to j inclusive is all that's needed. Then, when you write assert(initialized(A,i,j)); A[j+1] = 0; assert(initialized(A,i,j+1); you've expressed how much of the array is now initialized. Then automatic theorem proving takes over and proves that if the first assert is true, so is the second. This eliminates the need for "unsafe" in many places. A fancier type system is not needed. It only takes one wrong piece of unsafe code to enable a buffer overflow attack.
- steveklabnik 9y agoYou'd be surprised at how little unsafe you need. Consider https://os.phil-opp.com/page-tables/ https://os.phil-opp.com/page-tables/ for example, it uses the type system effectively to eliminate some issues, in the end, it only has two calls to unsafe. On some level, yes, it's the best we can do. Unless you want to add hardware semantics into your language. Unless you do, writing two bytes to 0xB8000 is going to be an arbitrary memory access. At least, that's my take.
- cwzwarich 9y ago> Consider https://os.phil-opp.com/page-tables/ https://os.phil-opp.com/page-tables/ for example, it uses the type system effectively to eliminate some issues, in the end, it only has two calls to unsafe. However, those two calls to unsafe rely on an invariant maintained by the rest of the code that is not marked unsafe. Effectively, the entire module has to be treated as unsafe code by the programmer.
- steveklabnik 9y agoYes, absolutely. It's still a far cry from what many people assume, which is that the whole shebang must be.
- cwzwarich 9y agoFor kernel-level programming, I think Rust is basically the same as C or C++ in this regard. I haven't seen a kernel written in Rust where I am convinced there is a submodule that can be treated as if its correctness doesn't impact the safety of the entire kernel.
- Rusky 9y ago> a submodule that can be treated as if its correctness doesn't impact the safety of the entire kernel. You don't have to go all that way to be better than C or C++. Rust can give you a submodule A whose correctness doesn't impact the safety of any uses of submodule B.
- traverseda 9y agoHaving never used rust and having barely used C, here are some uninformed opinions. A memory safe language for the kernel doesn't seem possible using "traditional" safety tools. I'd expect a "safer" language would look a lot like formal verification. If you can make a safer language that doesn't look a lot like formal verification, you can probably automate a lot of formal verification. There are still some big advantages to rust. By explicitly marking unsafe code, you've made it a lot easier to figure out what needs to be focused on. I imagine you can even formally verify just the unsafe parts of your ecosystem. Also, rusts tooling just seems more pleasant.
- pjmlp 9y agoESPOL for Burroughs B55000 in 1961 already allowed for some kind of safety. It was a systems programming language based on Algol and already had the notion of unsafe code for low level code. Those modules had to be marked as UNSAFE and could only be executed if allowed to do so by the sysadmin. Similar exemples could be provided about PL/S, PL/8, Mesa, Modula-2, Modula-3 and many others. One big contribution that Rust brings into the table is to educate a whole generation that belives the myth that C was the very first systems programming language, that there are actually safer alternatives.
- catnaroek 9y ago> Those modules had to be marked as UNSAFE and could only be executed if allowed to do so by the sysadmin. Safety vs. authorization are completely orthogonal. The former is an objective property of a program, to be established or refuted with formal proof. The latter depends on the subjective whims of one specific human being, in this case, the sysadmin. And, in my experience, the typical sysadmin is not a good enough semanticist to determine whether a program is safe or not.
- pjmlp 9y agoThe point being that unsafe code had to be marked as such to be compilable and that tainted the binary, which was only allowed to execute after given permission. Formal proofs can also contain logic errors. Any kind of safety improvement is better than what C offers.
- 013a 9y agoEven if you wrote your entire program in unsafe, you're still enjoying an experience at least as good as C++; you just don't get the compile time memory safety. And its unlikely your entire program would ever have to be in unsafe.