13 ms·
This makes me so excited! Great job to Alex and the entire team getting this done. I know Rust is not the most perfect language for some. For me, it’s really e
by bluejekyll 9y ago
This makes me so excited! Great job to Alex and the entire team getting this done.
I know Rust is not the most perfect language for some. For me, it’s really exciting that there is finally a high level language which can scale for every potential use case.
Rust is great for kernels and the embedded space; Excels in the systems space; Is gaining traction in the native application space; Has great tooling for web backends; and, now, with WASM, it’s capable of targeting the web (Yes, without DOM access this is limited, but my understanding is that is coming at some point in the future).
Are there any other runtime free, memory safe languages that can target all of those use cases?
- tyingq 9y ago"Rust is great for kernels and the embedded space" I'm no expert, but from a layperson's perspective, the rust usage in these areas depends on a lot of "unsafe" blocks, fighting the borrow checker, etc. Does that ratchet it down from "great" for this space to something like "ok" or "usable"? Or is this just the best we can do in terms of a memory safe language for kernel and/or bare metal embedded?
- geofft 9y agoThe point of unsafe blocks is to contain unsafety. In a language like C, the natural pattern is to expose raw pointers through the entire codebase and document how to use them correctly, when they can be mutated, who's responsible for freeing them, etc. In Rust, the natural pattern is to implement wrapper types and other abstractions that themselves make heavy use of unsafe blocks but whose APIs are safe. This isn't unique to kernels. The standard heap-allocated vector type in Rust (on which the standard string type is based) is full of unsafe blocks: https://doc.rust-lang.org/src/alloc/vec.rs.html https://doc.rust-lang.org/src/alloc/vec.rs.html The problem of implementing a heap-allocated vector, with some elements uninitialized but available for use and with the ability to reallocate the entire vector and copy the members, is not a problem that lends itself well to the borrow checker's view of the world, so raw pointers are a fine way to solve it. Use of vectors themselves, their iterators, references to items in the vectors, etc. does match the borrow checker's view of the world, so the average Rust program running in userspace doesn't have to care that the standard library itself isn't implemented in safe Rust. So copious usage of the "unsafe" keyword in a codebase can be a sign that the code as a whole is more safe: instead of having large portions of the code be unsafe (or instead of having the code pretend it's containing unsafety when it's not), only the parts that need raw memory access use it, and they maintain invariants used by the rest of the code. It is certainly possible to do better - there are no machine-checked proofs that unsafe code does what it claims and upholds the invariants it claims to uphold. There could be, and I'm really interested to see what people do with languages that lend themselves better to proofs (usually dependently-typed, Turing-incomplete-but-very-powerful languages). But right now the state of the art is that such languages are a pain to use and writing proofs is even more of a pain. I think that's where the compromise is right now. The borrow checker is, arguably, a system for proving things about your code; it just only knows how to prove specific things. (Way more than C's type system proves, but way less than you'd prove in a perfect world.)
- tyingq 9y agoTerrific, in-depth, answer...and I appreciate it.
- javajosh 9y agoA really nice answer. It's too bad, I suppose, that the same abstraction can't be used "all the way down". I suppose then we'd be stuck with some high-level encoding of the chip's ISA, perhaps generalized to allow the programmer the ability to define new instructions, and an increased number of registers, etc.
- kzrdude 9y agoThis is a teaser / preview and not necessarily what is going to come. But Rust could evolve to providing checks like this: https://i.imgur.com/oqtEDoV.png https://i.imgur.com/oqtEDoV.png From miri, page 27 in their slide deck. The miri project has changed a lot since the slides were written — and it is now close to being merged into rustc. miri: https://github.com/solson/miri https://github.com/solson/miri
- Animats 9y agoThe usual problem in C is ambiguity over "how big is it", the cause of most buffer overflows. Rust tries to contain that, but doesn't solve the problem at the language level. Part of the trouble is that there's no way to talk about uninitialized memory in Rust. If you implement something like a growable array (rust calls this a "Vector", a poorly chosen name), it has space in use and space that's uninitialized and unused. There's no way to express that concept within the language. If you had a way in Rust to say that only elements 0..N are initialized, and only they can be accessed, you could express that situation. Initializing another slot means more elements are now initialized. This is run-time checkable with some overhead. Once you can say that, you can prove it. The proof techniques are well known. The first step is to be able to talk about it within the language. A predicate like initialized(A,i,j) meaning that array A is initialized from elements i to j inclusive is all that's needed. Then, when you write assert(initialized(A,i,j)); A[j+1] = 0; assert(initialized(A,i,j+1); you've expressed how much of the array is now initialized. Then automatic theorem proving takes over and proves that if the first assert is true, so is the second. This eliminates the need for "unsafe" in many places. A fancier type system is not needed. It only takes one wrong piece of unsafe code to enable a buffer overflow attack.
- steveklabnik 9y agoYou'd be surprised at how little unsafe you need. Consider https://os.phil-opp.com/page-tables/ https://os.phil-opp.com/page-tables/ for example, it uses the type system effectively to eliminate some issues, in the end, it only has two calls to unsafe. On some level, yes, it's the best we can do. Unless you want to add hardware semantics into your language. Unless you do, writing two bytes to 0xB8000 is going to be an arbitrary memory access. At least, that's my take.
- cwzwarich 9y ago> Consider https://os.phil-opp.com/page-tables/ https://os.phil-opp.com/page-tables/ for example, it uses the type system effectively to eliminate some issues, in the end, it only has two calls to unsafe. However, those two calls to unsafe rely on an invariant maintained by the rest of the code that is not marked unsafe. Effectively, the entire module has to be treated as unsafe code by the programmer.
- steveklabnik 9y agoYes, absolutely. It's still a far cry from what many people assume, which is that the whole shebang must be.
- cwzwarich 9y agoFor kernel-level programming, I think Rust is basically the same as C or C++ in this regard. I haven't seen a kernel written in Rust where I am convinced there is a submodule that can be treated as if its correctness doesn't impact the safety of the entire kernel.
- Rusky 9y ago> a submodule that can be treated as if its correctness doesn't impact the safety of the entire kernel. You don't have to go all that way to be better than C or C++. Rust can give you a submodule A whose correctness doesn't impact the safety of any uses of submodule B.
- traverseda 9y agoHaving never used rust and having barely used C, here are some uninformed opinions. A memory safe language for the kernel doesn't seem possible using "traditional" safety tools. I'd expect a "safer" language would look a lot like formal verification. If you can make a safer language that doesn't look a lot like formal verification, you can probably automate a lot of formal verification. There are still some big advantages to rust. By explicitly marking unsafe code, you've made it a lot easier to figure out what needs to be focused on. I imagine you can even formally verify just the unsafe parts of your ecosystem. Also, rusts tooling just seems more pleasant.
- pjmlp 9y agoESPOL for Burroughs B55000 in 1961 already allowed for some kind of safety. It was a systems programming language based on Algol and already had the notion of unsafe code for low level code. Those modules had to be marked as UNSAFE and could only be executed if allowed to do so by the sysadmin. Similar exemples could be provided about PL/S, PL/8, Mesa, Modula-2, Modula-3 and many others. One big contribution that Rust brings into the table is to educate a whole generation that belives the myth that C was the very first systems programming language, that there are actually safer alternatives.
- catnaroek 9y ago> Those modules had to be marked as UNSAFE and could only be executed if allowed to do so by the sysadmin. Safety vs. authorization are completely orthogonal. The former is an objective property of a program, to be established or refuted with formal proof. The latter depends on the subjective whims of one specific human being, in this case, the sysadmin. And, in my experience, the typical sysadmin is not a good enough semanticist to determine whether a program is safe or not.
- pjmlp 9y agoThe point being that unsafe code had to be marked as such to be compilable and that tainted the binary, which was only allowed to execute after given permission. Formal proofs can also contain logic errors. Any kind of safety improvement is better than what C offers.
- 013a 9y agoEven if you wrote your entire program in unsafe, you're still enjoying an experience at least as good as C++; you just don't get the compile time memory safety. And its unlikely your entire program would ever have to be in unsafe.
- justincormack 9y agoI think another contender here would be OCaml/Reason, which has both JavaScript and WebAsm targeting, is used in systems space (eg Mirage). Its a bit different in technical choices (functional, GC, for example). Facebook is using Reason for Messenger for example, on mobile.
- d215 9y agoHonest question, Is reason really targeting wasm right now? Do you know a resource describing this? I cannot find anything about this.
- bad_user 9y agoAFAIK Reason is a frontend for OCaml and can target whatever OCaml can target. Note that for targeting JavaScript they are using BuckleScript, an OCaml backend that's not built by Facebook.
- rlp 9y agoI couldn't find anything on this either. The assembly emitters in their repo don't suggest a WASM backend: https://github.com/ocaml/ocaml/tree/trunk/asmcomp https://github.com/ocaml/ocaml/tree/trunk/asmcomp
- k__ 9y agohttps://github.com/sebmarkbage/ocamlrun-wasm https://github.com/sebmarkbage/ocamlrun-wasm reason == ocaml
- stewbrew 9y agoYes, but a byte code interpreter ported to wasm via emscripten != a compiler with a WASM backend.
- k__ 9y agoIsn't the stuff that comes out of the OCaml compiler byte code? As far as I can tell, OCaml has a GC and WASM (at the moment) doesn't so this seems like the only viable solution to me.
- Thaxll 9y agoRust is not really good for web backends, it has no library for talking to popular service, the construct for networking are still uncertain / unstable / hard to use.
- bluejekyll 9y agoFor reference, what I have used that makes me claim this without being misleading (not necessarily all at the same time): https://rocket.rs/ https://rocket.rs/, https://rusoto.github.io/rusoto/rusoto_core/index.html https://rusoto.github.io/rusoto/rusoto_core/index.html, https://docs.rs/postgres/0.15.1/postgres/ https://docs.rs/postgres/0.15.1/postgres/, https://docs.rs/kafka/0.7.0/kafka/ https://docs.rs/kafka/0.7.0/kafka/, https://docs.rs/reqwest/0.8.1/reqwest/ https://docs.rs/reqwest/0.8.1/reqwest/, as well as many other supporting libraries. I've not run into many things where there isn't already support. Also, in terms of networking, if your doing blocking IO, the stdlib is fine. Otherwise, from my experience, I really like https://tokio.rs/ https://tokio.rs/ for non-blocking (I know people are torn on this one) and have used it in my DNS impl, https://github.com/bluejekyll/trust-dns https://github.com/bluejekyll/trust-dns For more info on web and Rust, this is a great page: http://www.arewewebyet.org/ http://www.arewewebyet.org/
- verletx64 9y ago> Rust is great for the embedded space Honestly, I would disagree here, unless we're talking in terms of potential, rather than actual ability right now. There's certainly some level of support, but I really did not get the feeling that Rust was 'great' for embedded, rather that it could be.
- bluejekyll 9y ago> but I really did not get the feeling that Rust was 'great' for embedded, rather that it could be. My experience is very limited here, but I have used TockOS on one of their dev boards. That was very pleasant, no stdlib makes it a little harder than std Rust, but still felt more deterministic than writing in C.
- verletx64 9y agoI was thinking a lot in terms of device support too, I guess. That's important to me; I know with C that I don't have to think too hard about which device, at least in terms of "Does this board support my language well". It's probably even more important for professional embedded engineers, where the hardware decisions may not even be made by the software developers, for various reasons.
- stmw 9y agoGood point, although one advantage with Rust is that it plays well with C -- so you could keep some C drivers or libraries, and do the rest of the work in Rust. That reduces attack surface area while providing a smoother on-ramp to full device support.
- Const-me 9y ago> one advantage with Rust is that it plays well with C Advantage over what? This can be said about the majority of programming languages For example, I’ve recently done an embedded ARM Linux project where I used C# and .NET Core for higher level parts, C++ for lower level, [DllImport] and C API between the two.
- FrappucinoFill 9y ago>which can scale for every potential use case For a certain type of lowish-level programming, yes. But there's a reason that many projects include something like Lua for the majority of the actual application-level work.
- rapind 9y agoNim comes to mind. https://nim-lang.org/ https://nim-lang.org/
- dom96 9y agoThank you for mentioning Nim here. :) To expand a bit, Nim features its own JS backend with a high proportion of the stdlib supporting it. You can access the DOM via the `dom` module[1] and build some pretty cool stuff[2][3]. It doesn't target WebAssembly (yet) however. On the systems side, Nim compiles to C/C++/ObjC and manages memory via a soft real-time GC (or via a choice of other GCs including boehm). 1 - https://nim-lang.org/docs/dom.html https://nim-lang.org/docs/dom.html 2 - https://nim-lang.org/araq/karax.html https://nim-lang.org/araq/karax.html 3 - https://picheta.me/snake/ https://picheta.me/snake/ (source available here: https://github.com/dom96/snake https://github.com/dom96/snake)
- santaclaus 9y ago> Is gaining traction in the native application space Are there any Rust-first UI frameworks rivaling something like Qt, yet?
- executesorder66 9y agoDoes gtk-rs count? http://gtk-rs.org/ http://gtk-rs.org/ https://github.com/gtk-rs https://github.com/gtk-rs
- stochastic_monk 9y agoModern C++ with RAII checks all of your boxes except for web backends. (Though I've heard good things about WT.) Disclosure: I only write software for scientific computing and have no interest or experience in web development.