6 ms·
If I'm writing my own out of tree Rust module (for the sole purpose of fun), I wonder how hard it would be to bring in external no-std crates? Rust as a langua
by tux3 4y ago
If I'm writing my own out of tree Rust module (for the sole purpose of fun), I wonder how hard it would be to bring in external no-std crates?
Rust as a language feels unusually high level for something running in kernelspace, although it never really sacrifices any control. But I wonder how far we can reasonably go in making kernelspace feel like userspace, by importing more of the language's 'vocabulary types', and maybe a few of the best utility crates for comfort.
But besides people having fun out of tree, the kernel has plenty of fundamental complexity that can't very well be reduced. That tends to be the good complexity, the meat of it.
What I would like to see is things that aren't essential complexity becoming nicer.
Simple cultural things like better error messages when the tooling fails, or documentation becoming more common rather than the exception. None of this is guaranteed to magically happen just because there's a bit of infrastructure where Make can call rustc. But here's to hoping =)
- gmadsen 4y agoif I was curious, what would be a couple of the good complexities?
- mikepurvis 4y agoNot the OP, but I would assume that necessary complexity is at least where the complexity mirrors whatever real world complexity it is implementing, thinking particularly of protocol stacks like usb, networking, etc.
- tux3 4y agoDepends on what kind of problems you you find interesting, but since it's so close to the metal, and since people want their kernel to be fast, there tends to be a lot of clever algorithms, intricate locking, architecture-specific considerations, dealing with the hardware, all sorts of pretty interesting things you wouldn't usually find in regular business logic. Take filesystem code for example. It can sound like a conceptually simple and well trodden topic, but if you try writing one there's a pretty large design space and tons of interesting problems to try to solve. And then because of concurrency and the need for performance, there can be very intricate locking to the point it can be challenging to hold the whole model in your head all at once (you can have truncates, writes, mmap, all sorts of very different operations in flight in parallel that interract in funny ways). Or memory models. Reading memory-barriers.txt is a lesson in humility, I believe you can count on your fingers and toes the number of people in this world who can truly claim intimate familiarity with all the corner-cases and details implied by the model in that little text file =) But take practically any big subsystem of the kernel, and there tends to be plenty of interesting domain-specific problems and challenges.
- quirino 4y agoI believe they're referring to the essential complexity of the domain. An operating system has some "lower bound" on how complex it can be, which is rather high. But some stuff doesn't _have_ to be hard. Like OP said, there's no inherent reason for error messages and failures to be cryptic or documentation to be sparse. The system could be more transparent or modular, easier to debug/analyze, etc. I think there's an example in GCC vs Clang here, but I'm not qualified to comment.
- biorach 4y agoThis is more a question of policy than mechanism - I'd imagine at least a provisional policy is in place, but the only to be found somewhere in the relevant kernel mailing lists.
- consp 4y ago> I wonder how hard it would be to bring in external no-std crates Wasn't this a thing that would not happen at all? at least that is what I remember from previous discussions. Another question would be how do you avoid an explosion of crates with different versions for each module.
- danhor 4y agoPolicy decisions for linux itself don't (necessarily) apply out-of-tree modules?
- kibwen 4y agoNot a kernel hacker, but if it's just for fun, I don't know if you'd need to limit yourself to no_std crates. Allocation is supported (if it wasn't, then Rust's stdlib wouldn't be adding fallible allocation APIs to support the kernel usecase (and even if crates in the wild aren't using these APIs and abort on OOM, again, this is just for fun)). Meanwhile, I feel like OS-specific stuff performed by libstd (e.g. threads, networking) must surely be supportable somehow, since the OS is right there and all its facilities still exist, even if you might need to adjust codegen to account for the difference between kernelspace and userspace (I assume you would need to compile against a specific target?).
- tialaramex 4y agoAlthough Rust for Linux has its own alloc, that isn't really alloc. It is like if you wanted Coke and got Pepsi, if you just wanted fizzy brown beverage you're fine, but if you actually cared this isn't OK. Rust's String (the mutable owned string type from alloc) implements AddAssign meaning you can write fname += ".txt" and that works, but Rust for Linux doesn't allow that. This is just a really obvious example, there are many other differences
- nextaccountic 4y agoYou can't use methods that may panic on OOM like Vec::push or Box::new. This means that close to 100% of crates that use the stdlib won't be usable in the kernel without modifications.
- mcronce 4y agoMy understanding isn't that you _can't_, but that you _aren't allowed to_ by kernel policy. (The risk being that the kernel will panic on OOM, which is not an acceptable way for the kernel to deal with OOM conditions.) If this is out of tree, AFAIK you can use these all you want. Whether or not you _should_ is obviously a different matter.
- tialaramex 4y agoNo, you can't - they don't exist in Rust for Linux. In Rust the alloc library's Vec has a function Vec::push(&mut self, value: T) https://doc.rust-lang.org/std/vec/struct.Vec.html#method.push https://doc.rust-lang.org/std/vec/struct.Vec.html#method.pus... In Rust for Linux their custom alloc library does not have that function offering instead Vec::try_push(&mut self, value: T) -> Result<(), TryReserveError> https://rust-for-linux.github.io/docs/alloc/vec/struct.Vec.html#method.try_push https://rust-for-linux.github.io/docs/alloc/vec/struct.Vec.h...
- G4E 4y agoHaving third party crates in ring 0 is kind of terrifiying when you think about supply chain attacks...
- kibwen 4y agoSo few people actually do their development/testing in truly-sandboxed environments that I don't think that there's actually much of a difference in practice between malicious code running in userspace vs malicious running in kernelspace ( https://xkcd.com/1200/ https://xkcd.com/1200/ ). Of course, I'd love if sandboxed dev environments became more usable and widespread.
- rowanG077 4y agoI would never expect them to do that without vendoring the package. Which mitigates that risk.
- esarbe 4y agoThey don't use cargo.
- sanxiyn 4y agoI expect it to be possible in near future that you develop, say, a web server in Rust in userspace, and with a few changes run it in kernelspace, with improved performance.
- ilovecaching 4y agoWe already have eBPF.
- intelVISA 4y agoio_uring too
- tga_d 4y agoRunning things in kernel space doesn't make them faster.
- insanitybit 4y agoIt can absolutely make things faster. If your program is running on data that exists in the kernel you don't need to copy it over to userland/ perform expensive context switching.
- tga_d 4y agoIt can make things faster is not the same as simply running it in the kernel will make it faster. There are definitely certain situations that making use of fewer context switches helps with, and where page mapping isn't sufficient, but if you just write an entire user-space web server, then just compile it as a kernel module, expecting it to see a notable performance improvement, you're going to be disappointed. edit to clarify: the reason it won't help (much) is because most of the things it would help with have already been optimized via features like CoW pages, the VDSO, sendfile, etc.
- insanitybit 4y ago
- LAC-Tech 4y agoalthough it never really sacrifices any control. It feels like it sacrifices control wrt when you allocate and free-memory. I understand it's deterministic in rust, but it's not explicitly controllable - or at least I've seen it be.
- loeg 4y agoYou have to bring your own memory in no-std. Implicit alloc is provided by the alloc crate (part of std).
- Others 4y agoI don’t follow. It’s exactly like C in this regard, except with Box::new instead of malloc.
- conradev 4y agoThe key difference as outlined in the article is that Box::new cannot fail in Rust, but the underlying malloc can. In userspace it is rare enough that a Rust panic is considered acceptable behavior, but this is not the case for the kernel. But, it is controllable for sure
- tialaramex 4y agoUnsurprisingly the kernel's Box does not have Box::new it provides instead Box::try_new which is fallible.
- kaba0 4y agoRust is a manual memory managed language, it literally has nothing inexpressible that C could do. Hell, C is a “higher level” language in that it doesn’t have any way for SIMD instructions. To add something concrete, you can manually free memory as well based on some dynamic property if you wish.