5 ms·

The Rust for Linux kernel project aims to enable writing Linux kernel device drivers in Rust. See: https://lwn.net/Articles/862018/ https://lwn.net/Articles/86

The Rust for Linux kernel project aims to enable writing Linux kernel device drivers in Rust. See: https://lwn.net/Articles/862018/ https://lwn.net/Articles/862018/ The issue of Rust's LLVM based compiler toolchain not targeting all CPU archs is intended to be solved by the gcc-rs project. See: https://lwn.net/Articles/871283/ https://lwn.net/Articles/871283/ I think the approach the Rust for Linux project is taking is wise: Not about outright re-writes but more about focusing on those subsystems where Rust's intrinsic safety and security properties helps the most.

maleldil 5y ago
Last I checked, one of the big problems was the Rust panics when you run out of memory, which is unacceptable in the kernel. Is there any progress on that?
- jerven 5y ago
  Yes a lot. https://rust-for-linux.github.io/docs/alloc/alloc/index.html https://rust-for-linux.github.io/docs/alloc/alloc/index.html
- tialaramex 5y ago
  Out of the box Rust doesn't provide any way to allocate heap memory, so, you can't run out if it. Your ordinary userspace apps use std (the standard library) which relies on the alloc crate, and that provides heap allocation which indeed panics if the allocation fails. Because it correctly guesses that your "clever" strategy to handle allocation failure actually isn't and will just triple fault anyway so it should cut to the chase. The kernel obviously doesn't have std, and Rust for Linux implements its own alloc crate. Linus' requirement that you can fail memory allocation just means all the calls in alloc that can actually allocate memory now return Result to indicate whether the allocation was successful. This isn't how you'd do things in userspace, but, this isn't userspace so fine.
  - abendstolz 5y ago
    Can you elaborate as to why "his isn't how you'd do things in userspace, but, this isn't userspace so fine" holds? Naive me - not a kernel dev at all - would argue that returning Result<Memory, AllocationError> is always better, even for userspace because it would allow me to additionally log something or gracefully deal with this. Even if I don't want to deal with it, I could just `.unwrap()` or `.expect('my error message')` it. Note: I am not trying to be snarky here, I genuinely don't know and would like to. If answering this is too complex, maybe you can point me in the right direction so I can ask the right questions to find answers myself? Thanks in any case!
    - cormacrelf 5y ago
      Tialaramex answered this in their post already, and you almost answered the question yourself: > I could just .unwrap() or .expect('my error message') it. Panicking can allocate. Allocating can fail. Failing can panic. Panicking can allocate. Allocating can fail. You can bite yourself in the ass like a real Ourobouros. IMO, a prerequisite to using fallible allocation APIs should be attempting to write your own allocator, handling the weird and wacky problem of initialising a data structure (for the heap) in such a way that if it fails, it fails without allocating but leaves some hint as to what went wrong.
      - abendstolz 5y ago
        Oh, wow, I was under the impression that the error message would be stack only, no heap involved, but as Result is part of the std library and not of core, this totally makes sense. So for `Rust for Linux` they also need to implement a `Result-like` type that is stack only based to solve this issue, right? If so, cool, thanks, you just made my day by tickling my learning nerves! :)
        cormacrelf 5y ago
        It has nothing to do with Result, whatsoever. Result does not allocate. If you used a Result that way, you could certainly try to "gracefully" handle the allocation failure, but if you think it would be easy, you would be wrong. As Tialaramex said, you are probably just going to make the problem worse because it is very difficult to ensure you do not attempt to allocate during allocation-failure-recovery. Rustc doesn't and can't really check this for you. It actually has to do with `panic!(...)`. When you use `unwrap()`/`expect("...")`, you use the panic macro under the hood; parts of the panicking infrastructure use a boxed trait object which could contain a static string or formatted String or anything else really. The box can allocate if it is not a ZST. I believe the alloc crate's default handler tries to avoid this kind of thing, so that it can't fail to allocate AGAIN in the failure-handling routine. It will likely do a better job than you could. This is a live issue at the moment, so to go into any more detail I'd have to read a bunch of recent Rust issues/PRs.
- geofft 5y ago
  Yes, this is called "fallible allocations." You add methods with the "try_" prefix that work like the existing methods, except they return a Result which fails if it's out of memory instead of panicking. We have a light / temporary fork of the Rust stdlib allocator with fallible allocation support: https://github.com/Rust-for-Linux/linux/tree/rust/rust/alloc https://github.com/Rust-for-Linux/linux/tree/rust/rust/alloc See e.g. https://github.com/Rust-for-Linux/linux/commit/487d7578bd036ba57b0998d7af69e0bdcc595da8 https://github.com/Rust-for-Linux/linux/commit/487d7578bd036...
- phkahler 5y ago
  >> one of the big problems was the Rust panics when you run out of memory, which is unacceptable in the kernel. If the kernel is running out of memory, IMHO that's a bug. The kernel is ultimately responsible for memory management right? It needs to prioritize itself over everything else or the system is in trouble.
  - esclerofilo 5y ago
    This sounds like a design choice that was made decades ago and permeated everything else in Linux. I don't think it's changeable at this point, if they wanted to.

maleldil 5y ago

Last I checked, one of the big problems was the Rust panics when you run out of memory, which is unacceptable in the kernel. Is there any progress on that?

jerven 5y ago
Yes a lot. https://rust-for-linux.github.io/docs/alloc/alloc/index.html https://rust-for-linux.github.io/docs/alloc/alloc/index.html
tialaramex 5y ago
Out of the box Rust doesn't provide any way to allocate heap memory, so, you can't run out if it. Your ordinary userspace apps use std (the standard library) which relies on the alloc crate, and that provides heap allocation which indeed panics if the allocation fails. Because it correctly guesses that your "clever" strategy to handle allocation failure actually isn't and will just triple fault anyway so it should cut to the chase. The kernel obviously doesn't have std, and Rust for Linux implements its own alloc crate. Linus' requirement that you can fail memory allocation just means all the calls in alloc that can actually allocate memory now return Result to indicate whether the allocation was successful. This isn't how you'd do things in userspace, but, this isn't userspace so fine.
- abendstolz 5y ago
  Can you elaborate as to why "his isn't how you'd do things in userspace, but, this isn't userspace so fine" holds? Naive me - not a kernel dev at all - would argue that returning Result<Memory, AllocationError> is always better, even for userspace because it would allow me to additionally log something or gracefully deal with this. Even if I don't want to deal with it, I could just `.unwrap()` or `.expect('my error message')` it. Note: I am not trying to be snarky here, I genuinely don't know and would like to. If answering this is too complex, maybe you can point me in the right direction so I can ask the right questions to find answers myself? Thanks in any case!
  - cormacrelf 5y ago
    Tialaramex answered this in their post already, and you almost answered the question yourself: > I could just .unwrap() or .expect('my error message') it. Panicking can allocate. Allocating can fail. Failing can panic. Panicking can allocate. Allocating can fail. You can bite yourself in the ass like a real Ourobouros. IMO, a prerequisite to using fallible allocation APIs should be attempting to write your own allocator, handling the weird and wacky problem of initialising a data structure (for the heap) in such a way that if it fails, it fails without allocating but leaves some hint as to what went wrong.
    - abendstolz 5y ago
      Oh, wow, I was under the impression that the error message would be stack only, no heap involved, but as Result is part of the std library and not of core, this totally makes sense. So for `Rust for Linux` they also need to implement a `Result-like` type that is stack only based to solve this issue, right? If so, cool, thanks, you just made my day by tickling my learning nerves! :)
      - cormacrelf 5y ago
        It has nothing to do with Result, whatsoever. Result does not allocate. If you used a Result that way, you could certainly try to "gracefully" handle the allocation failure, but if you think it would be easy, you would be wrong. As Tialaramex said, you are probably just going to make the problem worse because it is very difficult to ensure you do not attempt to allocate during allocation-failure-recovery. Rustc doesn't and can't really check this for you. It actually has to do with `panic!(...)`. When you use `unwrap()`/`expect("...")`, you use the panic macro under the hood; parts of the panicking infrastructure use a boxed trait object which could contain a static string or formatted String or anything else really. The box can allocate if it is not a ZST. I believe the alloc crate's default handler tries to avoid this kind of thing, so that it can't fail to allocate AGAIN in the failure-handling routine. It will likely do a better job than you could. This is a live issue at the moment, so to go into any more detail I'd have to read a bunch of recent Rust issues/PRs.
geofft 5y ago
Yes, this is called "fallible allocations." You add methods with the "try_" prefix that work like the existing methods, except they return a Result which fails if it's out of memory instead of panicking. We have a light / temporary fork of the Rust stdlib allocator with fallible allocation support: https://github.com/Rust-for-Linux/linux/tree/rust/rust/alloc https://github.com/Rust-for-Linux/linux/tree/rust/rust/alloc See e.g. https://github.com/Rust-for-Linux/linux/commit/487d7578bd036ba57b0998d7af69e0bdcc595da8 https://github.com/Rust-for-Linux/linux/commit/487d7578bd036...
phkahler 5y ago
>> one of the big problems was the Rust panics when you run out of memory, which is unacceptable in the kernel. If the kernel is running out of memory, IMHO that's a bug. The kernel is ultimately responsible for memory management right? It needs to prioritize itself over everything else or the system is in trouble.
- esclerofilo 5y ago
  This sounds like a design choice that was made decades ago and permeated everything else in Linux. I don't think it's changeable at this point, if they wanted to.