6 ms·

> Personally, I'm very excited about Rust in the kernel. Why?

gpm 6y ago
Speaking for myself and not the person you replied to - It would make me 10x more likely to work on the kernel, just because I enjoy programming in rust more than I enjoy programming in C. (At least given my current employment, any kernel contributions would be on my own time) - It would give me more faith in the security of other code people are contributing, things like binder in C strike me as pretty scary components of android from a security perspective, I'm much more comfortable running the same thing written in Rust. - I think it would generally reduce the number of kernel bugs in components written in it. Kernel bugs are rare, but and very frustrating when you encounter them. - I think it would increase the general productivity in kernel development. A better kernel helps everyone out. - It acts as validation for rust as a language. Obviously the kernel people shouldn't care about this in the slightest and it's not an argument for including it. However if it is included for other reasons (see above), it does help me argue that "rust would be a good fit for x" in other situations.
- caslon 6y ago
  >- I think it would increase the general productivity in kernel development. A better kernel helps everyone out. Don't you think the massive increase in compilation time would negate any productivity gains and probably decrease productivity overall?
  - gpm 6y ago
    No, because - I don't think it will be massive. - I especially don't think it will be that large for incremental builds, which is the main thing that matters. (But I'm not involved in this project, so I don't actually know how incremental the builds are...) - My experience is that the vast majority of programming time is spent fixing mistakes, not compiling. Rust reduces the amount of time spent fixing mistakes a lot more than it increases time spent compiling. - Rust moves many errors early in the compilation process (instead of when you try and test your code), which reduces iteration time instead of increasing it. I'm not involved in this project, but I imagine it's at a point where you could get some numbers for the fixed overhead that adding rust adds to the compile times. I'd be interested in seeing those numbers.
  - staticassertion 6y ago
    Have you compiled the Linux kernel before? I doubt Rust will be the bottleneck, it's a massive project - a tiny fraction being in Rust will be a blip.
    - caslon 6y ago
      Pretty frequently, yes. It doesn't take that long on modern, moderately-powered devices. Even when I was using a mid-tier device from a decade ago, compilation time was still around half an hour, which was less than most Rust projects I've encountered are on modern and reasonably high-end hardware, despite doing much more and being much larger.
      - staticassertion 6y ago
        Yep, that sounds exactly right to me - about 30 minutes on older hardware. That's very odd to me that you have 30 minute rust build times - as someone who works on a rust project professionally, with 10KLOC, that isn't my experience at all. If Rust gets into the kernel I would expect it to account for <1% of the code, so even if it were 100x slower to compile, which it isn't, I don't see it having an impact.
        bluecalm 6y ago
        Man, 10KLOC is a very small project. Obviously it compiles quickly. Linux kernel is almost 30 million lines of code.
        staticassertion 6y ago
        Yes... exactly. The Linux Kernel is 30 million lines of code - so how exactly will some minimiscule-by-comparison amount of Rust code slow down compile times considerably?
        bluecalm 6y ago
        If it's always going to be miniscule then why bother? If it has potential to grow to something not miniscule then it's important to it compiles quickly.
  - steveklabnik 6y ago
    "Productivity" is notoriously hard to measure in software. While slow compile times may slow you down, and hence reduce your productivity, if the compiler prevents hard to fix bugs, it still may increase your overall productivity. Consider things like https://hacks.mozilla.org/2021/04/eliminating-data-races-in-firefox-a-technical-report/ https://hacks.mozilla.org/2021/04/eliminating-data-races-in-... > Overall Rust appears to be fulfilling one of its original design goals: allowing us to write more concurrent code safely. Both WebRender and Stylo are very large and pervasively multi-threaded, but have had minimal threading issues. What issues we did find were mistakes in the implementations of low-level and explicitly unsafe multithreading abstractions — and those mistakes were simple to fix. > > This is in contrast to many of our C++ races, which often involved things being randomly accessed on different threads with unclear semantics, necessitating non-trivial refactorings of the code. Maybe the C++ was faster to compile, but in the end, fixing these issues took more time. There were more of them, and they were harder to track down. Nobody truly knows the answers to these questions in the general case yet, of course. My point is just that "slow compile == bad productivity" is not inherently true. Faster compile times are, of course, always desired no matter what.
    - diragon 6y ago
      > "Productivity" is notoriously hard to measure in software. We can look at history and results. In these terms, C (and perhaps C++ to some extent) is, I believe, the only productive programming language for making the low level parts of non-experimental kernels. As far as it can be publically measured, Rust so far has proven itself for application programming in a certain niche and not much more. It would be cool if it could prove itself in kernel space too -- we certainly need less system crashes caused by bad kernel-level code. Curiously though, it has been a very long time since I've last bumped into such a thing in Linux. This makes me suspect that Rust is trying to fix a problem here that's already been fixed in another way. As for ease of coding, C seems like a massively easier language to learn than Rust. But I might be wrong there. Any data about that, I wonder?
      - steveklabnik 6y ago
        Yes, that is true. But the only way to get that data is to do it. This work is one part of doing that. Someone has to be first :) (There are of course a ton of kernel-level things in Rust that don't pass the "non-experimental" bar for various people. As always, depends on exactly what you mean.) > Any data about that, I wonder? Possibly one of the only things harder to measure than productivity is ease of learning, haha! I had programmed in C for decades before Rust even existed. We do have a lot of people who say that they think Rust was easier to learn for them than C was. And of course many who believe the opposite. Not sure anything is conclusive in any direction. For example, it's quite possible that some people find C easier, and some people find Rust easier, and there will never be a clear winner. Time will tell.
  - E14n 6y ago
    If you are talking speed to production ready code then rust is really productive. The rust tooling picks up a lot of errors and leads you to spending more time fixing coding issues rather then compile to test your code.
- alfiedotwtf 6y ago
  > It would make me 10x more likely to work on the kernel, just because I enjoy programming in rust more than I enjoy programming in C That sound was a thousand people nodding in unison
  - guenthert 6y ago
    And it would be a million people if it were Python. That doesn't make it a good idea to encourage submissions to the kernel in Python. There is much more to kernel programming than the programming language. C might be inconvenient and lacking expressive power, but if people can't handle pointers and goto, what mess are they going to make with different address spaces and interrupts?
    - tialaramex 6y ago
      > C might be inconvenient and lacking expressive power, but if people can't handle pointers and goto, what mess are they going to make with different address spaces and interrupts? I have terrible news for you. People (well, humans, and I don't see anybody else signing up to maintain Linux) cannot in fact correctly handle pointers and goto. That's why they keep making mistakes. It's actually to be hoped that Rust can usefully express constraints it has today to prevent some of those problems onto things like address spaces. It'd be great if say, driver code which can confuse a virtual address with a physical one just won't compile rather than compiling and then mysteriously not working as expected or causing occasionally BUG() reports.
    - alfiedotwtf 6y ago
      > but if people can't handle pointers and goto Mitre shows that in fact people can't handle pointers and goto
- utxaa 6y ago
  what about interacting with the existing kernel code base? would data coming back from the kernel into rust space need to be wrapped to provide safety guarantees? or would it be necessary to turn safety features off? a bit confused about this.
  - steveklabnik 6y ago
    That is what this patch series is about. It will require using unsafe code to some degree, yes.
  - gpm 6y ago
    So... it doesn't really impact this discussion other than "shouldn't be an issue". I'll try and give a summary of what's happening technically, but HN is frankly the wrong form for a "how to use the C ffi in rust" tutorial. Also a big disclaimer that I don't know what precisely this project is doing, so I'm talking about C/Rust projects in general. Rust talks the C ffi really well. It can call external functions that follow the C abi the same way it can call native unsafe functions [1]. You can tell it to layout a struct the same way that C does, etc. Because calling the C abi requires unsafe code, it's common to provide wrappers around the C abi that are safe against missuse. I.e. that make it so that the only way to call C functions is the correct way. This is doing things like making it so the only way to get a `struct` is to call a (safe) `new` function that calls the (unsafe) C initializer internally, and exposing the C "methods" on that struct (that expect it to be initialized) as safe methods on the rust struct that internally call the unsafe C functions (and they can do so because they know the struct has been initialized). Obviously for any particular C api you have to look at what it requires to be called safely, and then figure out how to encode it in the type system, but that's usually surprisingly easy. Calling rust from C doesn't really require any "unsafe" code (other than the fact that C is basically a giant unsafe block by nature), because the assertion that you're calling it correctly happens on the C side of things, not the rust side of things. Just like rust can call C abi functions, rust can make it's functions follow the C abi by simply saying extern "C" fn foo() instead of fn foo() But many of the data structures you might pass from C to rust will need a wrapper to use "safely". E.g. if I pass a doubly linked list, it's going to need raw pointers more or less by nature (at least if rust wants to be able to mutate it), and someone is going to need to do a similar wrapping thing where they expose some functions that correctly work with the list, that internally use unsafe, but expose a safe api. [1] So what unsafe here means that the compiler doesn't know that the function is safe to call, so you have to tell it "I checked and how I'm using it is fine" by putting the call inside an unsafe block. This looks like the following. Note that you can also have unsafe native rust functions (e.g. if you want to index an array without checking the array bounds that's an unsafe function implemented in rust) unsafe { c_function_here(arg1, arg2) }
- brundolf 6y ago
  > It would make me 10x more likely to work on the kernel Seconding this. I don't think I would ever touch the Linux kernel with a twenty-foot-pole in C. In Rust? Maybe I'd give it a try and contribute one day. Knowing the exact set of constraints on my code, having them enforced, not having to hold the entire system in my head just to avoid a catastrophic memory error. These things make contributing to a project - especially one as incomprehensibly large and complex as the Linux kernel - orders of magnitude easier.
  - dwaite 6y ago
    its not that bad writing kernel code - just remember if you get anything wrong you'll hose your filesystem beyond repair ;-)
- darthrupert 6y ago
  How much is 10 x 0? ;)
  - orestarod 6y ago
    What makes you assume the probability of contributing is 0 though

gpm 6y ago

Speaking for myself and not the person you replied to - It would make me 10x more likely to work on the kernel, just because I enjoy programming in rust more than I enjoy programming in C. (At least given my current employment, any kernel contributions would be on my own time) - It would give me more faith in the security of other code people are contributing, things like binder in C strike me as pretty scary components of android from a security perspective, I'm much more comfortable running the same thing written in Rust. - I think it would generally reduce the number of kernel bugs in components written in it. Kernel bugs are rare, but and very frustrating when you encounter them. - I think it would increase the general productivity in kernel development. A better kernel helps everyone out. - It acts as validation for rust as a language. Obviously the kernel people shouldn't care about this in the slightest and it's not an argument for including it. However if it is included for other reasons (see above), it does help me argue that "rust would be a good fit for x" in other situations.

caslon 6y ago
>- I think it would increase the general productivity in kernel development. A better kernel helps everyone out. Don't you think the massive increase in compilation time would negate any productivity gains and probably decrease productivity overall?
- gpm 6y ago
  No, because - I don't think it will be massive. - I especially don't think it will be that large for incremental builds, which is the main thing that matters. (But I'm not involved in this project, so I don't actually know how incremental the builds are...) - My experience is that the vast majority of programming time is spent fixing mistakes, not compiling. Rust reduces the amount of time spent fixing mistakes a lot more than it increases time spent compiling. - Rust moves many errors early in the compilation process (instead of when you try and test your code), which reduces iteration time instead of increasing it. I'm not involved in this project, but I imagine it's at a point where you could get some numbers for the fixed overhead that adding rust adds to the compile times. I'd be interested in seeing those numbers.
- staticassertion 6y ago
  Have you compiled the Linux kernel before? I doubt Rust will be the bottleneck, it's a massive project - a tiny fraction being in Rust will be a blip.
  - caslon 6y ago
    Pretty frequently, yes. It doesn't take that long on modern, moderately-powered devices. Even when I was using a mid-tier device from a decade ago, compilation time was still around half an hour, which was less than most Rust projects I've encountered are on modern and reasonably high-end hardware, despite doing much more and being much larger.
    - staticassertion 6y ago
      Yep, that sounds exactly right to me - about 30 minutes on older hardware. That's very odd to me that you have 30 minute rust build times - as someone who works on a rust project professionally, with 10KLOC, that isn't my experience at all. If Rust gets into the kernel I would expect it to account for <1% of the code, so even if it were 100x slower to compile, which it isn't, I don't see it having an impact.
      - bluecalm 6y ago
        Man, 10KLOC is a very small project. Obviously it compiles quickly. Linux kernel is almost 30 million lines of code.
        staticassertion 6y ago
        Yes... exactly. The Linux Kernel is 30 million lines of code - so how exactly will some minimiscule-by-comparison amount of Rust code slow down compile times considerably?
        bluecalm 6y ago
        If it's always going to be miniscule then why bother? If it has potential to grow to something not miniscule then it's important to it compiles quickly.
- steveklabnik 6y ago
  "Productivity" is notoriously hard to measure in software. While slow compile times may slow you down, and hence reduce your productivity, if the compiler prevents hard to fix bugs, it still may increase your overall productivity. Consider things like https://hacks.mozilla.org/2021/04/eliminating-data-races-in-firefox-a-technical-report/ https://hacks.mozilla.org/2021/04/eliminating-data-races-in-... > Overall Rust appears to be fulfilling one of its original design goals: allowing us to write more concurrent code safely. Both WebRender and Stylo are very large and pervasively multi-threaded, but have had minimal threading issues. What issues we did find were mistakes in the implementations of low-level and explicitly unsafe multithreading abstractions — and those mistakes were simple to fix. > > This is in contrast to many of our C++ races, which often involved things being randomly accessed on different threads with unclear semantics, necessitating non-trivial refactorings of the code. Maybe the C++ was faster to compile, but in the end, fixing these issues took more time. There were more of them, and they were harder to track down. Nobody truly knows the answers to these questions in the general case yet, of course. My point is just that "slow compile == bad productivity" is not inherently true. Faster compile times are, of course, always desired no matter what.
  - diragon 6y ago
    > "Productivity" is notoriously hard to measure in software. We can look at history and results. In these terms, C (and perhaps C++ to some extent) is, I believe, the only productive programming language for making the low level parts of non-experimental kernels. As far as it can be publically measured, Rust so far has proven itself for application programming in a certain niche and not much more. It would be cool if it could prove itself in kernel space too -- we certainly need less system crashes caused by bad kernel-level code. Curiously though, it has been a very long time since I've last bumped into such a thing in Linux. This makes me suspect that Rust is trying to fix a problem here that's already been fixed in another way. As for ease of coding, C seems like a massively easier language to learn than Rust. But I might be wrong there. Any data about that, I wonder?
    - steveklabnik 6y ago
      Yes, that is true. But the only way to get that data is to do it. This work is one part of doing that. Someone has to be first :) (There are of course a ton of kernel-level things in Rust that don't pass the "non-experimental" bar for various people. As always, depends on exactly what you mean.) > Any data about that, I wonder? Possibly one of the only things harder to measure than productivity is ease of learning, haha! I had programmed in C for decades before Rust even existed. We do have a lot of people who say that they think Rust was easier to learn for them than C was. And of course many who believe the opposite. Not sure anything is conclusive in any direction. For example, it's quite possible that some people find C easier, and some people find Rust easier, and there will never be a clear winner. Time will tell.
- E14n 6y ago
  If you are talking speed to production ready code then rust is really productive. The rust tooling picks up a lot of errors and leads you to spending more time fixing coding issues rather then compile to test your code.
alfiedotwtf 6y ago
> It would make me 10x more likely to work on the kernel, just because I enjoy programming in rust more than I enjoy programming in C That sound was a thousand people nodding in unison
- guenthert 6y ago
  And it would be a million people if it were Python. That doesn't make it a good idea to encourage submissions to the kernel in Python. There is much more to kernel programming than the programming language. C might be inconvenient and lacking expressive power, but if people can't handle pointers and goto, what mess are they going to make with different address spaces and interrupts?
  - tialaramex 6y ago
    > C might be inconvenient and lacking expressive power, but if people can't handle pointers and goto, what mess are they going to make with different address spaces and interrupts? I have terrible news for you. People (well, humans, and I don't see anybody else signing up to maintain Linux) cannot in fact correctly handle pointers and goto. That's why they keep making mistakes. It's actually to be hoped that Rust can usefully express constraints it has today to prevent some of those problems onto things like address spaces. It'd be great if say, driver code which can confuse a virtual address with a physical one just won't compile rather than compiling and then mysteriously not working as expected or causing occasionally BUG() reports.
  - alfiedotwtf 6y ago
    > but if people can't handle pointers and goto Mitre shows that in fact people can't handle pointers and goto
utxaa 6y ago
what about interacting with the existing kernel code base? would data coming back from the kernel into rust space need to be wrapped to provide safety guarantees? or would it be necessary to turn safety features off? a bit confused about this.
- steveklabnik 6y ago
  That is what this patch series is about. It will require using unsafe code to some degree, yes.
- gpm 6y ago
  So... it doesn't really impact this discussion other than "shouldn't be an issue". I'll try and give a summary of what's happening technically, but HN is frankly the wrong form for a "how to use the C ffi in rust" tutorial. Also a big disclaimer that I don't know what precisely this project is doing, so I'm talking about C/Rust projects in general. Rust talks the C ffi really well. It can call external functions that follow the C abi the same way it can call native unsafe functions [1]. You can tell it to layout a struct the same way that C does, etc. Because calling the C abi requires unsafe code, it's common to provide wrappers around the C abi that are safe against missuse. I.e. that make it so that the only way to call C functions is the correct way. This is doing things like making it so the only way to get a `struct` is to call a (safe) `new` function that calls the (unsafe) C initializer internally, and exposing the C "methods" on that struct (that expect it to be initialized) as safe methods on the rust struct that internally call the unsafe C functions (and they can do so because they know the struct has been initialized). Obviously for any particular C api you have to look at what it requires to be called safely, and then figure out how to encode it in the type system, but that's usually surprisingly easy. Calling rust from C doesn't really require any "unsafe" code (other than the fact that C is basically a giant unsafe block by nature), because the assertion that you're calling it correctly happens on the C side of things, not the rust side of things. Just like rust can call C abi functions, rust can make it's functions follow the C abi by simply saying extern "C" fn foo() instead of fn foo() But many of the data structures you might pass from C to rust will need a wrapper to use "safely". E.g. if I pass a doubly linked list, it's going to need raw pointers more or less by nature (at least if rust wants to be able to mutate it), and someone is going to need to do a similar wrapping thing where they expose some functions that correctly work with the list, that internally use unsafe, but expose a safe api. [1] So what unsafe here means that the compiler doesn't know that the function is safe to call, so you have to tell it "I checked and how I'm using it is fine" by putting the call inside an unsafe block. This looks like the following. Note that you can also have unsafe native rust functions (e.g. if you want to index an array without checking the array bounds that's an unsafe function implemented in rust) unsafe { c_function_here(arg1, arg2) }
brundolf 6y ago
> It would make me 10x more likely to work on the kernel Seconding this. I don't think I would ever touch the Linux kernel with a twenty-foot-pole in C. In Rust? Maybe I'd give it a try and contribute one day. Knowing the exact set of constraints on my code, having them enforced, not having to hold the entire system in my head just to avoid a catastrophic memory error. These things make contributing to a project - especially one as incomprehensibly large and complex as the Linux kernel - orders of magnitude easier.
- dwaite 6y ago
  its not that bad writing kernel code - just remember if you get anything wrong you'll hose your filesystem beyond repair ;-)
darthrupert 6y ago
How much is 10 x 0? ;)
- orestarod 6y ago
  What makes you assume the probability of contributing is 0 though