11 ms·

Writing a Linux-compatible kernel in Rust

167 points by signa11 5y ago

asicsp 5y ago
Related discussion from yesterday: "Kerla: Monolithic kernel in Rust, aiming for Linux ABI compatibility" https://news.ycombinator.com/item?id=28986229 https://news.ycombinator.com/item?id=28986229
_ikke_ 5y ago
Discussed extensively yesterday: https://news.ycombinator.com/item?id=28986229 https://news.ycombinator.com/item?id=28986229
DeathArrow 5y ago
Masterful plan for Rust to take over the World: step 1: assume some parts of the Linux kernel will be written in Rust step 2: start writing a new kernel which feature some of the parts which won't be written in Rust in the Linux kernel step 3: add parts from step 1 to step 2 step 4: ask the community to write the missing parts from step 2 step 5: ??? step 6: profit!
lionkor 5y ago
Slightly off-topic but since the author mentions it; I personally cannot help but feel like the strong push for rust in the kernel is overstepping some sort of boundary. Since the kernel started, you can't begin to count the number of "better than C" languages that appeared. Why, if C is so bad, wasnt one of the alternatives introduced to slowly replace C in the kernel years ago? Zig, D, Ada, ... they all offer massive benefits over C. The only reason there is a push for Rust so much harder than anything else can only be explained by the community and their almost aggressive spirit of "if its not written in Rust, it should be". No regard for software that works great, the engineers that put time and effort into it to make it secure and fast, if its not Rust it must be broken. Im excited to see the linux kernel improve even further, no matter what it takes, but it does rub me the wrong way and makes me stop and think a little bit when one particular language is held on such a gigantic pedestal.
- pjmlp 5y ago
  C was already bad when it appeared, the only reason it won, was UNIX being made available for a symbolic price with source code. Had AT&T been allowed to take commercial advantage of UNIX, we would probably be coding OSes in PL/I, BLISS, Mesa, Modula-2, Ada or some offspring of them.
  - laumars 5y ago
    I never understood how C took over the desktop space as well. Pascal was a way better language and already dominant for home computing back in the 80s. Then by the 90s everywhere was C and C++. I never quite understood how or why that happened.
    - jasode 5y ago
      >Pascal was a way better language and already dominant for home computing back in the 80s. Not sure what you mean there ("dominant"?) but in the 1980s, BASIC was the dominant language on home computers because that's what they included. Commodore VIC-20 & 64, Atari 400/800, Texas Instruments TI-99 TI-BASIC, and IBM home computers like the PC jr had IBM-BASIC which was licensed from Microsoft's GW-BASIC. Something like Turbo Pascal was only purchased by a minority of 1980s computer users. That's why 1980s home computer enthusiast magazines like Compute! published their code lists in BASIC: https://www.google.com/search?q=%22compute!%22+magazine+code+list&tbm=isch https://www.google.com/search?q=%22compute!%22+magazine+code... >I never understood how C took over the desktop space as well. I'm guessing it's because C Language was already popular on the commercial side because of UNIX implementations which spread into microcomputers. Microsoft dabbled in XENIX which was AT&T UNIX. In the 1980s when popular desktop programs such as DOS, Lotus-123, and WordPerfect migrated from pure assembly to <high_level_language> ... they all ended up choosing C instead of Pascal. Why did industry converge on C? Maybe because there were more C compilers for various platforms. (The portability and cross-platform angle.)
      - laumars 5y ago
        > Not sure what you mean there ("dominant"?) but in the 1980s, BASIC was the dominant language on home computers because that's what they included. That was late 70s / early 80s and the firmware was all assembly with most software being written in machine code because the speed of those BASIc interpreters left a lot to be desired. I’m talking later in the decade when operating systems became the norm. Early versions of Windows and Mac OS were written in Pascal.
simion314 5y ago
I would like to see more innovative kernels and not toy projects here on HN, and I mean kernels implemented by people with years of real world experience that bring new ideas (any dude can read some blog and book and badly re implement a mini subset of Linux)
- pbininda 5y ago
  Hello everybody out there using minix - I'm doing a (free) operating system (just a hobby, won't be big and professional like gnu) for 386(486) AT clones. This has been brewing since april, and is starting to get ready. Written by Linus Torvalds on Sunday, 25 August, 1991 in comp.os.minix
  - rvz 5y ago
    Eventually, a full OS or OS kernel written in Rust is going to compete with Linux whether if you like it or not, unless either the maintainers want to have fun fixing the same old C vulnerabilities and bugs for more years or they adopt Rust themselves. You may not like it, Linus or his maintainers may not like it, but that is where the future is heading. Downvoters: I know. The future is Rust and change is scary for something like Linux. Good luck finding mountains of use-after-frees or memory corruption vulnerabilities which fixing those in every part of the kernel is a losing battle that Rust has already eliminated in the first place.
  - simion314 5y ago
    You should then search and find all ShitApp made in Rust and post it on HN because the chance that it will be good in 15 years is not zero, probably 1/10^10^10^10 .
- hnarn 5y ago
  If you want a new kernel that is "not a toy project" then pay someone to write it.
  - simion314 5y ago
    What I want is that HN will have less toy stuff upvoted because the title contains a specific keyword. I will submit tomorrow my X11 re implementation in Rust, it is a thing I made in weekends, full of bugs , it is missing everything and I am not experienced in anything graphics related but I read half a book about X11 and a fvew Rust blogs and now I will be on HN first page and get tons of stars on GitHub. Long story short, HN should not be IMO a Rust forum where any toy or shit re-implementation is admitted but it is what it is, fanboys will up-vote anything until the Rust image gets completely destroyed.
Jnr 5y ago
> "[..] just for fun. It does NOT aim to replace the Linux kernel" Yeah, I have heard this "just a hobby, won't be big and professional like gnu" before. :) https://en.wikipedia.org/wiki/History_of_Linux https://en.wikipedia.org/wiki/History_of_Linux
- brabel 5y ago
  You show one example, but I can show you 1,000 others where it really didn't get anywhere, just look at a few GitHub profiles :D
deleted 5y ago
[deleted]
alberth 5y ago
When did Linus become ok with Rust in the kernel? Given that Rust can cause a panic, I thought he was adamantly against Rust for low level kernel support. https://lkml.org/lkml/2021/4/14/1099 https://lkml.org/lkml/2021/4/14/1099
- fbanon 5y ago
  >When did Linus become ok with Rust in the kernel? He didn't. But he knows politics.
- jgilias 5y ago
  Note that this submission is about a learning project where someone aims to write a (minimal?) Linux ABI compatible kernel. To directly answer your question though. It's not that 'Rust memory allocations' panic. Rather it was the case that allocations widely used when the initiative was started could panic. This is being solved by writing a library with non-panicking memory allocations to be used in kernel development [1]. However, I'm not involved with this in any way, so I may be wrong. 1. https://rust-for-linux.github.io/docs/alloc/ https://rust-for-linux.github.io/docs/alloc/
- _flux 5y ago
  Work is being done to get rid of fallible code, such as allocations: https://www.theregister.com/2021/07/05/rust_for_linux_kernel_project/ https://www.theregister.com/2021/07/05/rust_for_linux_kernel... Anyway, I don't see paniccing as something a Linux kernel must never ever do. After all, we are all familiar with kernel oopses and panics, so clearly the existing body code already does it some situations. Just have the Rust code do that, if it wasn't its current behavior. One can argue Rust is one way to get rid of those panics/oops by e.g. statically guaranteeing a pointer is not null after a certain point.
- moltonel3x 5y ago
  He (and other prominent kernel developers) has manifested his interest many times, but is taking a strict "wait and see" approach (as you would expect from any big project manager). Whether it happens or not, Rust has gotten closer to inclusion in Linux than any other language.
blackgoose821 5y ago
Wouldn't writing a new kernel cause unexpected errors?
DrBazza 5y ago
Where doesn't Rust 'compete' with C, and where is C the better choice in a kernel? Sometimes an OS rewrite, even in a different language, can be faster for some tasks (IIRC Singularity and/or Midori outperform(ed) Windows on some tasks, but I can't find the link).
- xondono 5y ago
  > Where doesn't Rust 'compete' with C, and where is C the better choice in a kernel? My opinion as someone who writes C for embedded devices and has started to play with rust: C is better than Rust primarily when the C code already exists and the Rust code doesn’t. Rewriting the kernel will probably have little impact in terms of performance for the user, the main benefits would be for developers, because Rust can be more expressive (less boilerplate and simpler ways to do some things), and it should have less bugs, since it’s performing more complex checks than C code would. For me as an individual, I’d prefer to be writing Rust, I find it much more ergonomic (for my workflow at least). The main drawback I have is that no one else in my team really knows Rust (although that’s slowly changing).
  - DrBazza 5y ago
    I'm wondering more about accessing hardware primitives - if you spend a dozen lines marshalling unsafe code around and ultimately writing the equivalent C code in Rust, and slightly slower (perhaps the compiler optimizes anyway?), is it worth the effort?
    - xondono 5y ago
      > if you spend a dozen lines marshalling unsafe code around and ultimately writing the equivalent C code in Rust, and slightly slower (perhaps the compiler optimizes anyway?), is it worth the effort? The thing is, that's not generally where most of the bugs and development blockers lie here. When embedded development gets stuck, is generally because: a) A part of the hardware doesn't work like you think it works. b) Two or more independent parts of code (think ISRs) are conflicting in some non trivial way. Having well structured HALs helps with (a), while memory safety can prevent some errors on (b). One of the nice things I've seen with Rust is how high quality the HALs look, even if some of the tooling looks a little shaky for my taste. Of course part of that is that manufacturer provided HALs are generally quite bad... Also having lots of people share the same unsafe blocks is the best way to keep the bugs away, and sharing libraries with cargo beats the real way C code gets shared in embedded development (mostly through copy/paste from StackOverflow...).

167 points by signa11 5y ago

Related discussion from yesterday: "Kerla: Monolithic kernel in Rust, aiming for Linux ABI compatibility" https://news.ycombinator.com/item?id=28986229 https://news.ycombinator.com/item?id=28986229

_ikke_ 5y ago

Discussed extensively yesterday: https://news.ycombinator.com/item?id=28986229 https://news.ycombinator.com/item?id=28986229

DeathArrow 5y ago

Masterful plan for Rust to take over the World: step 1: assume some parts of the Linux kernel will be written in Rust step 2: start writing a new kernel which feature some of the parts which won't be written in Rust in the Linux kernel step 3: add parts from step 1 to step 2 step 4: ask the community to write the missing parts from step 2 step 5: ??? step 6: profit!

lionkor 5y ago

Slightly off-topic but since the author mentions it; I personally cannot help but feel like the strong push for rust in the kernel is overstepping some sort of boundary. Since the kernel started, you can't begin to count the number of "better than C" languages that appeared. Why, if C is so bad, wasnt one of the alternatives introduced to slowly replace C in the kernel years ago? Zig, D, Ada, ... they all offer massive benefits over C. The only reason there is a push for Rust so much harder than anything else can only be explained by the community and their almost aggressive spirit of "if its not written in Rust, it should be". No regard for software that works great, the engineers that put time and effort into it to make it secure and fast, if its not Rust it must be broken. Im excited to see the linux kernel improve even further, no matter what it takes, but it does rub me the wrong way and makes me stop and think a little bit when one particular language is held on such a gigantic pedestal.

pjmlp 5y ago
C was already bad when it appeared, the only reason it won, was UNIX being made available for a symbolic price with source code. Had AT&T been allowed to take commercial advantage of UNIX, we would probably be coding OSes in PL/I, BLISS, Mesa, Modula-2, Ada or some offspring of them.
- laumars 5y ago
  I never understood how C took over the desktop space as well. Pascal was a way better language and already dominant for home computing back in the 80s. Then by the 90s everywhere was C and C++. I never quite understood how or why that happened.
  - jasode 5y ago
    >Pascal was a way better language and already dominant for home computing back in the 80s. Not sure what you mean there ("dominant"?) but in the 1980s, BASIC was the dominant language on home computers because that's what they included. Commodore VIC-20 & 64, Atari 400/800, Texas Instruments TI-99 TI-BASIC, and IBM home computers like the PC jr had IBM-BASIC which was licensed from Microsoft's GW-BASIC. Something like Turbo Pascal was only purchased by a minority of 1980s computer users. That's why 1980s home computer enthusiast magazines like Compute! published their code lists in BASIC: https://www.google.com/search?q=%22compute!%22+magazine+code+list&tbm=isch https://www.google.com/search?q=%22compute!%22+magazine+code... >I never understood how C took over the desktop space as well. I'm guessing it's because C Language was already popular on the commercial side because of UNIX implementations which spread into microcomputers. Microsoft dabbled in XENIX which was AT&T UNIX. In the 1980s when popular desktop programs such as DOS, Lotus-123, and WordPerfect migrated from pure assembly to <high_level_language> ... they all ended up choosing C instead of Pascal. Why did industry converge on C? Maybe because there were more C compilers for various platforms. (The portability and cross-platform angle.)
    - laumars 5y ago
      > Not sure what you mean there ("dominant"?) but in the 1980s, BASIC was the dominant language on home computers because that's what they included. That was late 70s / early 80s and the firmware was all assembly with most software being written in machine code because the speed of those BASIc interpreters left a lot to be desired. I’m talking later in the decade when operating systems became the norm. Early versions of Windows and Mac OS were written in Pascal.

simion314 5y ago

I would like to see more innovative kernels and not toy projects here on HN, and I mean kernels implemented by people with years of real world experience that bring new ideas (any dude can read some blog and book and badly re implement a mini subset of Linux)

pbininda 5y ago
Hello everybody out there using minix - I'm doing a (free) operating system (just a hobby, won't be big and professional like gnu) for 386(486) AT clones. This has been brewing since april, and is starting to get ready. Written by Linus Torvalds on Sunday, 25 August, 1991 in comp.os.minix
- rvz 5y ago
  Eventually, a full OS or OS kernel written in Rust is going to compete with Linux whether if you like it or not, unless either the maintainers want to have fun fixing the same old C vulnerabilities and bugs for more years or they adopt Rust themselves. You may not like it, Linus or his maintainers may not like it, but that is where the future is heading. Downvoters: I know. The future is Rust and change is scary for something like Linux. Good luck finding mountains of use-after-frees or memory corruption vulnerabilities which fixing those in every part of the kernel is a losing battle that Rust has already eliminated in the first place.
- simion314 5y ago
  You should then search and find all ShitApp made in Rust and post it on HN because the chance that it will be good in 15 years is not zero, probably 1/10^10^10^10 .
hnarn 5y ago
If you want a new kernel that is "not a toy project" then pay someone to write it.
- simion314 5y ago
  What I want is that HN will have less toy stuff upvoted because the title contains a specific keyword. I will submit tomorrow my X11 re implementation in Rust, it is a thing I made in weekends, full of bugs , it is missing everything and I am not experienced in anything graphics related but I read half a book about X11 and a fvew Rust blogs and now I will be on HN first page and get tons of stars on GitHub. Long story short, HN should not be IMO a Rust forum where any toy or shit re-implementation is admitted but it is what it is, fanboys will up-vote anything until the Rust image gets completely destroyed.

Jnr 5y ago

> "[..] just for fun. It does NOT aim to replace the Linux kernel" Yeah, I have heard this "just a hobby, won't be big and professional like gnu" before. :) https://en.wikipedia.org/wiki/History_of_Linux https://en.wikipedia.org/wiki/History_of_Linux

brabel 5y ago
You show one example, but I can show you 1,000 others where it really didn't get anywhere, just look at a few GitHub profiles :D

deleted 5y ago

[deleted]

alberth 5y ago

When did Linus become ok with Rust in the kernel? Given that Rust can cause a panic, I thought he was adamantly against Rust for low level kernel support. https://lkml.org/lkml/2021/4/14/1099 https://lkml.org/lkml/2021/4/14/1099

fbanon 5y ago
>When did Linus become ok with Rust in the kernel? He didn't. But he knows politics.
jgilias 5y ago
Note that this submission is about a learning project where someone aims to write a (minimal?) Linux ABI compatible kernel. To directly answer your question though. It's not that 'Rust memory allocations' panic. Rather it was the case that allocations widely used when the initiative was started could panic. This is being solved by writing a library with non-panicking memory allocations to be used in kernel development [1]. However, I'm not involved with this in any way, so I may be wrong. 1. https://rust-for-linux.github.io/docs/alloc/ https://rust-for-linux.github.io/docs/alloc/
_flux 5y ago
Work is being done to get rid of fallible code, such as allocations: https://www.theregister.com/2021/07/05/rust_for_linux_kernel_project/ https://www.theregister.com/2021/07/05/rust_for_linux_kernel... Anyway, I don't see paniccing as something a Linux kernel must never ever do. After all, we are all familiar with kernel oopses and panics, so clearly the existing body code already does it some situations. Just have the Rust code do that, if it wasn't its current behavior. One can argue Rust is one way to get rid of those panics/oops by e.g. statically guaranteeing a pointer is not null after a certain point.
moltonel3x 5y ago
He (and other prominent kernel developers) has manifested his interest many times, but is taking a strict "wait and see" approach (as you would expect from any big project manager). Whether it happens or not, Rust has gotten closer to inclusion in Linux than any other language.

blackgoose821 5y ago

Wouldn't writing a new kernel cause unexpected errors?

DrBazza 5y ago

Where doesn't Rust 'compete' with C, and where is C the better choice in a kernel? Sometimes an OS rewrite, even in a different language, can be faster for some tasks (IIRC Singularity and/or Midori outperform(ed) Windows on some tasks, but I can't find the link).

xondono 5y ago
> Where doesn't Rust 'compete' with C, and where is C the better choice in a kernel? My opinion as someone who writes C for embedded devices and has started to play with rust: C is better than Rust primarily when the C code already exists and the Rust code doesn’t. Rewriting the kernel will probably have little impact in terms of performance for the user, the main benefits would be for developers, because Rust can be more expressive (less boilerplate and simpler ways to do some things), and it should have less bugs, since it’s performing more complex checks than C code would. For me as an individual, I’d prefer to be writing Rust, I find it much more ergonomic (for my workflow at least). The main drawback I have is that no one else in my team really knows Rust (although that’s slowly changing).
- DrBazza 5y ago
  I'm wondering more about accessing hardware primitives - if you spend a dozen lines marshalling unsafe code around and ultimately writing the equivalent C code in Rust, and slightly slower (perhaps the compiler optimizes anyway?), is it worth the effort?
  - xondono 5y ago
    > if you spend a dozen lines marshalling unsafe code around and ultimately writing the equivalent C code in Rust, and slightly slower (perhaps the compiler optimizes anyway?), is it worth the effort? The thing is, that's not generally where most of the bugs and development blockers lie here. When embedded development gets stuck, is generally because: a) A part of the hardware doesn't work like you think it works. b) Two or more independent parts of code (think ISRs) are conflicting in some non trivial way. Having well structured HALs helps with (a), while memory safety can prevent some errors on (b). One of the nice things I've seen with Rust is how high quality the HALs look, even if some of the tooling looks a little shaky for my taste. Of course part of that is that manufacturer provided HALs are generally quite bad... Also having lots of people share the same unsafe blocks is the best way to keep the bugs away, and sharing libraries with cargo beats the real way C code gets shared in embedded development (mostly through copy/paste from StackOverflow...).