10 ms·

Learn Rust by writing a small OS

534 points by elamje 5y ago

faitswulff 5y ago
Note that this guide is in the middle of a big rewrite to include a UEFI compatible bootloader. The author posts updates here: https://rust-osdev.com/ https://rust-osdev.com/
- ModernMech 5y ago
  Yes, while it may seem on the surface like the blog hasn't been updated in a while, the whole rust x86 project is under active development: https://github.com/rust-osdev/x86_64 https://github.com/rust-osdev/x86_64
- rectang 5y ago
  What are the completeness levels of the various editions (edition-1, edition-2, and edition-3)? In which editions, if any, is the OS anywhere close to feature-complete (i.e. close to meeting a base set of pedagogical goals as a "small OS" for teaching purposes). It's not clear to me from either the top-level docs or a skim of the update posts whether this effort is essentially a loose collection of (useful!) blog posts, or whether they close the loop to form a coherent and complete whole.
asicsp 5y ago
Previous discussions (2+ years old): https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=https%3A%2F%2Fos.phil-opp.com%2F&sort=byPopularity&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
pdevr 5y ago
This and the intermezzOS project linked below are both good, thanks for sharing those links. Is there any similar OS project (or even book!) that uses the latest operating system principles and non-hypothetical (i.e., unlike MIX or MINIX) languages?
rch 5y ago
I've been following the development of a Rust real-time OS for a while, and just got a RISC-V Longan Nano to run it on. It rare to cram so many interesting things to learn into a single side project.
- Steltek 5y ago
  How was RISC-V for setting up the toolchain? ESP32 was a bit confusing (I'm also a Rust newbie) and AVR/Arduino is stuck on an old version (2021-01-07) due to a bug.
jovial_cavalier 5y ago
A similar project: https://intermezzos.github.io/ https://intermezzos.github.io/
- johndoe0815 5y ago
  Stephen Marz' blog article series about writing an OS for RISC-V in Rust is also worth mentioning: https://osblog.stephenmarz.com https://osblog.stephenmarz.com
- steveklabnik 5y ago
  Someday I hope to get back to this. Temped to redo it for ARM this time though, given that I now have even more experience with it than x86 and it’s simpler in many ways. We’ll see.
burlesona 5y ago
This looks like an awesome resource, thanks for sharing! Question: at the point my impression is that Rust is the consensus best overall systems programming language (assuming you’re starting something new). Is that just me or so others share that perception? I’ve mostly done high level programming in my career but have wanted to eventually move into some barebones hardware stuff for fun. So I’m curious if Rust really is the best way to go these days (versus my old assumption that I should try to master C).
- ashton314 5y ago
  C is valuable because you shoot yourself in the foot over and over. Once you’ve dealt with use-after-frees, segfaults, memory corruption, then you will really appreciate what Rust does for you. Sure, you can learn Rust up front, but I on,y appreciated it after I had spent some time acquainting myself firsthand with the problems it’s trying to solve. Just my opinion though—Rust is a ton of fun.
  - nicoburns 5y ago
    My perspective was kinda the opposite: I'd already heard of use-after-frees, segfaults, memory corruption, etc. And I was very glad that I didn't have to deal with these as a programmer in high-level languages (JS, etc). As such, learning C was pretty daunting, knowing that even expert-level C programmers tended to hit into these issues, and there seeming to be 1001 rules that one has to learn to avoid them. Rust provided a way into low-level programming without having to deal with any of these things at all. All I had to do was learn a few concepts like allocation and ownership. Easy-peasy compared to the above! This was especially true as when I learnt Rust I had need of it in production at work. I would not have had the confidence to put C code into production as a novice C programmer with no oversight, but with Rust this wasn't a problem at all.
  - 10000truths 5y ago
    C is valuable because it puts an emphasis on ABI. In that sense, it’s a gentle introduction to assembly - functions map 1-to-1 with generated subroutines (modulo inlining), and no monomorphization/templating encourages code reuse at the instruction level. Those philosophies are important stepping stones to understanding concepts like dynamic linking, calling conventions, how registers behave, etc. Of course, you could emulate this behavior with Rust using extern and unsafe and whatnot, but the affordances of Rust steer you away from those details in favor of the higher level abstractions it offers. Which is what you want for 99% of software development, but when you’re writing platform specific code (e.g. OS startup, context switching, optimized SIMD) that a compiler can’t reliably generate, it helps to be able to quickly prototype something with C and then tweak certain instructions in the generated assembly subroutine until you get what you want.
eminence32 5y ago
Even if you don't want to write a small OS, or don't want to learn rust, this is still an excellent resource for people wanting to learn about low-level OS and hardware details.
- MR4D 5y ago
  I'm here to second this. While I have no interest (nor time!) in writing an OS, just skimming through this you can see the author put a significant amount of work and polish into this.
Pooge 5y ago
Slightly off-topic: has anyone here read Rust in Action[1]? How does it compare to other resources? I'm mostly a back-end engineer, but I've read the Rust Book[2] and I'd like to learn more about systems programming. [1]: https://www.manning.com/books/rust-in-action https://www.manning.com/books/rust-in-action [2]: https://doc.rust-lang.org/book/ https://doc.rust-lang.org/book/
- Serow225 5y ago
  I really like it. It’s half language intro, and half learn-via-projects. It worked well for my learning style :)
- ansible 5y ago
  I bought it when there was a kind of 2-for-1 deal with Rust in Motion a while back. I haven't spent too much time reading it though. From what I've browsed it seems competently written. It seems to cover the topics of typical interest in systems programming. The other resources I would point you towards are Rust by Example: https://doc.rust-lang.org/rust-by-example/ https://doc.rust-lang.org/rust-by-example/ And the Rust Cookbook: https://rust-lang-nursery.github.io/rust-cookbook/ https://rust-lang-nursery.github.io/rust-cookbook/ Though the cookbook is kind of out-of-date. I'm actually in the process of updating / expanding it, but that effort is far from ready for presentation.
- vincenv 5y ago
  Another book I found very helpful is Programming Rust [1], it expands on many concepts in the Rust book, the chapters on closures, iterators, collections, concurrency and async have very good explanations on how things work. [1]: https://www.oreilly.com/library/view/programming-rust-2nd/9781492052586/ https://www.oreilly.com/library/view/programming-rust-2nd/97...
  - Pooge 5y ago
    Is it adequate to learn systems programming concepts? I already know the concepts of Rust, so I'd rather learn the former. It's fine if the book teaches both, though.
vinc 5y ago
I've been playing with OS development for 2 years while writing MOROS[1] thanks to this project, it's such a great resource when you don't know where to start! With this and the help of the OSDev Wiki[2] you can really build your own toy OS. [1]: https://github.com/vinc/moros https://github.com/vinc/moros [2]: https://wiki.osdev.org https://wiki.osdev.org
Foe 5y ago
Anyone know of anything like this for (modern) C++?
- jonpalmisc 5y ago
  It's not a tutorial series, but you may get something out of browsing the source for https://github.com/SerenityOS/serenity https://github.com/SerenityOS/serenity in the meantime. I'm also curious if there are any recent/relevant C++ OS development guides.
FranchuFranchu 5y ago
I really like this website's design. It's an elegant way to group topics.
ModernMech 5y ago
I've followed this project from the beginning, but I would say in general this is only a good way to learn Rust if you already know how to write an OS. If you've never written an OS before, then following this will be an exercise in simultaneously learning Rust and operating systems, and that's a lot for anyone to expect of themself. This OS may be "small" but that should not be taken to mean "simple" (although the presentation is excellent and understandable). Good on you if you can pull it off though!
- toomanydoubts 5y ago
  I mostly agree but I don't think it's unreasonable for an experienced developer with a bunch of languages on their toolbelt to learn OS development while learning a new language.
nu11ptr 5y ago
I only skimmed, but the reason listed for using the nightly compiler (inline asm) just went away - inline asm was stabilized in 1.59 yesterday. However, there might be other reasons nightly is used, but I just saw this at a glance and thought I'd mention. Always better to use stable if you can.
- steveklabnik 5y ago
  Not all of inline asm is stable yet. The OS I work on at work still has to stay on nightly due to the bits that weren’t (and naked fns, which are apparently also on their way to stable in the nearish future.) That said those are the only nightly features we use, staying on stable when you can is good, and many projects will be able to go to stable due to this, which is awesome. Been waiting on this for years.
globular-toast 5y ago
This looks great! It took me so long to figure out how to start writing bare metal code for my raspberry pi so I could make a toy OS. This information is pretty hard to come by for some reason. I'm going to follow along so I can learn some Rust.
zomglings 5y ago
Wow, it is obvious that the author put a lot of love and care into writing this. This makes me excited to go through the book and I will have to try and make the time to do so. Kudos.
dang 5y ago
Related: Writing an OS in Rust: Async/Await - https://news.ycombinator.com/item?id=22727985 https://news.ycombinator.com/item?id=22727985 - March 2020 (103 comments) Writing an OS in Rust: Advanced Paging - https://news.ycombinator.com/item?id=19017108 https://news.ycombinator.com/item?id=19017108 - Jan 2019 (141 comments) Writing an OS in Rust: Introduction to Paging - https://news.ycombinator.com/item?id=18903235 https://news.ycombinator.com/item?id=18903235 - Jan 2019 (79 comments) Writing an OS in Rust: Hardware Interrupts - https://news.ycombinator.com/item?id=18274235 https://news.ycombinator.com/item?id=18274235 - Oct 2018 (63 comments) Writing an OS in Rust, Second Edition - https://news.ycombinator.com/item?id=16556481 https://news.ycombinator.com/item?id=16556481 - March 2018 (40 comments) Writing an OS in Rust: Handling Exceptions - https://news.ycombinator.com/item?id=13961020 https://news.ycombinator.com/item?id=13961020 - March 2017 (41 comments) Writing an OS in Rust: Returning from Exceptions - https://news.ycombinator.com/item?id=12548066 https://news.ycombinator.com/item?id=12548066 - Sept 2016 (54 comments) Writing an OS in Rust: Better Exception Messages - https://news.ycombinator.com/item?id=12218867 https://news.ycombinator.com/item?id=12218867 - Aug 2016 (31 comments) Writing an OS in Rust: Catching CPU Exceptions - https://news.ycombinator.com/item?id=11791694 https://news.ycombinator.com/item?id=11791694 - May 2016 (41 comments) Writing an OS in Rust: Remap the Kernel - https://news.ycombinator.com/item?id=10822479 https://news.ycombinator.com/item?id=10822479 - Jan 2016 (25 comments) Writing an OS in Rust - https://news.ycombinator.com/item?id=10807816 https://news.ycombinator.com/item?id=10807816 - Dec 2015 (34 comments) Writing an OS in Rust: Allocating Frames - https://news.ycombinator.com/item?id=10569463 https://news.ycombinator.com/item?id=10569463 - Nov 2015 (14 comments) Writing an OS in Rust - https://news.ycombinator.com/item?id=10448136 https://news.ycombinator.com/item?id=10448136 - Oct 2015 (34 comments)
thinkharderdev 5y ago
Wow, I really have no interest in OS development but that is one of the best resources I have ever seen for understanding Rust. Just read the section on async/await and it demystified everything I didn't understand.
lambda_dn 5y ago
I tried to learn Rust but found it very difficult and complicated to learn. I kept having to fight with the compiler to do even the simplest of memory/ lifetime management. I am a seasoned C developer and for me C was much easier. I think this will be the downfall of Rust as I hear it all the time from other developers who try it out.
- deleted 5y ago
  [deleted]
- rackjack 5y ago
  Before anybody takes this specific prediction too seriously, this guy posted numerous misleading claims about Actix's use of `unsafe` in another thread before deleting them and seems to have a strange vendetta against Rust in general.
musicale 5y ago
See also: https://cs140e.sergio.bz/about/ https://cs140e.sergio.bz/about/

534 points by elamje 5y ago

Note that this guide is in the middle of a big rewrite to include a UEFI compatible bootloader. The author posts updates here: https://rust-osdev.com/ https://rust-osdev.com/

ModernMech 5y ago
Yes, while it may seem on the surface like the blog hasn't been updated in a while, the whole rust x86 project is under active development: https://github.com/rust-osdev/x86_64 https://github.com/rust-osdev/x86_64
rectang 5y ago
What are the completeness levels of the various editions (edition-1, edition-2, and edition-3)? In which editions, if any, is the OS anywhere close to feature-complete (i.e. close to meeting a base set of pedagogical goals as a "small OS" for teaching purposes). It's not clear to me from either the top-level docs or a skim of the update posts whether this effort is essentially a loose collection of (useful!) blog posts, or whether they close the loop to form a coherent and complete whole.

asicsp 5y ago

Previous discussions (2+ years old): https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=https%3A%2F%2Fos.phil-opp.com%2F&sort=byPopularity&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

pdevr 5y ago

This and the intermezzOS project linked below are both good, thanks for sharing those links. Is there any similar OS project (or even book!) that uses the latest operating system principles and non-hypothetical (i.e., unlike MIX or MINIX) languages?

rch 5y ago

I've been following the development of a Rust real-time OS for a while, and just got a RISC-V Longan Nano to run it on. It rare to cram so many interesting things to learn into a single side project.

Steltek 5y ago
How was RISC-V for setting up the toolchain? ESP32 was a bit confusing (I'm also a Rust newbie) and AVR/Arduino is stuck on an old version (2021-01-07) due to a bug.

jovial_cavalier 5y ago

A similar project: https://intermezzos.github.io/ https://intermezzos.github.io/

johndoe0815 5y ago
Stephen Marz' blog article series about writing an OS for RISC-V in Rust is also worth mentioning: https://osblog.stephenmarz.com https://osblog.stephenmarz.com
steveklabnik 5y ago
Someday I hope to get back to this. Temped to redo it for ARM this time though, given that I now have even more experience with it than x86 and it’s simpler in many ways. We’ll see.

burlesona 5y ago

This looks like an awesome resource, thanks for sharing! Question: at the point my impression is that Rust is the consensus best overall systems programming language (assuming you’re starting something new). Is that just me or so others share that perception? I’ve mostly done high level programming in my career but have wanted to eventually move into some barebones hardware stuff for fun. So I’m curious if Rust really is the best way to go these days (versus my old assumption that I should try to master C).

ashton314 5y ago
C is valuable because you shoot yourself in the foot over and over. Once you’ve dealt with use-after-frees, segfaults, memory corruption, then you will really appreciate what Rust does for you. Sure, you can learn Rust up front, but I on,y appreciated it after I had spent some time acquainting myself firsthand with the problems it’s trying to solve. Just my opinion though—Rust is a ton of fun.
- nicoburns 5y ago
  My perspective was kinda the opposite: I'd already heard of use-after-frees, segfaults, memory corruption, etc. And I was very glad that I didn't have to deal with these as a programmer in high-level languages (JS, etc). As such, learning C was pretty daunting, knowing that even expert-level C programmers tended to hit into these issues, and there seeming to be 1001 rules that one has to learn to avoid them. Rust provided a way into low-level programming without having to deal with any of these things at all. All I had to do was learn a few concepts like allocation and ownership. Easy-peasy compared to the above! This was especially true as when I learnt Rust I had need of it in production at work. I would not have had the confidence to put C code into production as a novice C programmer with no oversight, but with Rust this wasn't a problem at all.
- 10000truths 5y ago
  C is valuable because it puts an emphasis on ABI. In that sense, it’s a gentle introduction to assembly - functions map 1-to-1 with generated subroutines (modulo inlining), and no monomorphization/templating encourages code reuse at the instruction level. Those philosophies are important stepping stones to understanding concepts like dynamic linking, calling conventions, how registers behave, etc. Of course, you could emulate this behavior with Rust using extern and unsafe and whatnot, but the affordances of Rust steer you away from those details in favor of the higher level abstractions it offers. Which is what you want for 99% of software development, but when you’re writing platform specific code (e.g. OS startup, context switching, optimized SIMD) that a compiler can’t reliably generate, it helps to be able to quickly prototype something with C and then tweak certain instructions in the generated assembly subroutine until you get what you want.

eminence32 5y ago

Even if you don't want to write a small OS, or don't want to learn rust, this is still an excellent resource for people wanting to learn about low-level OS and hardware details.

MR4D 5y ago
I'm here to second this. While I have no interest (nor time!) in writing an OS, just skimming through this you can see the author put a significant amount of work and polish into this.

Pooge 5y ago

Slightly off-topic: has anyone here read Rust in Action[1]? How does it compare to other resources? I'm mostly a back-end engineer, but I've read the Rust Book[2] and I'd like to learn more about systems programming. [1]: https://www.manning.com/books/rust-in-action https://www.manning.com/books/rust-in-action [2]: https://doc.rust-lang.org/book/ https://doc.rust-lang.org/book/

Serow225 5y ago
I really like it. It’s half language intro, and half learn-via-projects. It worked well for my learning style :)
ansible 5y ago
I bought it when there was a kind of 2-for-1 deal with Rust in Motion a while back. I haven't spent too much time reading it though. From what I've browsed it seems competently written. It seems to cover the topics of typical interest in systems programming. The other resources I would point you towards are Rust by Example: https://doc.rust-lang.org/rust-by-example/ https://doc.rust-lang.org/rust-by-example/ And the Rust Cookbook: https://rust-lang-nursery.github.io/rust-cookbook/ https://rust-lang-nursery.github.io/rust-cookbook/ Though the cookbook is kind of out-of-date. I'm actually in the process of updating / expanding it, but that effort is far from ready for presentation.
vincenv 5y ago
Another book I found very helpful is Programming Rust [1], it expands on many concepts in the Rust book, the chapters on closures, iterators, collections, concurrency and async have very good explanations on how things work. [1]: https://www.oreilly.com/library/view/programming-rust-2nd/9781492052586/ https://www.oreilly.com/library/view/programming-rust-2nd/97...
- Pooge 5y ago
  Is it adequate to learn systems programming concepts? I already know the concepts of Rust, so I'd rather learn the former. It's fine if the book teaches both, though.

vinc 5y ago

I've been playing with OS development for 2 years while writing MOROS[1] thanks to this project, it's such a great resource when you don't know where to start! With this and the help of the OSDev Wiki[2] you can really build your own toy OS. [1]: https://github.com/vinc/moros https://github.com/vinc/moros [2]: https://wiki.osdev.org https://wiki.osdev.org

Foe 5y ago

Anyone know of anything like this for (modern) C++?

jonpalmisc 5y ago
It's not a tutorial series, but you may get something out of browsing the source for https://github.com/SerenityOS/serenity https://github.com/SerenityOS/serenity in the meantime. I'm also curious if there are any recent/relevant C++ OS development guides.

FranchuFranchu 5y ago

I really like this website's design. It's an elegant way to group topics.

ModernMech 5y ago

I've followed this project from the beginning, but I would say in general this is only a good way to learn Rust if you already know how to write an OS. If you've never written an OS before, then following this will be an exercise in simultaneously learning Rust and operating systems, and that's a lot for anyone to expect of themself. This OS may be "small" but that should not be taken to mean "simple" (although the presentation is excellent and understandable). Good on you if you can pull it off though!

toomanydoubts 5y ago
I mostly agree but I don't think it's unreasonable for an experienced developer with a bunch of languages on their toolbelt to learn OS development while learning a new language.

nu11ptr 5y ago

I only skimmed, but the reason listed for using the nightly compiler (inline asm) just went away - inline asm was stabilized in 1.59 yesterday. However, there might be other reasons nightly is used, but I just saw this at a glance and thought I'd mention. Always better to use stable if you can.

steveklabnik 5y ago
Not all of inline asm is stable yet. The OS I work on at work still has to stay on nightly due to the bits that weren’t (and naked fns, which are apparently also on their way to stable in the nearish future.) That said those are the only nightly features we use, staying on stable when you can is good, and many projects will be able to go to stable due to this, which is awesome. Been waiting on this for years.

globular-toast 5y ago

This looks great! It took me so long to figure out how to start writing bare metal code for my raspberry pi so I could make a toy OS. This information is pretty hard to come by for some reason. I'm going to follow along so I can learn some Rust.

zomglings 5y ago

Wow, it is obvious that the author put a lot of love and care into writing this. This makes me excited to go through the book and I will have to try and make the time to do so. Kudos.

dang 5y ago

Related: Writing an OS in Rust: Async/Await - https://news.ycombinator.com/item?id=22727985 https://news.ycombinator.com/item?id=22727985 - March 2020 (103 comments) Writing an OS in Rust: Advanced Paging - https://news.ycombinator.com/item?id=19017108 https://news.ycombinator.com/item?id=19017108 - Jan 2019 (141 comments) Writing an OS in Rust: Introduction to Paging - https://news.ycombinator.com/item?id=18903235 https://news.ycombinator.com/item?id=18903235 - Jan 2019 (79 comments) Writing an OS in Rust: Hardware Interrupts - https://news.ycombinator.com/item?id=18274235 https://news.ycombinator.com/item?id=18274235 - Oct 2018 (63 comments) Writing an OS in Rust, Second Edition - https://news.ycombinator.com/item?id=16556481 https://news.ycombinator.com/item?id=16556481 - March 2018 (40 comments) Writing an OS in Rust: Handling Exceptions - https://news.ycombinator.com/item?id=13961020 https://news.ycombinator.com/item?id=13961020 - March 2017 (41 comments) Writing an OS in Rust: Returning from Exceptions - https://news.ycombinator.com/item?id=12548066 https://news.ycombinator.com/item?id=12548066 - Sept 2016 (54 comments) Writing an OS in Rust: Better Exception Messages - https://news.ycombinator.com/item?id=12218867 https://news.ycombinator.com/item?id=12218867 - Aug 2016 (31 comments) Writing an OS in Rust: Catching CPU Exceptions - https://news.ycombinator.com/item?id=11791694 https://news.ycombinator.com/item?id=11791694 - May 2016 (41 comments) Writing an OS in Rust: Remap the Kernel - https://news.ycombinator.com/item?id=10822479 https://news.ycombinator.com/item?id=10822479 - Jan 2016 (25 comments) Writing an OS in Rust - https://news.ycombinator.com/item?id=10807816 https://news.ycombinator.com/item?id=10807816 - Dec 2015 (34 comments) Writing an OS in Rust: Allocating Frames - https://news.ycombinator.com/item?id=10569463 https://news.ycombinator.com/item?id=10569463 - Nov 2015 (14 comments) Writing an OS in Rust - https://news.ycombinator.com/item?id=10448136 https://news.ycombinator.com/item?id=10448136 - Oct 2015 (34 comments)

thinkharderdev 5y ago

Wow, I really have no interest in OS development but that is one of the best resources I have ever seen for understanding Rust. Just read the section on async/await and it demystified everything I didn't understand.

lambda_dn 5y ago

I tried to learn Rust but found it very difficult and complicated to learn. I kept having to fight with the compiler to do even the simplest of memory/ lifetime management. I am a seasoned C developer and for me C was much easier. I think this will be the downfall of Rust as I hear it all the time from other developers who try it out.

deleted 5y ago
[deleted]
rackjack 5y ago
Before anybody takes this specific prediction too seriously, this guy posted numerous misleading claims about Actix's use of `unsafe` in another thread before deleting them and seems to have a strange vendetta against Rust in general.

musicale 5y ago

See also: https://cs140e.sergio.bz/about/ https://cs140e.sergio.bz/about/