4 ms·

Making a RISC-V Operating System Using Rust (2019)

113 points by azhenley 3y ago

opengears 3y ago
This blog post is from 2019 (with updates from 2020). I guess Rust has matured since then (and especially the RISCV support in Rust). Does anyone have more insight into this?
- proto_lambda 3y ago
  Since this is a from-scratch implementation, there's not much support from the language side to begin with. Except for some build system details that may have been made less annoying, I don't think much has changed.
  - junon 3y ago
    Ehhhh yes and no, you're mostly correct. 2021 is currently the shelf year for the latest edition of Rust. However rust versions everything, so even if you have a new compiler that removed support for something, you can still pull the old toolchains and build legacy codebases (like this one). My Rust OS not only pins the edition (which is the default anyway with `cargo init`) but also uses a toolchain file to pin the date of the nightly build I want contributors to build with. Rust tooling is really fantastic in this regard.
    - flandre 3y ago
      > However rust versions everything. I chuckled a little, after all, GCC isn't versioned and there's no companies out there still using GCC 4 or whatever kind of suffering they prefer.
      - junon 3y ago
        That wasn't my point at all, if you read the rest of the comment. I think it was clear I was referring to the ability pin and fetch specific versions of the toolchain when necessary, directly from Rust's tooling.
- jvdvegt 3y ago
  Some more clicking got me to updates from mid 2022: https://blog.stephenmarz.com/ https://blog.stephenmarz.com/
  - worik 3y ago
    Nothing for a year
twic 3y ago
Random RISC-V question: how do you make function calls in RV64? I have read that 'call' in assembly is a pseudo-instruction, which uses auipc and jalr to build the destination address. But surely that can only supply 32 bits of address? Is there a different sequence on 64 bit RISC-V? Or have I fundamentally misunderstood?
- dzaima 3y ago
  Direct function calls are usually done relative to the program counter (pc in auipc). So you get 32 bits of displacement, which is fine as long as your code is less than 4GB, and you get position-independent code for free. This is the same as on x86-64 and aarch64 too, which also have ~32 bits for call target immediates. (also, compilers have "code model" options for choosing the size of code to support)
  - MSFT_Edging 3y ago
    I assume there's also support for a kind of calculated long jump done at elf load time? I know that's done as part of the relocations on aarch64.
- mytailorisrich 3y ago
  My understanding is that AUIPC builds the destination address by adding a 32 bit offset to the current PC, i.e. it is relative, and that this works with both 32 and 64 bit addresses. The difference is that, on 32-bit systems this allows to immediately access any address while on 64-bit systems you are limited to this relative 32-bit window.
- Taniwha 3y ago
  Either: "jal lr, dest" or "jalr lr, offset(reg)" Both save the address of the next instruction in lr, the first is a relative branch the second is to the address pointed by a register plus an offset Use "jalr r0, 0(lr)" to return Anything else is just syntactic sugar in the assembler to set up the reg for jalr
deleted 3y ago
[deleted]
Symmetry 3y ago
What excites me most about RISC-V is the ability to potentially experiment at a systems level. I love the Mill project and they have a ton of interesting and clever ideas but they're trying to innovate along so many dimensions at once that I feel like it's almost a forgone conclusion that they'll fail. What I'd really like is for someone to design a chip with their ideas about backless memory and putting address translation at cache/memory border and do a RISC-V implementation of that. Then you could try getting a working OS with that up, and then maybe incrementally try out other ideas rather than just trying to change the world all at once. We've had this paradigm about how memory and permission and OS resource management works that we've have since the PDP-11 and I think we ought to be looking at how to move beyond that. http://millcomputing.com/wiki/Memory http://millcomputing.com/wiki/Memory
ThinkBeat 3y ago
This is great. It is the right way forward. Trying to smush Rust into old operating systems is in my opinion (and I might be the only one with the opinion) not an optimal use of resources. We need to build operating systems from the ground up in languages like Rust. We also need to let go of the UNIX way of doing things and move beyond it. It cannot possibly be the best there can ever be. Build them for modern hardware and even more so future hardware we can see coming down the line. Build them with security in mind from the ground up. Things like ransomware could be made close to impossible by an operating system. What will that be? I am not sure. I am hoping for a lot of people to try a lot of new things and allow things to evolve.
- Clamchop 3y ago
  A greenfield OS with real aspirations is a moonshot no matter what it's written in, so if we want to see systems-level investment in Rust that yields practical benefits...
- kjs3 3y ago
  We also need to let go of the UNIX way of doing things and move beyond it. So...dump all existing unix/linux software in the bin and start writing things like browsers and desktop managers and office suites from scratch. Have fun...see you in 20-30 years.
  - remexre 3y ago
    Sounds like the entire Midori project only ran for 15, and iiuc it got all of that on top of a memory-safe kernel that was reinventing the language it was written in as the project was going.
    - kjs3 3y ago
      Where can I buy that laptop running Midori?
      - remexre 3y ago
zoogeny 3y ago
Just a random thought: but wasn't C written specifically at first for the PDP series of mini computers? On the other side, the language evolved to write Unix. It feels we don't really have the same kind of match in modern programming languages. I wonder what a modern equivalent of C would look like that was tailor fitted to RISC-V ISA and an OS that was targeted to that device? It just is the case that one of the most successful programming languages of all time, C, was shaped by the existing hardware and the needs of the OS the language designers were building. But now we are going in the other direction by starting with an existing programming language and presumably shaping the OS to its strengths.
- phicoh 3y ago
  C (and I use the term loosely, not in de strict definition of standard C) is very well suited for low level programming of RISC processors. It not clear that a new language would be a big benefit. The benefit of C is to abstract from assembler and make Unix portable between architectures. C still works very well for that today. The big question is whether we still want to write new production code in C. For one, the C standard is horrible. Just about everything is undefined behavior. The second is that you don't want to spend a lot of time checking code for array bound errors, use after free, other problems with pointers, variables or data structures that are not (properly) initialized, etc. In addition, for large projects, all of the manual namespace management is also a bit of a bore. One big issue with C is that more or less, the CPU architecture has to treat memory as an array of bytes. There is just too much code that assumes that this is the case. Every data pointer can be converted to a void *, all function pointers are alike, etc. With Rust, there is a chance that radically new architectures are possible, if compatibility with C is no longer required.
  - flandre 3y ago
    > the C standard is horrible. Just about everything is undefined behavior How is this horrible? This is the core defining feature of C. It is equivalent of saying that a phallic shaped object with a sharp edge is terrible because you may cut/stab yourself and bleed to death, therefore dull equivalent is better (always). This is known as a knife. Knives with extremely sharp blades exist because grinding at the thing you're trying to cut for hours isn't something all of us enjoy. Have you tried not hurting yourself instead before calling it terrible? Sure, one idiot may die because of it, some unlucky guy may get killed with one, but really, I prefer using a real knife when cooking my food and I will never change my mind about that until I die. > In addition, for large projects, all of the manual namespace management is also a bit of a bore. When you're writing in C, fun is the least of your worries. I can recommend C++ for big projects, noone but yourself forces use of templates and classes in that department. I can't speak for people writing for controllers made in 1984 with proprietary compilers that probably aren't even maintained anymore, but that's not my problem. For projects like Linux, C++ would be more than sufficient. It's ironic that Rust has been accepted to Linux kernel because what he said about C++ over a decade ago couldn't be more true for Rust aswell, even though all you need to fix that is a bit of discipline, ironic given C programming is practically 99.97% of that. > One big issue with C is that more or less, the CPU architecture has to treat memory as an array of bytes. Not relevant, virtual memory is used in practically all modern hardware and "virtual" here means that you have arrays and have no clue what it's truly like already in hardware. Even kernels run in this mode, and all kernel code is written with this expectation. Before you make up such bullshit nonsense, you should tell us about this memory model that's superior to dumb arrays, because until then, noone has ever thought of a better one.
kaycebasques 3y ago
I struggled to get a repo up and running so that I could follow along with this series. I think I managed to simplify things a bit here: https://github.com/kaycebasques/rust-riscv64-qemu-starter https://github.com/kaycebasques/rust-riscv64-qemu-starter
Aerbil313 3y ago
Relevant, also an OS written in and made possible by Rust: https://www.theseus-os.com/ https://www.theseus-os.com/ Rust’s compile-time guarantees make an entirely different, single address space, single privilege level OS architecture possible. There is no kernel/userspace distinction. I hope it succeeds.
cdcarter 3y ago
This is an alright tutorial, and Rust is still pretty stable, but the QEMU implementation of RISC-V supervisor mode has changed a little bit and you'll need to fiddle with PMP and the MMU a little more. I also found that there's a fair amount of code Stephen never introduces or talks about on the videos, and just has dropped in the repo. This can make cobbling together your own version of this a little tougher, but maybe a fun experience.

113 points by azhenley 3y ago

This blog post is from 2019 (with updates from 2020). I guess Rust has matured since then (and especially the RISCV support in Rust). Does anyone have more insight into this?

proto_lambda 3y ago
Since this is a from-scratch implementation, there's not much support from the language side to begin with. Except for some build system details that may have been made less annoying, I don't think much has changed.
- junon 3y ago
  Ehhhh yes and no, you're mostly correct. 2021 is currently the shelf year for the latest edition of Rust. However rust versions everything, so even if you have a new compiler that removed support for something, you can still pull the old toolchains and build legacy codebases (like this one). My Rust OS not only pins the edition (which is the default anyway with `cargo init`) but also uses a toolchain file to pin the date of the nightly build I want contributors to build with. Rust tooling is really fantastic in this regard.
  - flandre 3y ago
    > However rust versions everything. I chuckled a little, after all, GCC isn't versioned and there's no companies out there still using GCC 4 or whatever kind of suffering they prefer.
    - junon 3y ago
      That wasn't my point at all, if you read the rest of the comment. I think it was clear I was referring to the ability pin and fetch specific versions of the toolchain when necessary, directly from Rust's tooling.
jvdvegt 3y ago
Some more clicking got me to updates from mid 2022: https://blog.stephenmarz.com/ https://blog.stephenmarz.com/
- worik 3y ago
  Nothing for a year

twic 3y ago

Random RISC-V question: how do you make function calls in RV64? I have read that 'call' in assembly is a pseudo-instruction, which uses auipc and jalr to build the destination address. But surely that can only supply 32 bits of address? Is there a different sequence on 64 bit RISC-V? Or have I fundamentally misunderstood?

dzaima 3y ago
Direct function calls are usually done relative to the program counter (pc in auipc). So you get 32 bits of displacement, which is fine as long as your code is less than 4GB, and you get position-independent code for free. This is the same as on x86-64 and aarch64 too, which also have ~32 bits for call target immediates. (also, compilers have "code model" options for choosing the size of code to support)
- MSFT_Edging 3y ago
  I assume there's also support for a kind of calculated long jump done at elf load time? I know that's done as part of the relocations on aarch64.
mytailorisrich 3y ago
My understanding is that AUIPC builds the destination address by adding a 32 bit offset to the current PC, i.e. it is relative, and that this works with both 32 and 64 bit addresses. The difference is that, on 32-bit systems this allows to immediately access any address while on 64-bit systems you are limited to this relative 32-bit window.
Taniwha 3y ago
Either: "jal lr, dest" or "jalr lr, offset(reg)" Both save the address of the next instruction in lr, the first is a relative branch the second is to the address pointed by a register plus an offset Use "jalr r0, 0(lr)" to return Anything else is just syntactic sugar in the assembler to set up the reg for jalr

deleted 3y ago

[deleted]

Symmetry 3y ago

What excites me most about RISC-V is the ability to potentially experiment at a systems level. I love the Mill project and they have a ton of interesting and clever ideas but they're trying to innovate along so many dimensions at once that I feel like it's almost a forgone conclusion that they'll fail. What I'd really like is for someone to design a chip with their ideas about backless memory and putting address translation at cache/memory border and do a RISC-V implementation of that. Then you could try getting a working OS with that up, and then maybe incrementally try out other ideas rather than just trying to change the world all at once. We've had this paradigm about how memory and permission and OS resource management works that we've have since the PDP-11 and I think we ought to be looking at how to move beyond that. http://millcomputing.com/wiki/Memory http://millcomputing.com/wiki/Memory

ThinkBeat 3y ago

This is great. It is the right way forward. Trying to smush Rust into old operating systems is in my opinion (and I might be the only one with the opinion) not an optimal use of resources. We need to build operating systems from the ground up in languages like Rust. We also need to let go of the UNIX way of doing things and move beyond it. It cannot possibly be the best there can ever be. Build them for modern hardware and even more so future hardware we can see coming down the line. Build them with security in mind from the ground up. Things like ransomware could be made close to impossible by an operating system. What will that be? I am not sure. I am hoping for a lot of people to try a lot of new things and allow things to evolve.

Clamchop 3y ago
A greenfield OS with real aspirations is a moonshot no matter what it's written in, so if we want to see systems-level investment in Rust that yields practical benefits...
kjs3 3y ago
We also need to let go of the UNIX way of doing things and move beyond it. So...dump all existing unix/linux software in the bin and start writing things like browsers and desktop managers and office suites from scratch. Have fun...see you in 20-30 years.
- remexre 3y ago
  Sounds like the entire Midori project only ran for 15, and iiuc it got all of that on top of a memory-safe kernel that was reinventing the language it was written in as the project was going.
  - kjs3 3y ago
    Where can I buy that laptop running Midori?
    - remexre 3y ago

zoogeny 3y ago

Just a random thought: but wasn't C written specifically at first for the PDP series of mini computers? On the other side, the language evolved to write Unix. It feels we don't really have the same kind of match in modern programming languages. I wonder what a modern equivalent of C would look like that was tailor fitted to RISC-V ISA and an OS that was targeted to that device? It just is the case that one of the most successful programming languages of all time, C, was shaped by the existing hardware and the needs of the OS the language designers were building. But now we are going in the other direction by starting with an existing programming language and presumably shaping the OS to its strengths.

phicoh 3y ago
C (and I use the term loosely, not in de strict definition of standard C) is very well suited for low level programming of RISC processors. It not clear that a new language would be a big benefit. The benefit of C is to abstract from assembler and make Unix portable between architectures. C still works very well for that today. The big question is whether we still want to write new production code in C. For one, the C standard is horrible. Just about everything is undefined behavior. The second is that you don't want to spend a lot of time checking code for array bound errors, use after free, other problems with pointers, variables or data structures that are not (properly) initialized, etc. In addition, for large projects, all of the manual namespace management is also a bit of a bore. One big issue with C is that more or less, the CPU architecture has to treat memory as an array of bytes. There is just too much code that assumes that this is the case. Every data pointer can be converted to a void *, all function pointers are alike, etc. With Rust, there is a chance that radically new architectures are possible, if compatibility with C is no longer required.
- flandre 3y ago
  > the C standard is horrible. Just about everything is undefined behavior How is this horrible? This is the core defining feature of C. It is equivalent of saying that a phallic shaped object with a sharp edge is terrible because you may cut/stab yourself and bleed to death, therefore dull equivalent is better (always). This is known as a knife. Knives with extremely sharp blades exist because grinding at the thing you're trying to cut for hours isn't something all of us enjoy. Have you tried not hurting yourself instead before calling it terrible? Sure, one idiot may die because of it, some unlucky guy may get killed with one, but really, I prefer using a real knife when cooking my food and I will never change my mind about that until I die. > In addition, for large projects, all of the manual namespace management is also a bit of a bore. When you're writing in C, fun is the least of your worries. I can recommend C++ for big projects, noone but yourself forces use of templates and classes in that department. I can't speak for people writing for controllers made in 1984 with proprietary compilers that probably aren't even maintained anymore, but that's not my problem. For projects like Linux, C++ would be more than sufficient. It's ironic that Rust has been accepted to Linux kernel because what he said about C++ over a decade ago couldn't be more true for Rust aswell, even though all you need to fix that is a bit of discipline, ironic given C programming is practically 99.97% of that. > One big issue with C is that more or less, the CPU architecture has to treat memory as an array of bytes. Not relevant, virtual memory is used in practically all modern hardware and "virtual" here means that you have arrays and have no clue what it's truly like already in hardware. Even kernels run in this mode, and all kernel code is written with this expectation. Before you make up such bullshit nonsense, you should tell us about this memory model that's superior to dumb arrays, because until then, noone has ever thought of a better one.

kaycebasques 3y ago

I struggled to get a repo up and running so that I could follow along with this series. I think I managed to simplify things a bit here: https://github.com/kaycebasques/rust-riscv64-qemu-starter https://github.com/kaycebasques/rust-riscv64-qemu-starter

Aerbil313 3y ago

Relevant, also an OS written in and made possible by Rust: https://www.theseus-os.com/ https://www.theseus-os.com/ Rust’s compile-time guarantees make an entirely different, single address space, single privilege level OS architecture possible. There is no kernel/userspace distinction. I hope it succeeds.

cdcarter 3y ago

This is an alright tutorial, and Rust is still pretty stable, but the QEMU implementation of RISC-V supervisor mode has changed a little bit and you'll need to fiddle with PMP and the MMU a little more. I also found that there's a fair amount of code Stephen never introduces or talks about on the videos, and just has dropped in the repo. This can make cobbling together your own version of this a little tougher, but maybe a fun experience.