7 ms·

Making a RISC-V Operating System Using Rust

300 points by fogus 7y ago

tomlong 7y ago
I see the author and I share a naming convention for our miscellaneous scripts, and `do.sh` appears prominently in both our work. I subsequently branched out to e.g. `do-backups.sh`, where despite the `do-` being superfluous, I quite like the aesthetic. Slightly more on topic: I really enjoy blog series like these, with plenty of detail on esoteric topics I really have no idea about, or on the face of it much interest in. They're a fun way to increase the breadth of topics I have a superficial knowledge of.
azhenley 7y ago
I've been pushing the author to write this up for a year now (we both work at University of Tennessee). He teaches a lot of OS-oriented courses and makes them for fun so he is perfect to write this. Next I'm trying to convince him to make an OS in Zig for Raspberry Pi with a focus on graphics. If you have any suggestions for him, shoot him an email!
- jermaustin1 7y ago
  What would a "focus on graphics" mean? A GUI and windowing system, or just graphics processing?
  - azhenley 7y ago
    I want it to be a video game console :) I think our students would have a lot of fun with that.
    - mrec 7y ago
      If you do, definitely ping flohofwoe, who a) maintains a Zig port of his lightweight Sokol graphics API and b) has a penchant for retro consoles. https://github.com/floooh/sokol-zig https://github.com/floooh/sokol-zig
      - flohofwoe 7y ago
        The zig wrapper is only a minimal experiment and work in progress now, also I've hit a little bug :) https://github.com/ziglang/zig/issues/3211 https://github.com/ziglang/zig/issues/3211 BUT the sokol_app.h and sokol_gfx.h headers require an underlying platform 3D-API (e.g. GL, D3D11 or Metal) and window system to setup the swap chain. On a bare metal machine with only a 2D framebuffer they're not all that useful unfortunately.
        mrec 7y ago
        Ah yes, I didn't really think that one through. I know Redox OS (Rust) has OpenGL working, but I think they're just using the Mesa software renderer.
monocasa 7y ago
On a related note, MIT seems to have ported xv6 to riscv. https://github.com/mit-pdos/xv6-riscv https://github.com/mit-pdos/xv6-riscv
Lichtso 7y ago
Looks similar to my effort to interface RISC-V emulators and bare metal C: https://github.com/Lichtso/riscv-llvm-templates https://github.com/Lichtso/riscv-llvm-templates
GRBurst 7y ago
I really really like the first sentence > RISC-V ("risk five") and the Rust programming language both start with an R, so naturally they fit together
- agumonkey 7y ago
  I expect the os to be fully based around and r7rs loaded racket erlang and R.
- phkahler 7y ago
  "Because I wanted to" should be reason enough. But for those who demand a better justification, this is very specific ;-)
- DannyB2 7y ago
  When RISC-V inevitably renames itself CISC-V, then what language would naturally fit together? And don't say cobol.
  - antoinealb 7y ago
    C++?
  - kbenson 7y ago
    Why nor CRISC-V? Or CaRISC-V? Or (eww) RaCISC-V? It's not like current "CISC" architectures haven't taken a lot of the good aspects of RISC already too...
  - gpm 7y ago
    Cython with a rust core?
  - mindcrime 7y ago
    Ceylon?[1] Crystal?[2] Church?[3] Coq?[4] [1]: https://ceylon-lang.org/ https://ceylon-lang.org/ [2]: https://crystal-lang.org/ https://crystal-lang.org/ [3]: https://en.wikipedia.org/wiki/Church_(programming_language) https://en.wikipedia.org/wiki/Church_(programming_language) [4]: https://en.wikipedia.org/wiki/Coq https://en.wikipedia.org/wiki/Coq
GRBurst 7y ago
Since there are missing parts: Do you have a sheduling plan for the remaining posts?
- azhenley 7y ago
  One per week.
coldnose 7y ago
Rust is build around the expectation that allocations happen automatically and can never fail. I’m curious to see how they deal with this in a kernel...
- SAI_Peregrinus 7y ago
  The Rust standard library is. The Rust Core library (the far more minimal library that gets used when you mark a crate as #![no_std]) does not. That's how the other "OS in Rust" and "allocator in Rust" projects work.
  - wahern 7y ago
    Well, this OS in Rust project simply commits the same sins as the standard library by using a custom allocation routine that panics on allocation failure: https://os.phil-opp.com/heap-allocation/#allocations-in-rust https://os.phil-opp.com/heap-allocation/#allocations-in-rust Saying that Rust the language doesn't require a non-failing allocator misses the point--the ergonomics of the language make dealing with allocation failure difficult; sufficiently difficult that none of the projects I've seen actually bother attempting it. See https://cs.brown.edu/research/pubs/theses/ugrad/2015/light.alex.pdf https://cs.brown.edu/research/pubs/theses/ugrad/2015/light.a..., which explains idiomatic Rust instructs developers to return by value, relying on caller assignment to types like Box (which uses exchange_malloc under the hood), to handle heap allocation. Basically, the strategy for dynamic object management in Rust is predicated on hidden heap allocations. So of course it's not necessary. But good luck writing an entire operating system otherwise. Even Redox OS doesn't bother trying to fight the language in this regard: https://gitlab.redox-os.org/redox-os/slab_allocator/blob/master/src/lib.rs https://gitlab.redox-os.org/redox-os/slab_allocator/blob/mas...
    - jimktrains2 7y ago
      > the ergonomics of the language make dealing with allocation failure difficult; sufficiently difficult that none of the projects I've seen actually bother attempting it. Honest question: what languages make this ergonomic and can you share any projects that handle this gracefully?
loquor 7y ago
For someone with a standard CS undergraduate education, how much effort (in hrs/week) would such an endeavour require?
- DannyB2 7y ago
  Old wives tail from ancient times, before CompuServe: The first 20% of the effort gets you about 80% of the results. So everything seems exciting. But to that that last 20% of the results requires 80% of the total work. There is a ton of un-fun, un-glamorous work getting a gazillion device drivers written, for example.
  - nickpsecurity 7y ago
    You can always use a hypervisor, a driver OS that re-uses Windows/Linux drivers, and the new OS in a VM using virtual drivers. That's what some L4-based setups, including commercial OKL4, did. They also let you write native drivers directly on the microkernel or in OS VM's for situations where effort was justified. I'm surprised more haven't done this. Rump kernels are the closest trend.
    - panpanna 7y ago
      > I'm surprised more haven't done this. Because this is generally not secure. The driver OS will have access to hardware that can bypass the memory restrictions set upon it by the microkernel. There is sometimes special hardware to address this but they are too complex to manage from the kernel.
      - nickpsecurity 7y ago
        "Because this is generally not secure." Most OS's aren't designed to be that secure, though. That's why I wonder why it hasn't been tried more for usability. A security-focused setup certainly has more to be concerned about. Like I advocated with Xen, a good start would be making the host OpenBSD. They should be able to get hardware that would be compatible.
duckqlz 7y ago
Wow! What an awesome tutorial/book. I look forward to reading more chapters. This is already an extremely useful resource since it covers cross compilation from an x86 system. I love to play around with this kind of stuff in my free time but connecting to dev boards and constantly shifting paradigms was annoying enough to limit my enthusiasm. It took a lot of research to find out how to do what the author fits in a single chapter. Thank you for this awesome post!
mastrsushi 7y ago
Making X.....but using Rust!!!
turblety 7y ago
Amazing tutorial, and really exciting for RISC-V. I hope he creates a patreon or some other donation platform, as it must take some time to write this up. I'm sure some people would be willing to send a bit his way.
- azhenley 7y ago
  Done! Thanks for the suggestion. The Patreon link is at the top of the blog.
  - etskinner 7y ago
    One suggestion: Your patreon page only mentions the Aarch64 project, not the Rust on RISC-V project. That might alienate people coming from the Rust blog; they might think they landed in the wrong spot (I sure did)
ashort11 7y ago
I took Programing Languages from him, as well as TA'd for his Operating Systems class. He is an amazing lecturer, and I am glad he is releasing something like this to the public! Definitely something worthwhile to read. Looking forward to more content being released.

300 points by fogus 7y ago

I see the author and I share a naming convention for our miscellaneous scripts, and `do.sh` appears prominently in both our work. I subsequently branched out to e.g. `do-backups.sh`, where despite the `do-` being superfluous, I quite like the aesthetic. Slightly more on topic: I really enjoy blog series like these, with plenty of detail on esoteric topics I really have no idea about, or on the face of it much interest in. They're a fun way to increase the breadth of topics I have a superficial knowledge of.

azhenley 7y ago

I've been pushing the author to write this up for a year now (we both work at University of Tennessee). He teaches a lot of OS-oriented courses and makes them for fun so he is perfect to write this. Next I'm trying to convince him to make an OS in Zig for Raspberry Pi with a focus on graphics. If you have any suggestions for him, shoot him an email!

jermaustin1 7y ago
What would a "focus on graphics" mean? A GUI and windowing system, or just graphics processing?
- azhenley 7y ago
  I want it to be a video game console :) I think our students would have a lot of fun with that.
  - mrec 7y ago
    If you do, definitely ping flohofwoe, who a) maintains a Zig port of his lightweight Sokol graphics API and b) has a penchant for retro consoles. https://github.com/floooh/sokol-zig https://github.com/floooh/sokol-zig
    - flohofwoe 7y ago
      The zig wrapper is only a minimal experiment and work in progress now, also I've hit a little bug :) https://github.com/ziglang/zig/issues/3211 https://github.com/ziglang/zig/issues/3211 BUT the sokol_app.h and sokol_gfx.h headers require an underlying platform 3D-API (e.g. GL, D3D11 or Metal) and window system to setup the swap chain. On a bare metal machine with only a 2D framebuffer they're not all that useful unfortunately.
      - mrec 7y ago
        Ah yes, I didn't really think that one through. I know Redox OS (Rust) has OpenGL working, but I think they're just using the Mesa software renderer.

monocasa 7y ago

On a related note, MIT seems to have ported xv6 to riscv. https://github.com/mit-pdos/xv6-riscv https://github.com/mit-pdos/xv6-riscv

Lichtso 7y ago

Looks similar to my effort to interface RISC-V emulators and bare metal C: https://github.com/Lichtso/riscv-llvm-templates https://github.com/Lichtso/riscv-llvm-templates

GRBurst 7y ago

I really really like the first sentence > RISC-V ("risk five") and the Rust programming language both start with an R, so naturally they fit together

agumonkey 7y ago
I expect the os to be fully based around and r7rs loaded racket erlang and R.
phkahler 7y ago
"Because I wanted to" should be reason enough. But for those who demand a better justification, this is very specific ;-)
DannyB2 7y ago
When RISC-V inevitably renames itself CISC-V, then what language would naturally fit together? And don't say cobol.
- antoinealb 7y ago
  C++?
- kbenson 7y ago
  Why nor CRISC-V? Or CaRISC-V? Or (eww) RaCISC-V? It's not like current "CISC" architectures haven't taken a lot of the good aspects of RISC already too...
- gpm 7y ago
  Cython with a rust core?
- mindcrime 7y ago
  Ceylon?[1] Crystal?[2] Church?[3] Coq?[4] [1]: https://ceylon-lang.org/ https://ceylon-lang.org/ [2]: https://crystal-lang.org/ https://crystal-lang.org/ [3]: https://en.wikipedia.org/wiki/Church_(programming_language) https://en.wikipedia.org/wiki/Church_(programming_language) [4]: https://en.wikipedia.org/wiki/Coq https://en.wikipedia.org/wiki/Coq

GRBurst 7y ago

Since there are missing parts: Do you have a sheduling plan for the remaining posts?

azhenley 7y ago
One per week.

coldnose 7y ago

Rust is build around the expectation that allocations happen automatically and can never fail. I’m curious to see how they deal with this in a kernel...

SAI_Peregrinus 7y ago
The Rust standard library is. The Rust Core library (the far more minimal library that gets used when you mark a crate as #![no_std]) does not. That's how the other "OS in Rust" and "allocator in Rust" projects work.
- wahern 7y ago
  Well, this OS in Rust project simply commits the same sins as the standard library by using a custom allocation routine that panics on allocation failure: https://os.phil-opp.com/heap-allocation/#allocations-in-rust https://os.phil-opp.com/heap-allocation/#allocations-in-rust Saying that Rust the language doesn't require a non-failing allocator misses the point--the ergonomics of the language make dealing with allocation failure difficult; sufficiently difficult that none of the projects I've seen actually bother attempting it. See https://cs.brown.edu/research/pubs/theses/ugrad/2015/light.alex.pdf https://cs.brown.edu/research/pubs/theses/ugrad/2015/light.a..., which explains idiomatic Rust instructs developers to return by value, relying on caller assignment to types like Box (which uses exchange_malloc under the hood), to handle heap allocation. Basically, the strategy for dynamic object management in Rust is predicated on hidden heap allocations. So of course it's not necessary. But good luck writing an entire operating system otherwise. Even Redox OS doesn't bother trying to fight the language in this regard: https://gitlab.redox-os.org/redox-os/slab_allocator/blob/master/src/lib.rs https://gitlab.redox-os.org/redox-os/slab_allocator/blob/mas...
  - jimktrains2 7y ago
    > the ergonomics of the language make dealing with allocation failure difficult; sufficiently difficult that none of the projects I've seen actually bother attempting it. Honest question: what languages make this ergonomic and can you share any projects that handle this gracefully?

loquor 7y ago

For someone with a standard CS undergraduate education, how much effort (in hrs/week) would such an endeavour require?

DannyB2 7y ago
Old wives tail from ancient times, before CompuServe: The first 20% of the effort gets you about 80% of the results. So everything seems exciting. But to that that last 20% of the results requires 80% of the total work. There is a ton of un-fun, un-glamorous work getting a gazillion device drivers written, for example.
- nickpsecurity 7y ago
  You can always use a hypervisor, a driver OS that re-uses Windows/Linux drivers, and the new OS in a VM using virtual drivers. That's what some L4-based setups, including commercial OKL4, did. They also let you write native drivers directly on the microkernel or in OS VM's for situations where effort was justified. I'm surprised more haven't done this. Rump kernels are the closest trend.
  - panpanna 7y ago
    > I'm surprised more haven't done this. Because this is generally not secure. The driver OS will have access to hardware that can bypass the memory restrictions set upon it by the microkernel. There is sometimes special hardware to address this but they are too complex to manage from the kernel.
    - nickpsecurity 7y ago
      "Because this is generally not secure." Most OS's aren't designed to be that secure, though. That's why I wonder why it hasn't been tried more for usability. A security-focused setup certainly has more to be concerned about. Like I advocated with Xen, a good start would be making the host OpenBSD. They should be able to get hardware that would be compatible.

duckqlz 7y ago

Wow! What an awesome tutorial/book. I look forward to reading more chapters. This is already an extremely useful resource since it covers cross compilation from an x86 system. I love to play around with this kind of stuff in my free time but connecting to dev boards and constantly shifting paradigms was annoying enough to limit my enthusiasm. It took a lot of research to find out how to do what the author fits in a single chapter. Thank you for this awesome post!

mastrsushi 7y ago

Making X.....but using Rust!!!

turblety 7y ago

Amazing tutorial, and really exciting for RISC-V. I hope he creates a patreon or some other donation platform, as it must take some time to write this up. I'm sure some people would be willing to send a bit his way.

azhenley 7y ago
Done! Thanks for the suggestion. The Patreon link is at the top of the blog.
- etskinner 7y ago
  One suggestion: Your patreon page only mentions the Aarch64 project, not the Rust on RISC-V project. That might alienate people coming from the Rust blog; they might think they landed in the wrong spot (I sure did)

ashort11 7y ago

I took Programing Languages from him, as well as TA'd for his Operating Systems class. He is an amazing lecturer, and I am glad he is releasing something like this to the public! Definitely something worthwhile to read. Looking forward to more content being released.