6 ms·

I wish I had more spare time to help work on something like this. I've been reading all of these posts since the beginning, and they are all so excellent. I kne

I wish I had more spare time to help work on something like this. I've been reading all of these posts since the beginning, and they are all so excellent. I knew a lot of these things abstractly, but this explains it all in such solid detail, it's awesome. Thank you! Now a question for people in the know about micro-kernel architectures. One thing often claimed is that the message passing is too expensive, causing essentially double the work per system call. After reading this, I started wondering, and now want to read more about it: is it possible to use page faults and a handler to effectively build a message passing setup in the microkernel? Essentially using the pagefault as the mechanism to switch stacks from one kernel-service to another? Is this novel, stupid, or even faster than existing mechanisms? Obviously I need to go read how L4 and others do this. What's great about posts like this is that it makes me want to go learn about it...

adamnemecek 10y ago
> Obviously I need to go read how L4 and others do this. The L4 guy wrote a paper on it. https://www.cs.nyu.edu/~mwalfish/classes/15fa/ref/liedtke93improving.pdf https://www.cs.nyu.edu/~mwalfish/classes/15fa/ref/liedtke93i... I think I've been intrigued by the same idea, I believe that Rust enables you to build a reactive OS which would simplify a lot of programming. Given that interrupts are basically events, it's a match made in heaven. If someone wants to talk about this (by "this" I guess I mean the new OS architectures that are now possible due to Rust) more, shoot me an email (it's in my profile).
- mastax 10y ago
  You may be interested in these projects: https://github.com/ryanra/RustOS https://github.com/ryanra/RustOS RustOS is hobby OS project that's pretty bare bones, but has similar ideas to what you've mentioned. It's essentially a port of the Rust standard library to bare metal, relying on the safety of the language rather than hardware features. It's sort-of a unikernel in implementation if not intention. https://github.com/helena-project/tock https://github.com/helena-project/tock Tock is an interesting academic project to create an embedded OS. It still uses more traditional hardware protection for user processes, since these can be written in any language. The kernel, however, is composed of cooperatively-scheduled components called capsules, which rely on the language for isolation and don't have the overhead associated with real processes. See this doc for more details: https://www.tockos.org/documentation/design/ https://www.tockos.org/documentation/design/
  - adamnemecek 10y ago
    Both of those look really dope. How did you learn about them?
    - kibwen 10y ago
      reddit.com/r/rust is a useful resource for keeping abreast of Rust-related happenings. For other OS stuff in Rust, see Redox (a microkernel) and Robigalia (Rust on seL4).
      - adamnemecek 10y ago
        I think that the thing I'm most interested in is what new architectures does Rust allow for. Redox is cool but Unix has outlived it's usefulness at this point. Also. the Unix philosophy is the very anti-thesis of what I think is the Rust philosophy. I can imagine that a Rust port of QNX could be a big deal tho (QNX is/was only like 500KLOC of C, which could be like 100KLOC of Rust). But I think that we need mode advanced systems, e.g. don't need as much virtual memory protection because that's handled by the language. There's actually quite a few things in the current OSs that seem to be pure anachronisms.
        bluejekyll 10y ago
        > don't need as much virtual memory protection because that's handled by the language. This is true at compile time. But what does it do for you at runtime when bad actors will be trying to break any and all assumptions? The guarantees in the kernel need to be vigilant against attackers, so while the kernel itself might be simpler, the boundary with processes still needs a lot of constraints at runtime.
        krzat 10y ago
        I think the assumption here is that OS would only allow binaries built from safe rust code.
- qznc 10y ago
  There is also this nice L4 overview paper "From L3 to seL4 What Have We Learnt in 20 Years of L4 Microkernels?" http://sigops.org/sosp/sosp13/papers/p133-elphinstone.pdf http://sigops.org/sosp/sosp13/papers/p133-elphinstone.pdf
- milesrout 10y ago
  I'm curious why you seem to think that Rust makes new OS architectures possible. Rust is a language, and languages have nothing to do with OSes. Do you mean new kernel structures?
  - adamnemecek 10y ago
    Because you can write more complex systems with it. E.g. asynchrony is much easier, therefore you can use it more frequently. Like you can write pretty complex kernel code without worrying about anything. Similarly, some of the errors that are commonly handled by the OS are made impossible to express in Rust. Your OS can me made much leaner without loosing anything.
    - milesrout 10y ago
      That's just not true. You don't get any of Rust's threading guarantees at the kernel level because a gigantic portion of your code is `unsafe` anyway, and has to be. (See Redox). And 'some of the errors that are commonly handled by the OS' might be theoretically impossible to express in Rust, but they can still happen, so they still need to be handled.
      - adamnemecek 10y ago
        > You don't get any of Rust's threading guarantees at the kernel level because a gigantic portion of your code is `unsafe` anyway, and has to be. (See Redox). How do you not get "any"? You might lose some, because of unsafe code, but not all. > And 'some of the errors that are commonly handled by the OS' might be theoretically impossible to express in Rust, but they can still happen, so they still need to be handled. Sure but it's a fundamentally different situation. The types of errors might be pretty different.
the_duke 10y ago
The main performance problem isn't message passing itself but the required context switches. With a monolithic kernel, with disk io as an example and presented in a very primitive form: Userspace -> Kernel -> Userspace. Makes 2. With a microkernel: Userspace -> Kernel -> IO Service -> Disk Driver -> (IO Service) -> Kernel -> Userspace Makes 5-6 context switches (at least) for a single disk read, since everything runs in an isolated process, and a context switch is quite expensive (saving & restoring registers, etc). You can mitigate and optimize this though...
jacquesm 10y ago
> is it possible to use page faults and a handler to effectively build a message passing setup in the microkernel? Yes. (Been there, done that, 20 years ago). > Essentially using the pagefault as the mechanism to switch stacks from one kernel-service to another? You wouldn't use the pagefault for that but you'd switch the context (TSS or equivalent), this you need to do anyway when you switch from one process to another, or from a process to the kernel. But passing the message could be done via paging. > Is this novel, No. > stupid No. Definitely not. > or even faster than existing mechanisms? It's not faster than the current syscall arrangement, but it is faster than copying data from one process to another, which is the way a message passing kernel would normally pass messages from one process to another.
Animats 10y ago
Moving data from one process to another by messing with the MMU state to move a page from one process to another has been tried. Mach did that. Unless you're moving megabytes of data, on modern CPUs, messing with the MMU state is usually a lose. Caches and translation lookaside buffers have to be flushed, which is worse than copying. Copying is a cheap operation on most modern CPUs, because the data paths are wide and the data you're sending is probably in cache. Also, the data has to be page-aligned for this to work.

adamnemecek 10y ago

> Obviously I need to go read how L4 and others do this. The L4 guy wrote a paper on it. https://www.cs.nyu.edu/~mwalfish/classes/15fa/ref/liedtke93improving.pdf https://www.cs.nyu.edu/~mwalfish/classes/15fa/ref/liedtke93i... I think I've been intrigued by the same idea, I believe that Rust enables you to build a reactive OS which would simplify a lot of programming. Given that interrupts are basically events, it's a match made in heaven. If someone wants to talk about this (by "this" I guess I mean the new OS architectures that are now possible due to Rust) more, shoot me an email (it's in my profile).

mastax 10y ago
You may be interested in these projects: https://github.com/ryanra/RustOS https://github.com/ryanra/RustOS RustOS is hobby OS project that's pretty bare bones, but has similar ideas to what you've mentioned. It's essentially a port of the Rust standard library to bare metal, relying on the safety of the language rather than hardware features. It's sort-of a unikernel in implementation if not intention. https://github.com/helena-project/tock https://github.com/helena-project/tock Tock is an interesting academic project to create an embedded OS. It still uses more traditional hardware protection for user processes, since these can be written in any language. The kernel, however, is composed of cooperatively-scheduled components called capsules, which rely on the language for isolation and don't have the overhead associated with real processes. See this doc for more details: https://www.tockos.org/documentation/design/ https://www.tockos.org/documentation/design/
- adamnemecek 10y ago
  Both of those look really dope. How did you learn about them?
  - kibwen 10y ago
    reddit.com/r/rust is a useful resource for keeping abreast of Rust-related happenings. For other OS stuff in Rust, see Redox (a microkernel) and Robigalia (Rust on seL4).
    - adamnemecek 10y ago
      I think that the thing I'm most interested in is what new architectures does Rust allow for. Redox is cool but Unix has outlived it's usefulness at this point. Also. the Unix philosophy is the very anti-thesis of what I think is the Rust philosophy. I can imagine that a Rust port of QNX could be a big deal tho (QNX is/was only like 500KLOC of C, which could be like 100KLOC of Rust). But I think that we need mode advanced systems, e.g. don't need as much virtual memory protection because that's handled by the language. There's actually quite a few things in the current OSs that seem to be pure anachronisms.
      - bluejekyll 10y ago
        > don't need as much virtual memory protection because that's handled by the language. This is true at compile time. But what does it do for you at runtime when bad actors will be trying to break any and all assumptions? The guarantees in the kernel need to be vigilant against attackers, so while the kernel itself might be simpler, the boundary with processes still needs a lot of constraints at runtime.
        krzat 10y ago
        I think the assumption here is that OS would only allow binaries built from safe rust code.
qznc 10y ago
There is also this nice L4 overview paper "From L3 to seL4 What Have We Learnt in 20 Years of L4 Microkernels?" http://sigops.org/sosp/sosp13/papers/p133-elphinstone.pdf http://sigops.org/sosp/sosp13/papers/p133-elphinstone.pdf
milesrout 10y ago
I'm curious why you seem to think that Rust makes new OS architectures possible. Rust is a language, and languages have nothing to do with OSes. Do you mean new kernel structures?
- adamnemecek 10y ago
  Because you can write more complex systems with it. E.g. asynchrony is much easier, therefore you can use it more frequently. Like you can write pretty complex kernel code without worrying about anything. Similarly, some of the errors that are commonly handled by the OS are made impossible to express in Rust. Your OS can me made much leaner without loosing anything.
  - milesrout 10y ago
    That's just not true. You don't get any of Rust's threading guarantees at the kernel level because a gigantic portion of your code is `unsafe` anyway, and has to be. (See Redox). And 'some of the errors that are commonly handled by the OS' might be theoretically impossible to express in Rust, but they can still happen, so they still need to be handled.
    - adamnemecek 10y ago
      > You don't get any of Rust's threading guarantees at the kernel level because a gigantic portion of your code is `unsafe` anyway, and has to be. (See Redox). How do you not get "any"? You might lose some, because of unsafe code, but not all. > And 'some of the errors that are commonly handled by the OS' might be theoretically impossible to express in Rust, but they can still happen, so they still need to be handled. Sure but it's a fundamentally different situation. The types of errors might be pretty different.

the_duke 10y ago

The main performance problem isn't message passing itself but the required context switches. With a monolithic kernel, with disk io as an example and presented in a very primitive form: Userspace -> Kernel -> Userspace. Makes 2. With a microkernel: Userspace -> Kernel -> IO Service -> Disk Driver -> (IO Service) -> Kernel -> Userspace Makes 5-6 context switches (at least) for a single disk read, since everything runs in an isolated process, and a context switch is quite expensive (saving & restoring registers, etc). You can mitigate and optimize this though...

jacquesm 10y ago

> is it possible to use page faults and a handler to effectively build a message passing setup in the microkernel? Yes. (Been there, done that, 20 years ago). > Essentially using the pagefault as the mechanism to switch stacks from one kernel-service to another? You wouldn't use the pagefault for that but you'd switch the context (TSS or equivalent), this you need to do anyway when you switch from one process to another, or from a process to the kernel. But passing the message could be done via paging. > Is this novel, No. > stupid No. Definitely not. > or even faster than existing mechanisms? It's not faster than the current syscall arrangement, but it is faster than copying data from one process to another, which is the way a message passing kernel would normally pass messages from one process to another.

Animats 10y ago

Moving data from one process to another by messing with the MMU state to move a page from one process to another has been tried. Mach did that. Unless you're moving megabytes of data, on modern CPUs, messing with the MMU state is usually a lose. Caches and translation lookaside buffers have to be flushed, which is worse than copying. Copying is a cheap operation on most modern CPUs, because the data paths are wide and the data you're sending is probably in cache. Also, the data has to be page-aligned for this to work.