6 ms·

One of the challenges to building an entirely new kernel is the vast amount of hardware support in the form of drivers and the features the existing kernel expo

One of the challenges to building an entirely new kernel is the vast amount of hardware support in the form of drivers and the features the existing kernel exports to userland in the form of syscalls. Would it be possible for a hypothetical new kernel, presumably written in Rust, to run an existing kernel such as Linux in a VM, just to tap into its drivers and emulate its syscalls? As more drivers are ported to the base kernel, the reliance on the donor kernel would shrink over time. Edit: This was an aside. Of course I realize that the conference was about adding Rust code to the Linux kernel, and not about rewriting any large, important, and functional codebase in language-of-the-day™. Hence my speculative language: "hypothetical" new kernel, "presumably" written in Rust.

cmckn 6y ago
This conference talk considered a re-write out-of-scope; they were only discussing how new code could be written in Rust. Interesting idea, though!
ColanR 6y ago
I was thinking about something similar recently - if the hardware drivers could somehow be abstracted from the rest of the linux kernel (as if...), then suddenly a ton of experimental kernels could have widespread hardware availability. It would probably shake up the OS ecosystem to no small degree, but I imagine the fresh ideas that could be experimented with by non-experts would be hugely beneficial. Isn't that the big argument against so many new projects? "It looks cool, but the hardware support isn't there." FreeBSD might be a better target though, since that's specifically designed to be compatible with everything.
- alex7o 6y ago
  This exists and is called rumpkerbels if I remember correctly.
  - puzzledobserver 6y ago
    rumpkerbels? Mind sharing a link? Google Search is curiously empty.
    - steveklabnik 6y ago
      They typo'd https://en.wikipedia.org/wiki/Rump_kernel https://en.wikipedia.org/wiki/Rump_kernel
- GoOnThenDoTell 6y ago
  Apple IOKit
  - pjmlp 6y ago
    Already in replacement process by Driver Kit and the long term roadmap to migrate all kernel extensions to userspace.
- genghizkhan 6y ago
  > FreeBSD might be a better target though, since that's specifically designed to be compatible with everything. NetBSD, perhaps, would be a better choice for "tries to run everywhere"?
  - ColanR 6y ago
    You're right, I confused the two.
- gibspaulding 6y ago
  Complete layperson here, so don't take my word, but isn't this sort of the idea behind microkernels such as GNU Hurd? https://www.gnu.org/software/hurd/microkernel.html https://www.gnu.org/software/hurd/microkernel.html
JoshTriplett 6y ago
The point of this LPC session was how to incrementally introduce Rust in the existing Linux kernel, with all its existing drivers and syscalls and similar. A rewrite would indeed be a daunting and problematic proposition. There are kernels written in Rust, such as Redox, but those are separate projects, and that's not what this conference session or article are talking about. Standing reminder: the Rust project is not a fan of rabid Rust over-evangelism (e.g. "Rewrite It In Rust", "Rust Evangelism Strike Force"), and sees it as damaging and unhelpful. We discourage that kind of thing wherever we see it. We're much more in favor of a measured, cautious approach.
- aspaceman 6y ago
  > the Rust project is not a fan of rabid Rust over-evangelism (e.g. "Rewrite It In Rust", "Rust Evangelism Strike Force"), and sees it as damaging and unhelpful. I was really caught off guard by this stuff recently. I was around the rust subreddit a lot back in 2013-2015 and never saw stuff like that. Stepped out while working on other stuff and now it seems common. Super strange.
  - progval 6y ago
    As an occasional rust programmer: you don't see this kind of behavior from the rust community. You see it when a random person opens an issue on a bugtracker you are subscribed to. Then it's usually prompty closed by a maintainer because the issue is very low effort, and the author doesn't offer to do any work themselves. Examples: https://gitlab.com/fdroid/fdroidclient/-/issues/1049 https://gitlab.com/fdroid/fdroidclient/-/issues/1049 and https://github.com/rapid7/metasploit-framework/issues/9092 https://github.com/rapid7/metasploit-framework/issues/9092
    - aspaceman 6y ago
      Makes sense. Sorry wasn’t trying to say it’s anyone’s fault at all. Just was weird as someone who’s been on and off that programming for awhile now.
ATsch 6y ago
I'm not sure why you'd need a VM for this. It should be much easier to implement bridges for major kernel apis like character devices, as I understand some BSDs already do?
rphln 6y ago
I might be wrong, but isn't there already a precedent for something similar in the form of the XNU kernel? From a cursory glance, it is basically a BSD and a Mach kernel running side by side.
dmitrygr 6y ago
in some ways, this is how Service Console works in vmware ESX: https://en.wikipedia.org/wiki/VMware_ESXi https://en.wikipedia.org/wiki/VMware_ESXi
wkz 6y ago
Xenomai uses this model, I believe. It is written in C, but I don’t see why you couldn’t do something similar with Rust.
AlphaSite 6y ago
ESX once ran this way, it had native drivers and Linux drivers, where it would run Linux as a vm and have it handle drivers for unsupported hardware. theres some more documentation on Wikipedia[1], it later switched to using vmklinux as a shim for those drivers. [1] https://en.wikipedia.org/wiki/VMware_ESXi#Architecture https://en.wikipedia.org/wiki/VMware_ESXi#Architecture
jasonhansel 6y ago
L4Linux was (is?) a similar project: https://l4linux.org/overview.shtml https://l4linux.org/overview.shtml The idea of that project was to turn Linux into a user mode application run by a microkernel; rewriting that microkernel in Rust would essentially give you what you're suggesting.
Hello71 6y ago
this is a very old idea. ndiswrapper did this for network drivers back in 2003, and freebsd implements shim layers for linux graphics drivers, linux applications, some solaris kernel modules (dtrace, firewall, others?). the main problem is that nowadays, modern simple hardware has mostly standardized on a fairly small set of APIs (e.g. SATA/SCSI/NVMe for the majority of disks, UVC for the majority of cameras), and modern complex hardware requires complex shims. see: the amount of work put into Linux graphics kernel APIs.
weinzierl 6y ago
> "One of the challenges to building an entirely new kernel is the vast amount of hardware support in the form of drivers [..]" This reminds me of an interview with Linus Torvalds from many many years ago, when he was still very young. The interviewer asked him if he was afraid of getting replaced by someone young and hungry. Torvalds shrugged it of with something along the lines of: Nah, no one likes to do driver development.

cmckn 6y ago

This conference talk considered a re-write out-of-scope; they were only discussing how new code could be written in Rust. Interesting idea, though!

ColanR 6y ago

I was thinking about something similar recently - if the hardware drivers could somehow be abstracted from the rest of the linux kernel (as if...), then suddenly a ton of experimental kernels could have widespread hardware availability. It would probably shake up the OS ecosystem to no small degree, but I imagine the fresh ideas that could be experimented with by non-experts would be hugely beneficial. Isn't that the big argument against so many new projects? "It looks cool, but the hardware support isn't there." FreeBSD might be a better target though, since that's specifically designed to be compatible with everything.

alex7o 6y ago
This exists and is called rumpkerbels if I remember correctly.
- puzzledobserver 6y ago
  rumpkerbels? Mind sharing a link? Google Search is curiously empty.
  - steveklabnik 6y ago
    They typo'd https://en.wikipedia.org/wiki/Rump_kernel https://en.wikipedia.org/wiki/Rump_kernel
GoOnThenDoTell 6y ago
Apple IOKit
- pjmlp 6y ago
  Already in replacement process by Driver Kit and the long term roadmap to migrate all kernel extensions to userspace.
genghizkhan 6y ago
> FreeBSD might be a better target though, since that's specifically designed to be compatible with everything. NetBSD, perhaps, would be a better choice for "tries to run everywhere"?
- ColanR 6y ago
  You're right, I confused the two.
gibspaulding 6y ago
Complete layperson here, so don't take my word, but isn't this sort of the idea behind microkernels such as GNU Hurd? https://www.gnu.org/software/hurd/microkernel.html https://www.gnu.org/software/hurd/microkernel.html

JoshTriplett 6y ago

The point of this LPC session was how to incrementally introduce Rust in the existing Linux kernel, with all its existing drivers and syscalls and similar. A rewrite would indeed be a daunting and problematic proposition. There are kernels written in Rust, such as Redox, but those are separate projects, and that's not what this conference session or article are talking about. Standing reminder: the Rust project is not a fan of rabid Rust over-evangelism (e.g. "Rewrite It In Rust", "Rust Evangelism Strike Force"), and sees it as damaging and unhelpful. We discourage that kind of thing wherever we see it. We're much more in favor of a measured, cautious approach.

aspaceman 6y ago
> the Rust project is not a fan of rabid Rust over-evangelism (e.g. "Rewrite It In Rust", "Rust Evangelism Strike Force"), and sees it as damaging and unhelpful. I was really caught off guard by this stuff recently. I was around the rust subreddit a lot back in 2013-2015 and never saw stuff like that. Stepped out while working on other stuff and now it seems common. Super strange.
- progval 6y ago
  As an occasional rust programmer: you don't see this kind of behavior from the rust community. You see it when a random person opens an issue on a bugtracker you are subscribed to. Then it's usually prompty closed by a maintainer because the issue is very low effort, and the author doesn't offer to do any work themselves. Examples: https://gitlab.com/fdroid/fdroidclient/-/issues/1049 https://gitlab.com/fdroid/fdroidclient/-/issues/1049 and https://github.com/rapid7/metasploit-framework/issues/9092 https://github.com/rapid7/metasploit-framework/issues/9092
  - aspaceman 6y ago
    Makes sense. Sorry wasn’t trying to say it’s anyone’s fault at all. Just was weird as someone who’s been on and off that programming for awhile now.

ATsch 6y ago

I'm not sure why you'd need a VM for this. It should be much easier to implement bridges for major kernel apis like character devices, as I understand some BSDs already do?

rphln 6y ago

I might be wrong, but isn't there already a precedent for something similar in the form of the XNU kernel? From a cursory glance, it is basically a BSD and a Mach kernel running side by side.

dmitrygr 6y ago

in some ways, this is how Service Console works in vmware ESX: https://en.wikipedia.org/wiki/VMware_ESXi https://en.wikipedia.org/wiki/VMware_ESXi

wkz 6y ago

Xenomai uses this model, I believe. It is written in C, but I don’t see why you couldn’t do something similar with Rust.

AlphaSite 6y ago

ESX once ran this way, it had native drivers and Linux drivers, where it would run Linux as a vm and have it handle drivers for unsupported hardware. theres some more documentation on Wikipedia[1], it later switched to using vmklinux as a shim for those drivers. [1] https://en.wikipedia.org/wiki/VMware_ESXi#Architecture https://en.wikipedia.org/wiki/VMware_ESXi#Architecture

jasonhansel 6y ago

L4Linux was (is?) a similar project: https://l4linux.org/overview.shtml https://l4linux.org/overview.shtml The idea of that project was to turn Linux into a user mode application run by a microkernel; rewriting that microkernel in Rust would essentially give you what you're suggesting.

Hello71 6y ago

this is a very old idea. ndiswrapper did this for network drivers back in 2003, and freebsd implements shim layers for linux graphics drivers, linux applications, some solaris kernel modules (dtrace, firewall, others?). the main problem is that nowadays, modern simple hardware has mostly standardized on a fairly small set of APIs (e.g. SATA/SCSI/NVMe for the majority of disks, UVC for the majority of cameras), and modern complex hardware requires complex shims. see: the amount of work put into Linux graphics kernel APIs.

weinzierl 6y ago

> "One of the challenges to building an entirely new kernel is the vast amount of hardware support in the form of drivers [..]" This reminds me of an interview with Linus Torvalds from many many years ago, when he was still very young. The interviewer asked him if he was afraid of getting replaced by someone young and hungry. Torvalds shrugged it of with something along the lines of: Nah, no one likes to do driver development.