11 ms·

MotorOS: a Rust-first operating system for x64 VMs

328 points by sbt567 3y ago

rrr_oh_man 3y ago
[flagged]
- 38 3y ago
  [flagged]
  - yjftsjthsd-h 3y ago
    https://en.wikipedia.org/wiki/OpenBSD https://en.wikipedia.org/wiki/OpenBSD
    - omeid2 3y ago
      OpenBSD is safer than Linux in spite of C, not because of it.
    - wmf 3y ago
      OpenBSD has existed for 30 years. In retrospect I wonder if the time would have been better spent developing a more secure language and porting BSD to that instead of meticulously scrubbing C code for each new class of bug.
  - omeid2 3y ago
    It is fascinating how despite the thousands and thousands of high profile memory related RCE vulnerabilities, how this community doesn't seem to like pointing the fact that C is an unsafe language.
    - vlovich123 3y ago
      Keep in mind the other important difference for this project which is a microkernel design with most operations running in isolated user space processes/libraries. That also helps limit the ability to exploit vulnerabilities in one place to take over other parts of the system.
      - snvzz 3y ago
        Not just the other important, but the most important. A safety-friendly language can help, but the system architecture is most important.
    - tourmalinetaco 3y ago
      Rust is only “safe” because by default you’re not allowed to do certain things. If you applied those same limitations to C it would be just as if not more safe. And the things you’re not allowed to do in “safe” Rust is required in various applications that C excels at, meaning that for Rust to do the same it has to be “unsafe”.
      - bigstrat2003 3y ago
        Nobody has ever denied that unsafe Rust code exists, so I don't know why people keep acting like it's some gotcha that invalidates the value of Rust. The point is that in Rust, it is clearly delimited where those unsafe operations are occurring, so you can focus on making sure those specific parts of your code are solid. In C, you have to be vigilant about your entire code base, not just a handful of blocks. Moreover, most Rust programs don't need to use unsafe at all, so those authors can rest easier.
      - BirAdam 3y ago
        I am usually the first to say that Rust evangelists are annoying, but… your argument isn’t valid. That any tool can be dangerous when used wrongly enough is not an argument in favor of or against rust. Additionally, that any given tool might be better for a certain job over any other isn’t an argument for or against other tools more generally.
      - empath-nirvana 3y ago
        > If you applied those same limitations to C it would be just as if not more safe. If frogs had wings, they wouldn't bump their butts when they hop. If you add all the security guarantees of Rust to C you would have something substantially similar to rust. As to your last point, it's very easy in rust to isolate those operations so you can just go over those with a fine tooth comb and not have to worry about unsafe stuff appearing everywhere in your code.
    - snvzz 3y ago
      >pointing the fact that C is an unsafe language. The implication there is that Rust is safe. Reality is, it is not. Rust provides some tools C does not, which can help in writing safe programs. That is about it. In exchange, it is a relatively young language with all it implies. For instance, there are not yet any successful OSs written in Rust. Kudos to the authors for actually writing code to change this, instead of evangelizing Rust on HN.
      - gpm 3y ago
        > For instance, there are not yet any successful OSs written in Rust. The rate at which new successful OS's come about is extremely low. Despite that, at least one OS in rust is being successfully deployed commercially today ('hubris', thanks to oxide computing), there are numerous reasonably advanced 'amateur' OS projects in rust (redox os being a reasonably polished example), and rust has begun finding it's way into the two most popular existing OSes (both the linux and windows kernels - I believe the windows kernel is farther along here). By any reasonable metric, rust has been wildly successful in the OS writing space. No doubt, in large part thanks to evangelizing by people who said "this is a reasonable thing to do".
- cedws 3y ago
  It's not wrong. Linux does not have a great security track record, it was built on completely different security principles than the ones we have today. You can find public criticism from Theo de Raadt and grsecurity if you want to find out more. It's a kernel that has served us well, especially considering its ubiquity, but it will never be a "secure" kernel.
  - vlovich123 3y ago
    I think it’s a shock when compared with the insecurity of Windows also forgetting that that was the reality on the ground in the 90s and that Windows also had a larger share. Windows has since hardened their security model and could be a more secure out of the box experience than Linux (eg shipping with TPM and FDE set up correctly). But yeah, there’s plenty of things you’d do differently if you wanted to properly secure Linux. It just shows that economically that’s not the most important thing for companies using Linux.
    - cedws 3y ago
      I think you are confusing Linux the Kernel with Linux-based operating systems. The original quote from the README was referencing the kernel and made no comparison to Windows or the NT kernel.
      - vlovich123 3y ago
        The Linux kernel and distributions is what I’m talking about so yes compared with NT kernel and windows as a distribution. The design choices you’d make to build a secure OS (kernel and user space) looks very different. The microkernel design is a more secure design but techniques to make it work fast took a very long time to develop (+ computer HW also got fast enough that the overhead is no longer as big of a deal + we have multithreading everywhere that microkernel can sometimes exploit more naturally).
- refulgentis 3y ago
  AFAIK if you're an OS-head, this is meant in the sense that it was designed to be a *NIX, not fundamentally advance security or make better decisions. (but I'm curious to hear I'm wrong)
- fulafel 3y ago
  Everything is relative of course but it's a consensus view in OS literature that the Unix family of operating systems doesn't have strong security. There's a huge TCB in which bugs lead to vulnerabilities with high probability, as we see all the time. This is why eg cloud providers don't rely on the OS to isolate customers from each other. Since the reneissance of virtualization, many security focused systems have built on that, like Qubes OS, seL4-based virtualized systems, etc.
  - josh2600 3y ago
    Everything in security is layers of defense against a threat model. To design a secure system one must first ask who is going to attack it and with what forces. Making truly secure systems is an art and is rarified ground.
  - bionsystem 3y ago
    Idk if it's "the Unix family". Clouds are running with SmartOS for example with containers running on bare metal and I've not heard of security issues with this model.
    - fulafel 3y ago
      SmartOS is a fork of OpenSolaris right? Solaris used to have its share of public vulnerability discourse when it had more users, and it quieted down as the user base shrank and people stopped deploying it as a general purpouse server OS. In a slow moving niche OS I wouldn't put much weight on low volume of public security problem discourse especially in face of apparent architectural problems.
      - bionsystem 3y ago
        Yes it's a fork of OpenSolaris. Companies are running clouds with it, with containers of different clients running next to each other on bare metal (no VM to re-isolate). If it was so easy to exploit it would have been done already wouldn't it ?
- deleted 3y ago
  [deleted]
- BirAdam 3y ago
  Just because something is better than an alternative doesn’t make it “good,” it just makes it less bad.
- snvzz 3y ago
  It is an accurate statement. Linux is a monolithic UNIX-like OS, with all the implications of an accidental design that dates to the late 60s/early 70s. For starters, it has huge (Linux has MLoCs) TCB. And it is very trusted (undeservedly so) code, as it runs with supervisor privileges. One bug in these MLoCs is all it takes. There are far better system architectures out there.
- toyg 3y ago
  Yeah, I read that and thought "woah, shots fired". But I'm old enough to know the definition of "secure system" varies so widely from person to person, it's just pointless to bicker about it. Some people won't be happy until programs have to ask whether they can flip a bit in memory by presenting the user's blood for analysis.
- crabbone 3y ago
  You probably need to interpret this as saying "the system is written in such a way that the application developer needs to take extra steps to secure their applications, compared to how it could've been written". For example, privilege elevation in Linux is a very complex and complicated mechanism. The famed "sudo" command has a very long history of bugs. To the point that I remember RHEL peddling their own version ("ksudo" if memory serves... I cannot find any mentions of this, it was some 15 years ago). It's very hard to get things right. Especially, to prove that you've done things right, if you are after security. The system interface is designed "for comfort" rather than "for security". Many things could've been a lot more secure, but very tedious on application programmer (imagine SeLinux, but in hyperactive mode... that's every sysadmin's nightmare). In practice, with a lot of community effort, tools like "sudo" eventually reached a point where they are mostly reliable for what they are expected to do. So, to face practical threats, today, the system may be OK, but this isn't the testimony to its design, rather it is a consequence of how much effort was spent plugging the holes.
- chlorion 3y ago
  There are heaps and heaps of security issues in Linux (the kernel). Simply mounting a file system can result in your system being compromised if the file system was maliciously crafted. This isn't something that is actively protected against. Another issue I can think of OTTOMH, is unprivileged user namespaces causing lots of security issues to surface because certain parts of the kernel's code was written with the assumption that only "root" would be able to invoke it, so if you already were "root" securing this didn't matter. There are many, many more issues similar to these, too many for me to write about, and probably many that are way beyond my comprehension. I don't think it's wrong to say that Linux has not been very secure.
westurner 3y ago
"Maestro: A Linux-compatible kernel in Rust" (2023) https://news.ycombinator.com/item?id=38852360#38857185 https://news.ycombinator.com/item?id=38852360#38857185 ; redox-os, cosmic-de , Motūrus OS; MotorOS
cedws 3y ago
It sounds like a cool project and I hope it continues development, but there is such a huge graveyard of such projects that have never gone anywhere that I struggle to get excited about them anymore. Replacing Linux is really hard, even for specific uses like cloud.
- seanw444 3y ago
  Replacing Linux is hard even for the BSDs, and those are very well established and considered basically next-in-line for similar tasks.
  - haarts 3y ago
    You think BSDs are next-in-line? Why is that? I have the distinct impression BSDs will remain niche. Something radical (like MotorOS?) seems more likely.
    - dylukes 3y ago
      What do you think is running on your favorite game consoles (that aren't Xbox)? I'll give you a hint, it isn't Linux. Nintendo is using a custom OS but with a huge chunk of user space borrowed from FreeBSD. Sony on the other hand just went and forked FreeBSD outright. You might also want to look into what OS are being used for server environments. A lot more BSD there than you might have initially guessed.
      - xxs 3y ago
        > your favorite game consoles unless it's a steamDeck, ofc. No love for consoles, though.
        dylukes 3y ago
        If I'm in the mood for being a little glib... where do you think that Linux distribution that SteamOS is based on got its user-space (or more seriously, its drivers?) Also: the SteamDeck is by any reasonable standard a console. It just happens to run a windowed environment out of the box. Don't be that guy. If you want to pump Valve, focus instead on their contributions to the Wine project.
faitswulff 3y ago
The author wrote this on reddit: > What does "Rust-first" mean here? It means not only that both the (micro) kernel and the drivers are implemented in Rust, but also that Rust is the first (and only, at the moment) language that userspace programs can be written in. > Although technically one can reverse-engineer the Rust-based ABI and the provided Rust toolchain to write apps for Motor OS in e.g. C, that is some work. But standard Rust programs (using standard Rust library, without FFI) will just compile and run - see e.g. https://github.com/moturus/motor-os/tree/main/src/bin/httpd https://github.com/moturus/motor-os/tree/main/src/bin/httpd. > This Rust-first approach is rather unique, as e.g. Redox uses relibc and C-based kernel ABI as the glue... https://old.reddit.com/r/rust/comments/190znk5/mot%C5%ABrus_os_motor_os_a_rustfirst_operating_system/ https://old.reddit.com/r/rust/comments/190znk5/mot%C5%ABrus_...
- glonq 3y ago
  > Rust is the first (and only, at the moment) language that userspace programs can be written in. Shouldn't be much effort to make it compatible with Crablang, no? /s
- mananaysiempre 3y ago
  Unstable, undocumented, compiler-(version-)dependent ABIs are the reason Haiku for x86-32 still has to use GCC 2.95. I’m never not going to welcome a hobby OS, especially in an interesting language (still have fond memories of House). By all means, go for it. Just... it’s important to be aware of the prior art here.
  - zokier 3y ago
    Haiku was stuck on old GCC only because they wanted to support old BeOS binaries? But binary backwards/forwards compatibility is not requirement for many use-cases.
    - pjmlp 3y ago
      Haiku without the BeOS applications wouldn't have that much to show. This is the way to go on platforms that favour shipping binaries instead of source code. Outside Linux/BSD, stable ABI across OS versions is a concern, hence why Apple, Google and Microsoft platforms all offer ways to keep this going.
paulirotta 3y ago
I was skeptical at first- the healthy approach to any new tech. But thinking again, the efficiency and security gains from stripping away layers of cruft after rather compelling
airtonix 3y ago
[dead]
bluejekyll 3y ago
One thing I keep hoping to see in all of these kernels in Rust is an async first kernel. Is there something that makes this particularly difficult or do folks not see the value in it? I know from following along with Phil Oppermann’s OS in Rust series that is definitely possible, but these last few OS’ in Rust seem to not be attempting this, https://os.phil-opp.com/async-await/ https://os.phil-opp.com/async-await/
- spullara 3y ago
  There are so many posts about how async Rust isn't the right abstraction it would surprise me if someone doubled-down on it right now.
- pjmlp 3y ago
  It would help if async Rust was actually a fully done feature, instead of half way there. As Niko Matsakis puts it, async/await is Rust in hard mode, you don't need that when having to also worry about everything writing an OS from scratch entails.
  - bluejekyll 3y ago
    Yes, there might be some significant gaps that make kernel development much more difficult. Off the top of my head, the current state of async traits is still being fleshed out, but it’s coming. The async_trait macros rely on boxing, so I could see that as a big downside in the kernel space. And I know that we don’t yet have an async streaming interface stabilized in the std lib. I’m not sure if these are blocking issues or not, thus my question.
    - surajrmal 3y ago
      async trait macros are no longer needed for the majority of use cases as of 1.75. additionally, I doubt a hobby OS will hit bottlenecks where boxing like this is a problem. Async streaming not being standardized as an interface would only matter if the kernel wanted to use libraries, and I would assume most kernels will be light on external dependencies so it shouldn't matter. If you are targeting a posix syscall interface, a lot of that doesn't really require async to pull off. You need to switch threads whenever an operation is blocked rather than run unrelated work on that thread. I suppose you can still model that with async, but it certainly doesn't feel as helpful.
Ericson2314 3y ago
> Nix OS all exist because of Linux's complexity That said, I would be thrilled to build this and other alt OSes and their userlands with Nix / Nixpkgs :).
- nicce 3y ago
  > because of Linux's complexity Linux is not complex but there are some design issues with C ABI and glibc stability and symlinks.
  - emmelaich 3y ago
    Indeed I would say Linux is as simple as possible given various goals ... however, 'simple as possible' can still be complex!
    - Ericson2314 3y ago
      Linux is definitely complex. Linux's ABI stability (good!) also means we cannot really blame C / glibc so much, it has internalized that problem. I agree the goal posts make much of this complexity inevitable, but the solution is to stop trying to intersect everyone's objectives into a much narrower and daunting goal. The big galaxy-brained goal should be: 1. Make more of Linux Rust. 2. Leverage Rust's superior expressive power to switch from a "configuration" mindset (Linux's current gazillion options on single giant code base) to a "composition mindset": we want to use types to combine various subsystems in various flavors. 3. Something like this and Linux proper can share lots of library code / subsystems. We can blur the line between "separate kernels" and "separate 'distrobutions' mixing and matching stand-alone Linux components". I think this is the only sustainable way to reign in the accidental complexity.
weinzierl 3y ago
Sounds interesting, but it also reminds me of what Linus once said when asked about fearing competition. From my memory his answer was something like: I really like writing device drivers. Few people like that and until someone young and hungry comes along who likes that I'm not afraid of competition.
- wavemode 3y ago
  You don't need very many device drivers to run in a VM. The real question is, how useful is a non-C-compatible, non-Linux-compatible, VM-only OS. Maybe a little bit, for microservices?
  - gpm 3y ago
    Is this really non-C-compatible? Seems like it just needs a libc layer written in rust. Most C programs don't make syscalls directly anyways (on some OSes they aren't even able to). The rest of the question is definitely fair though.
  - weinzierl 3y ago
    According to The American Heritage Dictionary an operating system is: "Software designed to control the hardware of a specific data-processing system in order to allow users and application programs to make use of it." The question is how much Operating System is a software that delegates all "control the hardware" parts to the layer below. I know there are wider definitions of OS, but my point is this is not going to replace Linux. Without question projects like MotorOS are sill useful. Besides practical applications it's a nice idea. Just recently I wrote in another thread that I would love to have a glimpse into an alternative universe, where Pascal had won over C and everything was Pascal-based. The idea to have everything Rust-based is even more exciting.
    - deleted 3y ago
      [deleted]
  - slau 3y ago
    It's relatively trivial to have a `FROM scratch` dockerised microservice in Rust. I'm not convinced having an entire OS brings that much to the table.
micahcc 3y ago
> a simple multi-processor round robin (SMP) > the kernel is very small and does not block, so does not need to be preemptible I don't believe you and I don't even need to look at the code to know this is false.
- wavemode 3y ago
  Care to elaborate?
  - deleted 3y ago
    [deleted]
  - jacquesm 3y ago
    From the docs at the top of scheduler.rs // The scheduler. // // As the kernel supports wait/wake/swap, and no blocking in the kernel // other than wait, any kind of more sophisticated scheduling policy // can be implemented in the userspace (in theory; maybe tweaks are needed // to make things perform well in practive). // // So the default scheduler here is rather simple, but should work // for a lot of use cases. // // Priorities: see enum Priority // Cpu affinity: a single CPU hint // Sched groups: none: everything within a priority is round robin, so // a process with many threads will negatively affect a process with few threads. // // This should be OK for most situations: if there is a need for isolation, // just use another VM. That's elaboration enough I think?
andyferris 3y ago
Out of curiousity, why would a small kernel take a whole 200ms to start on a modern computer? Wouldn't it need to initialize some metadata for the memory pages, mount the filesystem, and try to launch an init process? I suppose there might be an ethernet driver and possibly something to pipe logs to ("stdout" for the VM) to initialize. Shouldn't that all take a few microseconds? Or is all the slowness in the host preparing the resources? (as in QEMU and KVM?)
- deleted 3y ago
  [deleted]
- Veserv 3y ago
  In general, lots of hardware requires long waits to initialize. As in, program a value then you must wait 50 ms before querying the registers or the hardware will give you non-deterministic garbage with no error indication. So, it depends on the hardware, though you are unlikely to need more than 1 second on anything other than truly degenerate hardware assuming you are initializing devices in parallel.
  - azinman2 3y ago
    But it’s only for VMs. So what hardware are you initializing?
    - Modified3019 3y ago
      I pass through a GPU and USB hub to a VM running on a machine in the garage. An optical video cable and network compatible USB extender brings the interface to a different room making it my primary “desktop” computer (and an outdated laptop as a backup device). Doesn’t get more silent and cool than this. Another VM on the garage machine gets a bunch of hard drives passed through to it. That said, hardware passthrough/VFIO is likely out of the current realistic scope for this project. VM boot times can be optimized if you never look for hardware to initialize in the first place. Though they are still likely initializing a network interface of some sort. “MicroVM” seems to be a term used when as much as possible is stripped from a VM, such as with https://github.com/firecracker-microvm/firecracker https://github.com/firecracker-microvm/firecracker
ianbicking 3y ago
It could be cool if there was a WASM container built in Rust that could run in this OS. I don't really have a sense of how complicated a WASM container is, though it seems non-trivial to even decide what such a thing is, so it would be nice not to treat the WASM container itself as the OS (i.e., more room to safely experiment). WASM also seems like it dodges the ABI issue by being more explicitly about composition instead of shared binary data structures.
- gpm 3y ago
  When you say wasm container, you mean something like wasmtime that provides a non-browser wasm runtime? https://github.com/bytecodealliance/wasmtime https://github.com/bytecodealliance/wasmtime
  - ianbicking 3y ago
    Yeah, just that sort of thing. I mean, heck, can you put #!wasmtime at the top of a wasm file? Would such a file literally run right now (assuming #! is supported, and wasmtime doesn't hit any MotorOS Rust limitations)?
    - ianbicking 3y ago
      Serendipitously I see this article about WASM and how it has all these nice properties as a runtime abstraction: https://wingolog.org/archives/2024/01/08/missing-the-point-of-webassembly https://wingolog.org/archives/2024/01/08/missing-the-point-o...
pipeline_peak 3y ago
I don’t think swapping out the first order language does anything other than boxes out potential support.
jacquesm 3y ago
It isn't hard to start writing a new operating system. It is very hard to support that operating system for the next 5 decades.
- xyst 3y ago
  Linus Torvalds resolve is unbreakable
  - jacquesm 3y ago
    Yes, it's most impressive. I couldn't do it. That kind of long term focus is an incredible asset.
keithalewis 3y ago
[flagged]
- deleted 3y ago
  [deleted]
aaa_aaa 3y ago
I think the most close to production ready "mostly" Rust OS is Fuchsia.
- Octoth0rpe 3y ago
  I've seen this stated in a couple places, but IIRC the kernel is C++ and the UI is Dart. I assume lots of things in between could be in rust, but do you know which? daemons, services, drivers, window server?
jmull 3y ago
> ...Docker, Nix OS, "serverless", etc. all exist because of Linux's complexity Yeah, this seems like it's more directly competing with those than Linux. I'd want to see those addressed in the "Why?" -- that is, why MotorOs instead of Docker, etc.?
- btreecat 3y ago
  My first thought was "can this run my containers?"
- crabbone 3y ago
  > why MotorOs instead of Docker, etc.? It basically says it already: > Motūrus OS is a microkernel-based operating system By and large, microkernel and containers solve the same problem. Except, I wouldn't call containers a "solution", more like a workaround. Not in the sense that containers by themselves are a workaround, but the way they are used is a workaround for the same problem. The way containers are used today, especially in the context of Kubernetes, is to finely slice the available physical resources. The orchestration allows to manage which part of the application gets what slice of the resource, thus attempting to cut down on waste that's typically generated in the world where resources are managed through VMs. Where the typical problem is that a single VM will require too many resources because of the bloated OS it needs to run and because it's hard to create VMs with very limited resources, since OSes usually come as a package deal. So, containers "solve" the problem by giving up VM optimization -- instead it's usually beneficial to create very beefy VMs, on top of which then a new virtualization layer is created with containers. This minimized the VM waste, but doesn't get rid of it entirely, and, of course, creates a lot of complications with all the indirection resulting from two-tiered virtualization. Microkernel is the opposite of this "solution" -- it's the attempt to make OSes more modular, and as such less demanding of resources. Ideally, in the world if microkernels you don't need containers (at least not for the thing they are usually used today): your VMs can slice the resources in the way at least as efficient as containers do (or, hopefully, even better). So... to predict the next possible question: why containers are so popular and micorkernels aren't: it's because the later is harder on the applications (even when applications don't actually need something, they often use it because it's available in a full-blown OS, applications aren't usually written with resource scarcity in mind). Secondly, containers, essentially, exist on top of somewhat uniform, somewhat stable interface of Linux kernel. Micorkernel VMs would expose users to the zoo of ideas hardware vendors put into their products, making portability difficult. Finally, for all its flaws, Kubernetes is a big system with many (even if not so great) solutions for many problems. So, programmers who fear technology feel like it gives them a safety net and will allow them to program from a more comfortable position of modifying YAML files, copying the most upvoted answers from StackOverflow. There won't be such an easy cake in microkernel world.
duped 3y ago
> Docker, Nix OS, "serverless", etc. all exist because of Linux's complexity Docker and NixOS exist because of userspace problems with package management and serverless exists because businesses want to pay for compute on demand.
dravigon 3y ago
isn't this similar to other microkernel and something like wasi/wasm kinda addressing this the correct way ?
lasiotus 3y ago
I'm the project author/dev. Thanks a lot for posting this, and for comments/discussions! I see two main concerns raised here: (a) long-term viability and support (b) compilers, binary compatibility, etc. While the first concern is definitely valid, and without a community this project will not succeed, I do believe that potential benefits of Motor OS (or a similarly focused/structured project) will eventually result in a widely used new operating system. There are major problems with Linux inside VMs (and sometimes outside), and the Linux devs are not focused on this enough to clean things up anytime soon. I work on Linux Kernel at my day job, I know. Re: compiler instability, binary compatibility, etc.: I'm sorry, I don't understand what is the issue here. The latest Linux kernel can be compiled with different GCC or LLVM toolchains on x86_64, and the result will happily run old binaries compiled years ago with who knows what. repr(C) structs in rust are stable... So why so many concerns here? Again, thank you all for your comments and questions - I'm happy to answer more (at least until my day job kicks in).
xyst 3y ago
Kind of cool as a hobby OS. But would be nice to backup your claims regarding performance with actual numbers.
- lasiotus 3y ago
  The lainding page at https://github.com/moturus/motor-os https://github.com/moturus/motor-os explicitly says that both networking and file I/O are slow and have to be improved. The only claim is about fast bootup, and the number is there, and can easily be verified. Where do you see any unsupported claims re: performance?

328 points by sbt567 3y ago

rrr_oh_man 3y ago

[flagged]

38 3y ago
[flagged]
- yjftsjthsd-h 3y ago
  https://en.wikipedia.org/wiki/OpenBSD https://en.wikipedia.org/wiki/OpenBSD
  - omeid2 3y ago
    OpenBSD is safer than Linux in spite of C, not because of it.
  - wmf 3y ago
    OpenBSD has existed for 30 years. In retrospect I wonder if the time would have been better spent developing a more secure language and porting BSD to that instead of meticulously scrubbing C code for each new class of bug.
- omeid2 3y ago
  It is fascinating how despite the thousands and thousands of high profile memory related RCE vulnerabilities, how this community doesn't seem to like pointing the fact that C is an unsafe language.
  - vlovich123 3y ago
    Keep in mind the other important difference for this project which is a microkernel design with most operations running in isolated user space processes/libraries. That also helps limit the ability to exploit vulnerabilities in one place to take over other parts of the system.
    - snvzz 3y ago
      Not just the other important, but the most important. A safety-friendly language can help, but the system architecture is most important.
  - tourmalinetaco 3y ago
    Rust is only “safe” because by default you’re not allowed to do certain things. If you applied those same limitations to C it would be just as if not more safe. And the things you’re not allowed to do in “safe” Rust is required in various applications that C excels at, meaning that for Rust to do the same it has to be “unsafe”.
    - bigstrat2003 3y ago
      Nobody has ever denied that unsafe Rust code exists, so I don't know why people keep acting like it's some gotcha that invalidates the value of Rust. The point is that in Rust, it is clearly delimited where those unsafe operations are occurring, so you can focus on making sure those specific parts of your code are solid. In C, you have to be vigilant about your entire code base, not just a handful of blocks. Moreover, most Rust programs don't need to use unsafe at all, so those authors can rest easier.
    - BirAdam 3y ago
      I am usually the first to say that Rust evangelists are annoying, but… your argument isn’t valid. That any tool can be dangerous when used wrongly enough is not an argument in favor of or against rust. Additionally, that any given tool might be better for a certain job over any other isn’t an argument for or against other tools more generally.
    - empath-nirvana 3y ago
      > If you applied those same limitations to C it would be just as if not more safe. If frogs had wings, they wouldn't bump their butts when they hop. If you add all the security guarantees of Rust to C you would have something substantially similar to rust. As to your last point, it's very easy in rust to isolate those operations so you can just go over those with a fine tooth comb and not have to worry about unsafe stuff appearing everywhere in your code.
  - snvzz 3y ago
    >pointing the fact that C is an unsafe language. The implication there is that Rust is safe. Reality is, it is not. Rust provides some tools C does not, which can help in writing safe programs. That is about it. In exchange, it is a relatively young language with all it implies. For instance, there are not yet any successful OSs written in Rust. Kudos to the authors for actually writing code to change this, instead of evangelizing Rust on HN.
    - gpm 3y ago
      > For instance, there are not yet any successful OSs written in Rust. The rate at which new successful OS's come about is extremely low. Despite that, at least one OS in rust is being successfully deployed commercially today ('hubris', thanks to oxide computing), there are numerous reasonably advanced 'amateur' OS projects in rust (redox os being a reasonably polished example), and rust has begun finding it's way into the two most popular existing OSes (both the linux and windows kernels - I believe the windows kernel is farther along here). By any reasonable metric, rust has been wildly successful in the OS writing space. No doubt, in large part thanks to evangelizing by people who said "this is a reasonable thing to do".
cedws 3y ago
It's not wrong. Linux does not have a great security track record, it was built on completely different security principles than the ones we have today. You can find public criticism from Theo de Raadt and grsecurity if you want to find out more. It's a kernel that has served us well, especially considering its ubiquity, but it will never be a "secure" kernel.
- vlovich123 3y ago
  I think it’s a shock when compared with the insecurity of Windows also forgetting that that was the reality on the ground in the 90s and that Windows also had a larger share. Windows has since hardened their security model and could be a more secure out of the box experience than Linux (eg shipping with TPM and FDE set up correctly). But yeah, there’s plenty of things you’d do differently if you wanted to properly secure Linux. It just shows that economically that’s not the most important thing for companies using Linux.
  - cedws 3y ago
    I think you are confusing Linux the Kernel with Linux-based operating systems. The original quote from the README was referencing the kernel and made no comparison to Windows or the NT kernel.
    - vlovich123 3y ago
      The Linux kernel and distributions is what I’m talking about so yes compared with NT kernel and windows as a distribution. The design choices you’d make to build a secure OS (kernel and user space) looks very different. The microkernel design is a more secure design but techniques to make it work fast took a very long time to develop (+ computer HW also got fast enough that the overhead is no longer as big of a deal + we have multithreading everywhere that microkernel can sometimes exploit more naturally).
refulgentis 3y ago
AFAIK if you're an OS-head, this is meant in the sense that it was designed to be a *NIX, not fundamentally advance security or make better decisions. (but I'm curious to hear I'm wrong)
fulafel 3y ago
Everything is relative of course but it's a consensus view in OS literature that the Unix family of operating systems doesn't have strong security. There's a huge TCB in which bugs lead to vulnerabilities with high probability, as we see all the time. This is why eg cloud providers don't rely on the OS to isolate customers from each other. Since the reneissance of virtualization, many security focused systems have built on that, like Qubes OS, seL4-based virtualized systems, etc.
- josh2600 3y ago
  Everything in security is layers of defense against a threat model. To design a secure system one must first ask who is going to attack it and with what forces. Making truly secure systems is an art and is rarified ground.
- bionsystem 3y ago
  Idk if it's "the Unix family". Clouds are running with SmartOS for example with containers running on bare metal and I've not heard of security issues with this model.
  - fulafel 3y ago
    SmartOS is a fork of OpenSolaris right? Solaris used to have its share of public vulnerability discourse when it had more users, and it quieted down as the user base shrank and people stopped deploying it as a general purpouse server OS. In a slow moving niche OS I wouldn't put much weight on low volume of public security problem discourse especially in face of apparent architectural problems.
    - bionsystem 3y ago
      Yes it's a fork of OpenSolaris. Companies are running clouds with it, with containers of different clients running next to each other on bare metal (no VM to re-isolate). If it was so easy to exploit it would have been done already wouldn't it ?
deleted 3y ago
[deleted]
BirAdam 3y ago
Just because something is better than an alternative doesn’t make it “good,” it just makes it less bad.
snvzz 3y ago
It is an accurate statement. Linux is a monolithic UNIX-like OS, with all the implications of an accidental design that dates to the late 60s/early 70s. For starters, it has huge (Linux has MLoCs) TCB. And it is very trusted (undeservedly so) code, as it runs with supervisor privileges. One bug in these MLoCs is all it takes. There are far better system architectures out there.
toyg 3y ago
Yeah, I read that and thought "woah, shots fired". But I'm old enough to know the definition of "secure system" varies so widely from person to person, it's just pointless to bicker about it. Some people won't be happy until programs have to ask whether they can flip a bit in memory by presenting the user's blood for analysis.
crabbone 3y ago
You probably need to interpret this as saying "the system is written in such a way that the application developer needs to take extra steps to secure their applications, compared to how it could've been written". For example, privilege elevation in Linux is a very complex and complicated mechanism. The famed "sudo" command has a very long history of bugs. To the point that I remember RHEL peddling their own version ("ksudo" if memory serves... I cannot find any mentions of this, it was some 15 years ago). It's very hard to get things right. Especially, to prove that you've done things right, if you are after security. The system interface is designed "for comfort" rather than "for security". Many things could've been a lot more secure, but very tedious on application programmer (imagine SeLinux, but in hyperactive mode... that's every sysadmin's nightmare). In practice, with a lot of community effort, tools like "sudo" eventually reached a point where they are mostly reliable for what they are expected to do. So, to face practical threats, today, the system may be OK, but this isn't the testimony to its design, rather it is a consequence of how much effort was spent plugging the holes.
chlorion 3y ago
There are heaps and heaps of security issues in Linux (the kernel). Simply mounting a file system can result in your system being compromised if the file system was maliciously crafted. This isn't something that is actively protected against. Another issue I can think of OTTOMH, is unprivileged user namespaces causing lots of security issues to surface because certain parts of the kernel's code was written with the assumption that only "root" would be able to invoke it, so if you already were "root" securing this didn't matter. There are many, many more issues similar to these, too many for me to write about, and probably many that are way beyond my comprehension. I don't think it's wrong to say that Linux has not been very secure.

westurner 3y ago

"Maestro: A Linux-compatible kernel in Rust" (2023) https://news.ycombinator.com/item?id=38852360#38857185 https://news.ycombinator.com/item?id=38852360#38857185 ; redox-os, cosmic-de , Motūrus OS; MotorOS

cedws 3y ago

It sounds like a cool project and I hope it continues development, but there is such a huge graveyard of such projects that have never gone anywhere that I struggle to get excited about them anymore. Replacing Linux is really hard, even for specific uses like cloud.

seanw444 3y ago
Replacing Linux is hard even for the BSDs, and those are very well established and considered basically next-in-line for similar tasks.
- haarts 3y ago
  You think BSDs are next-in-line? Why is that? I have the distinct impression BSDs will remain niche. Something radical (like MotorOS?) seems more likely.
  - dylukes 3y ago
    What do you think is running on your favorite game consoles (that aren't Xbox)? I'll give you a hint, it isn't Linux. Nintendo is using a custom OS but with a huge chunk of user space borrowed from FreeBSD. Sony on the other hand just went and forked FreeBSD outright. You might also want to look into what OS are being used for server environments. A lot more BSD there than you might have initially guessed.
    - xxs 3y ago
      > your favorite game consoles unless it's a steamDeck, ofc. No love for consoles, though.
      - dylukes 3y ago
        If I'm in the mood for being a little glib... where do you think that Linux distribution that SteamOS is based on got its user-space (or more seriously, its drivers?) Also: the SteamDeck is by any reasonable standard a console. It just happens to run a windowed environment out of the box. Don't be that guy. If you want to pump Valve, focus instead on their contributions to the Wine project.

faitswulff 3y ago

The author wrote this on reddit: > What does "Rust-first" mean here? It means not only that both the (micro) kernel and the drivers are implemented in Rust, but also that Rust is the first (and only, at the moment) language that userspace programs can be written in. > Although technically one can reverse-engineer the Rust-based ABI and the provided Rust toolchain to write apps for Motor OS in e.g. C, that is some work. But standard Rust programs (using standard Rust library, without FFI) will just compile and run - see e.g. https://github.com/moturus/motor-os/tree/main/src/bin/httpd https://github.com/moturus/motor-os/tree/main/src/bin/httpd. > This Rust-first approach is rather unique, as e.g. Redox uses relibc and C-based kernel ABI as the glue... https://old.reddit.com/r/rust/comments/190znk5/mot%C5%ABrus_os_motor_os_a_rustfirst_operating_system/ https://old.reddit.com/r/rust/comments/190znk5/mot%C5%ABrus_...

glonq 3y ago
> Rust is the first (and only, at the moment) language that userspace programs can be written in. Shouldn't be much effort to make it compatible with Crablang, no? /s
mananaysiempre 3y ago
Unstable, undocumented, compiler-(version-)dependent ABIs are the reason Haiku for x86-32 still has to use GCC 2.95. I’m never not going to welcome a hobby OS, especially in an interesting language (still have fond memories of House). By all means, go for it. Just... it’s important to be aware of the prior art here.
- zokier 3y ago
  Haiku was stuck on old GCC only because they wanted to support old BeOS binaries? But binary backwards/forwards compatibility is not requirement for many use-cases.
  - pjmlp 3y ago
    Haiku without the BeOS applications wouldn't have that much to show. This is the way to go on platforms that favour shipping binaries instead of source code. Outside Linux/BSD, stable ABI across OS versions is a concern, hence why Apple, Google and Microsoft platforms all offer ways to keep this going.

paulirotta 3y ago

I was skeptical at first- the healthy approach to any new tech. But thinking again, the efficiency and security gains from stripping away layers of cruft after rather compelling

airtonix 3y ago

[dead]

bluejekyll 3y ago

One thing I keep hoping to see in all of these kernels in Rust is an async first kernel. Is there something that makes this particularly difficult or do folks not see the value in it? I know from following along with Phil Oppermann’s OS in Rust series that is definitely possible, but these last few OS’ in Rust seem to not be attempting this, https://os.phil-opp.com/async-await/ https://os.phil-opp.com/async-await/

spullara 3y ago
There are so many posts about how async Rust isn't the right abstraction it would surprise me if someone doubled-down on it right now.
pjmlp 3y ago
It would help if async Rust was actually a fully done feature, instead of half way there. As Niko Matsakis puts it, async/await is Rust in hard mode, you don't need that when having to also worry about everything writing an OS from scratch entails.
- bluejekyll 3y ago
  Yes, there might be some significant gaps that make kernel development much more difficult. Off the top of my head, the current state of async traits is still being fleshed out, but it’s coming. The async_trait macros rely on boxing, so I could see that as a big downside in the kernel space. And I know that we don’t yet have an async streaming interface stabilized in the std lib. I’m not sure if these are blocking issues or not, thus my question.
  - surajrmal 3y ago
    async trait macros are no longer needed for the majority of use cases as of 1.75. additionally, I doubt a hobby OS will hit bottlenecks where boxing like this is a problem. Async streaming not being standardized as an interface would only matter if the kernel wanted to use libraries, and I would assume most kernels will be light on external dependencies so it shouldn't matter. If you are targeting a posix syscall interface, a lot of that doesn't really require async to pull off. You need to switch threads whenever an operation is blocked rather than run unrelated work on that thread. I suppose you can still model that with async, but it certainly doesn't feel as helpful.

Ericson2314 3y ago

> Nix OS all exist because of Linux's complexity That said, I would be thrilled to build this and other alt OSes and their userlands with Nix / Nixpkgs :).

nicce 3y ago
> because of Linux's complexity Linux is not complex but there are some design issues with C ABI and glibc stability and symlinks.
- emmelaich 3y ago
  Indeed I would say Linux is as simple as possible given various goals ... however, 'simple as possible' can still be complex!
  - Ericson2314 3y ago
    Linux is definitely complex. Linux's ABI stability (good!) also means we cannot really blame C / glibc so much, it has internalized that problem. I agree the goal posts make much of this complexity inevitable, but the solution is to stop trying to intersect everyone's objectives into a much narrower and daunting goal. The big galaxy-brained goal should be: 1. Make more of Linux Rust. 2. Leverage Rust's superior expressive power to switch from a "configuration" mindset (Linux's current gazillion options on single giant code base) to a "composition mindset": we want to use types to combine various subsystems in various flavors. 3. Something like this and Linux proper can share lots of library code / subsystems. We can blur the line between "separate kernels" and "separate 'distrobutions' mixing and matching stand-alone Linux components". I think this is the only sustainable way to reign in the accidental complexity.

weinzierl 3y ago

Sounds interesting, but it also reminds me of what Linus once said when asked about fearing competition. From my memory his answer was something like: I really like writing device drivers. Few people like that and until someone young and hungry comes along who likes that I'm not afraid of competition.

wavemode 3y ago
You don't need very many device drivers to run in a VM. The real question is, how useful is a non-C-compatible, non-Linux-compatible, VM-only OS. Maybe a little bit, for microservices?
- gpm 3y ago
  Is this really non-C-compatible? Seems like it just needs a libc layer written in rust. Most C programs don't make syscalls directly anyways (on some OSes they aren't even able to). The rest of the question is definitely fair though.
- weinzierl 3y ago
  According to The American Heritage Dictionary an operating system is: "Software designed to control the hardware of a specific data-processing system in order to allow users and application programs to make use of it." The question is how much Operating System is a software that delegates all "control the hardware" parts to the layer below. I know there are wider definitions of OS, but my point is this is not going to replace Linux. Without question projects like MotorOS are sill useful. Besides practical applications it's a nice idea. Just recently I wrote in another thread that I would love to have a glimpse into an alternative universe, where Pascal had won over C and everything was Pascal-based. The idea to have everything Rust-based is even more exciting.
  - deleted 3y ago
    [deleted]
- slau 3y ago
  It's relatively trivial to have a `FROM scratch` dockerised microservice in Rust. I'm not convinced having an entire OS brings that much to the table.

micahcc 3y ago

> a simple multi-processor round robin (SMP) > the kernel is very small and does not block, so does not need to be preemptible I don't believe you and I don't even need to look at the code to know this is false.

wavemode 3y ago
Care to elaborate?
- deleted 3y ago
  [deleted]
- jacquesm 3y ago
  From the docs at the top of scheduler.rs // The scheduler. // // As the kernel supports wait/wake/swap, and no blocking in the kernel // other than wait, any kind of more sophisticated scheduling policy // can be implemented in the userspace (in theory; maybe tweaks are needed // to make things perform well in practive). // // So the default scheduler here is rather simple, but should work // for a lot of use cases. // // Priorities: see enum Priority // Cpu affinity: a single CPU hint // Sched groups: none: everything within a priority is round robin, so // a process with many threads will negatively affect a process with few threads. // // This should be OK for most situations: if there is a need for isolation, // just use another VM. That's elaboration enough I think?

andyferris 3y ago

Out of curiousity, why would a small kernel take a whole 200ms to start on a modern computer? Wouldn't it need to initialize some metadata for the memory pages, mount the filesystem, and try to launch an init process? I suppose there might be an ethernet driver and possibly something to pipe logs to ("stdout" for the VM) to initialize. Shouldn't that all take a few microseconds? Or is all the slowness in the host preparing the resources? (as in QEMU and KVM?)

deleted 3y ago
[deleted]
Veserv 3y ago
In general, lots of hardware requires long waits to initialize. As in, program a value then you must wait 50 ms before querying the registers or the hardware will give you non-deterministic garbage with no error indication. So, it depends on the hardware, though you are unlikely to need more than 1 second on anything other than truly degenerate hardware assuming you are initializing devices in parallel.
- azinman2 3y ago
  But it’s only for VMs. So what hardware are you initializing?
  - Modified3019 3y ago
    I pass through a GPU and USB hub to a VM running on a machine in the garage. An optical video cable and network compatible USB extender brings the interface to a different room making it my primary “desktop” computer (and an outdated laptop as a backup device). Doesn’t get more silent and cool than this. Another VM on the garage machine gets a bunch of hard drives passed through to it. That said, hardware passthrough/VFIO is likely out of the current realistic scope for this project. VM boot times can be optimized if you never look for hardware to initialize in the first place. Though they are still likely initializing a network interface of some sort. “MicroVM” seems to be a term used when as much as possible is stripped from a VM, such as with https://github.com/firecracker-microvm/firecracker https://github.com/firecracker-microvm/firecracker

ianbicking 3y ago

It could be cool if there was a WASM container built in Rust that could run in this OS. I don't really have a sense of how complicated a WASM container is, though it seems non-trivial to even decide what such a thing is, so it would be nice not to treat the WASM container itself as the OS (i.e., more room to safely experiment). WASM also seems like it dodges the ABI issue by being more explicitly about composition instead of shared binary data structures.

gpm 3y ago
When you say wasm container, you mean something like wasmtime that provides a non-browser wasm runtime? https://github.com/bytecodealliance/wasmtime https://github.com/bytecodealliance/wasmtime
- ianbicking 3y ago
  Yeah, just that sort of thing. I mean, heck, can you put #!wasmtime at the top of a wasm file? Would such a file literally run right now (assuming #! is supported, and wasmtime doesn't hit any MotorOS Rust limitations)?
  - ianbicking 3y ago
    Serendipitously I see this article about WASM and how it has all these nice properties as a runtime abstraction: https://wingolog.org/archives/2024/01/08/missing-the-point-of-webassembly https://wingolog.org/archives/2024/01/08/missing-the-point-o...

pipeline_peak 3y ago

I don’t think swapping out the first order language does anything other than boxes out potential support.

jacquesm 3y ago

It isn't hard to start writing a new operating system. It is very hard to support that operating system for the next 5 decades.

xyst 3y ago
Linus Torvalds resolve is unbreakable
- jacquesm 3y ago
  Yes, it's most impressive. I couldn't do it. That kind of long term focus is an incredible asset.

keithalewis 3y ago

[flagged]

deleted 3y ago
[deleted]

aaa_aaa 3y ago

I think the most close to production ready "mostly" Rust OS is Fuchsia.

Octoth0rpe 3y ago
I've seen this stated in a couple places, but IIRC the kernel is C++ and the UI is Dart. I assume lots of things in between could be in rust, but do you know which? daemons, services, drivers, window server?

jmull 3y ago

> ...Docker, Nix OS, "serverless", etc. all exist because of Linux's complexity Yeah, this seems like it's more directly competing with those than Linux. I'd want to see those addressed in the "Why?" -- that is, why MotorOs instead of Docker, etc.?

btreecat 3y ago
My first thought was "can this run my containers?"
crabbone 3y ago
> why MotorOs instead of Docker, etc.? It basically says it already: > Motūrus OS is a microkernel-based operating system By and large, microkernel and containers solve the same problem. Except, I wouldn't call containers a "solution", more like a workaround. Not in the sense that containers by themselves are a workaround, but the way they are used is a workaround for the same problem. The way containers are used today, especially in the context of Kubernetes, is to finely slice the available physical resources. The orchestration allows to manage which part of the application gets what slice of the resource, thus attempting to cut down on waste that's typically generated in the world where resources are managed through VMs. Where the typical problem is that a single VM will require too many resources because of the bloated OS it needs to run and because it's hard to create VMs with very limited resources, since OSes usually come as a package deal. So, containers "solve" the problem by giving up VM optimization -- instead it's usually beneficial to create very beefy VMs, on top of which then a new virtualization layer is created with containers. This minimized the VM waste, but doesn't get rid of it entirely, and, of course, creates a lot of complications with all the indirection resulting from two-tiered virtualization. Microkernel is the opposite of this "solution" -- it's the attempt to make OSes more modular, and as such less demanding of resources. Ideally, in the world if microkernels you don't need containers (at least not for the thing they are usually used today): your VMs can slice the resources in the way at least as efficient as containers do (or, hopefully, even better). So... to predict the next possible question: why containers are so popular and micorkernels aren't: it's because the later is harder on the applications (even when applications don't actually need something, they often use it because it's available in a full-blown OS, applications aren't usually written with resource scarcity in mind). Secondly, containers, essentially, exist on top of somewhat uniform, somewhat stable interface of Linux kernel. Micorkernel VMs would expose users to the zoo of ideas hardware vendors put into their products, making portability difficult. Finally, for all its flaws, Kubernetes is a big system with many (even if not so great) solutions for many problems. So, programmers who fear technology feel like it gives them a safety net and will allow them to program from a more comfortable position of modifying YAML files, copying the most upvoted answers from StackOverflow. There won't be such an easy cake in microkernel world.

duped 3y ago

> Docker, Nix OS, "serverless", etc. all exist because of Linux's complexity Docker and NixOS exist because of userspace problems with package management and serverless exists because businesses want to pay for compute on demand.

dravigon 3y ago

isn't this similar to other microkernel and something like wasi/wasm kinda addressing this the correct way ?

lasiotus 3y ago

I'm the project author/dev. Thanks a lot for posting this, and for comments/discussions! I see two main concerns raised here: (a) long-term viability and support (b) compilers, binary compatibility, etc. While the first concern is definitely valid, and without a community this project will not succeed, I do believe that potential benefits of Motor OS (or a similarly focused/structured project) will eventually result in a widely used new operating system. There are major problems with Linux inside VMs (and sometimes outside), and the Linux devs are not focused on this enough to clean things up anytime soon. I work on Linux Kernel at my day job, I know. Re: compiler instability, binary compatibility, etc.: I'm sorry, I don't understand what is the issue here. The latest Linux kernel can be compiled with different GCC or LLVM toolchains on x86_64, and the result will happily run old binaries compiled years ago with who knows what. repr(C) structs in rust are stable... So why so many concerns here? Again, thank you all for your comments and questions - I'm happy to answer more (at least until my day job kicks in).

xyst 3y ago

Kind of cool as a hobby OS. But would be nice to backup your claims regarding performance with actual numbers.

lasiotus 3y ago
The lainding page at https://github.com/moturus/motor-os https://github.com/moturus/motor-os explicitly says that both networking and file I/O are slow and have to be improved. The only claim is about fast bootup, and the number is there, and can easily be verified. Where do you see any unsupported claims re: performance?