10 ms·
Writing an OS in Rust to run on RISC-V
- mikece 4y agoHow long before we create a meta language that can then be auto-translated to any number of languages — Rust included - before final compilation? (Or has this been tried 33 times and always failed?)
- nordsieck 4y ago> How long before we create a meta language that can then be auto-translated to any number of languages — Rust included - before final compilation? What reason would someone have for doing such a thing?
- asddubs 4y agoso a language that has the constraints of every single programming language at once? (at least if your goal is to have the output be halfway legible/idiomatic)
- drakythe 4y agoSomething like Haxe? https://haxe.org/ https://haxe.org/ Obviously not “any” language but it has more compile targets than your average bear.
- anyfoo 4y agoThat's what Rust, C, and many other languages are. "Meta languages" that ultimately compile into any number of machine language dialects.
- MaulingMonkey 4y agoThis is arguably already the state of things. Rust might get compiled down through MIR, down through LLVM IR, down to assembly or wasm... which then might be JIT or AOT (re)compiled into other bytecodes... which might perhaps be decompiled back up to C... and C might be retranslated back to horrific unsafe-spamming Rust by the likes of https://c2rust.com/ https://c2rust.com/. We've come full circle! The main issue is that retranslating high level languages into other high level languages isn't something that there's actually a lot of demand for, especially commercially, especially given the N x M translation matrix going on. So a lot of the projects "stabilize" (get abandoned). And automatically translating between the idioms of those languages gets even nastier in terms of matrix bloat. Well, you've got stuff like MSIL and JVM bytecodes which are higher level, and preserve more type information, and can be compiled to / decompiled from while still preserving more structure, but they still form competing incompatible ecosystems.
- anyfoo 4y agoThe very premise is bad. You want to use Rust precisely so that you can write Rust code. The selling point of Rust and most other languages is that they provide a sane, safe, convenient, easy... (pick some of them) interface to the programmer.
- l33t233372 4y agoThe Unity game engine created a library that transpiles C#(technically the byte-code) into C++[1]. The generated code isn’t particularly readable and it comes with some security issues[2] but the upshot is you have the benefits of a C++ compiler. [1]https://docs.unity3d.com/530/Documentation/Manual/IL2CPP.html https://docs.unity3d.com/530/Documentation/Manual/IL2CPP.htm... [2] https://github.com/djkaty/Il2CppInspector https://github.com/djkaty/Il2CppInspector
- j16sdiz 4y ago> the benefits of a C++ compiler. Is there any? I think most modern compiler share the same codegen backend. You don't get the benefit of the frontend when you transpile.
- MaulingMonkey 4y ago> Is there any? I think most modern compiler share the same codegen backend. JIT-focused .NET is one of the ecosystems disparate from AOT-focused LLVM. While there's a bit more cross pollination now, at the time the C# to C++ transpiler was authored, there was much less so. I've taken a stab at porting mono to a new platform at a similar time period - it was rather nontrivial (I ran out of time and thus failed.) Worse still, many of Unity's targets (e.g. iOS, XB1, ...) explicitly ban JIT technology in the name of security, which wasn't something I had to deal with. A MSIL bytecode -> C++ translator might be pretty quick and dirty... yet effective, gives you AOT compilation, avoids the need to explicitly target every architecture by hand. For all it's faults, it's not too terribly hard to figure out how to compile C++ for a given platform, typically, generally requiring exactly zero reverse engineering.
- jalino23 4y agowhat I need is a language the can directly call c and cross language compile with c so I don't have to write c! especially that nasty pre processor. cause there so much good c libraries that I want to consume and they're only available in c!!!
- mikesun 4y agoZig. https://ziglearn.org/chapter-4/ https://ziglearn.org/chapter-4/
- jey 4y agoI use Julia for this. Its ccall facility is very handy for C FFI calls, and Julia itself is very fast and performant with just a bit of type annotation. https://docs.julialang.org/en/v1/manual/calling-c-and-fortran-code/ https://docs.julialang.org/en/v1/manual/calling-c-and-fortra...
- jalino23 4y agojulia is so nice! can’t believe I didn’t try this before! thank you very much
- ilyt 4y agoCalling C from rust is easy enough
- josephg 4y agoLLVM languages can all natively link with C code, since any LLVM bytecode can directly call other LLVM bytecode. So Zig, C++, Obj-C, Swift and Rust can all be compiled together. But C compatibility is everywhere. Almost every language has a FFI (foreign function interface) for C code. C interop shows up in every language because at the end of the day every language needs to be able to talk to the operating system. For example, to read and write files your language needs to call functions in libc (or equivalent). And that library is written in C. So, Nodejs can call C via has native modules & NAPI. Java can call C through JNI. Ruby has the ffi gem. Python, Luajit, Go, C#, ... the list goes on. They all have a mechanism to call C code. If you're looking for another language that can live alongside your C code, you can choose any.
- askvictor 4y agoEiffel initially compiled to C, then used the machine's C compiler to compile to native code. Later it would support outputting to java bytecode and .net CIL; shouldn't be any reason not to have other output formats.
- jcranmer 4y agoSo the main question is... why? The closest you get to this sort of thing in practice is something like a parser generator, or an automatic interface generator. ANTLR and Tree-sitter both allow generating parsers in a variety of languages, but their input is basically a description of an AST in a highly specialized and extremely declarative context. Once you start having actual logic in your code generation, there is very little benefit to codegening to a high-level language instead of going directly to a compiler IR, a bytecode, or assembly directly. The places where you see that happening--JavaScript being the biggest one--is largely limited to where there is no other alternative.
- elcritch 4y agoNim compiles to C, C++, Objective-C, and Javascript. Technically it'd probably be possible to compile down to Rust. It's gc is essentially wrapping in Rc's. Though not sure it'd bring much vs linking to Nim's C output.
- fsckboy 4y agoC++ and Objective C started out as an extra layer compiling to C
- dwrodri 4y agoHIGHLY highly recommend checking out Stephen Marz's OS blog, where he walks through building a basic RISC-V OS in Rust step by step: https://osblog.stephenmarz.com/ https://osblog.stephenmarz.com/
- kop316 4y agoXous, the OS that runs on the precursor, may be of interest to look at too: https://github.com/betrusted-io/xous-core https://github.com/betrusted-io/xous-core It is written in Rust and is targeted for a RISC-V
- panick21_ 4y agoRedoxOS is the biggest and designed for desktop. Xous for devices. There is also Hubris and TockOS for embedded and IT. There are a few others.
- gxt 4y agoPlease please please, all new OSes need to batch all syscalls by default, and all security contexts need to take into account all of program-origin, program, and user. Also please limit whatever your equivalent of root to deny and delete. Thanks
- Nanana909 4y agoHi, just out of interest since I am a novice in the area, I’d appreciate if you could elaborate on this comment.
- ddulaney 4y ago> batch all syscalls by default Switching from user mode to kernel mode and back (a “context switch”) is expensive. Traditionally, OSes did that on every system call. You can go faster if you have some mechanism for sending more than one system call in a single context switch. > security context Basically, what is a program allowed to do? Traditionally, Unixy (nowadays, Linux and friends) limit this based on just the current user, assuming that the user trusts every program they run. But nowadays you often don’t trust every program you run, and want it to be in its own limited sandbox. > limit […] root Most OSes have a super user that can do anything (pid 0 on Unixy things; Windows is more complex but still has them). They’re saying to not have that, instead make the super user only capable of stopping problematic things, not creating new things. This is the one I’m more ambivalent on. At some point something has to set up the whole system, and the thing that kicks that off is going to look pretty superuser-like to me.
- gxt 4y ago1. Batching syscalls (and to me that implies copy-exactly-once IO, AKA Zero Copy in Linux/rust) means you can better manage the cost of performing syscalls and IO in high throughput and low latency use case. Eg if you expect a bunch of network packets, the physical card, driver, network stack should split packet headers and data in distinct buffers, each contiguous, in memory. With batched syscalls, you could also instruct the kernel to memmap a file in memory, and finally combine both syscalls into a single copy from Io to Io giving the memmap buffer as the output buffer to the network stack. I don't know how this could be done today, even with iouring, but I expect this would significantly outperform existing solutions as there would be a single copy operation instead of at least 3. 2. Per origin, per program, and per identity security context I think is required to deal away with the current prerequisite of all web browsers that the underlying system be uncompromised. Basically a world where every js bundle gets executed with it's own user as its own process and having to explicitly request access to your data. 3. Combined with the above to limit the risk of compromised root accounts, if they are limited to causing DOS and data loss it's much less dangerous than a world where your entire life can be usurped by assholes with a 0day. This implies major changes in driver architectures of OS/kernels but I think it's entirely unreasonable not to make these changes. The world has changed since the 90s.
- Octokiddie 4y agoWhat's the simplest way to get this running on hardware without emulation, assuming that one has no RISC-V hardware at all?
- FullyFunctional 4y agoEither port to a processor you have or build a RISC-V core yourself, say, with an FPGA. There is an abundance of designs you can use, but actually making a simple core that can support this isn't hard if you aren't too ambitious with the performance. (I know of at least one RISC-V core implemented with 74-series TTL chips and a vacuum-tube implementation is surely in the works somewhere).
- kqr2 4y ago(2019) ?
- gibbonsrcool 4y agoI’m asking this question as someone with zero experience in writing an OS, but who has done limited services development. Would it be an interesting research project to write an OS with the following traits: 1. The purpose of the OS would be to run microservices only 2. The OS would provide the minimum functionality to provide a platform for services to run on the web assembly system interface 3. Some interim solution for providing sockets would have to be built until wasi spec supports it 4. No file system, no writing to disc, OS gets booted from USB, services pulled and initialized from network Apologize ahead of time if this is a naive question and I need to jump into more traditional OS dev to get my bearing. I’m a mobile/services taking a break and I just happen to currently be intrigued by Rust and WASM.
- andrewchambers 4y agoYou could start your search looking for 'unikernel' tools, a few already exist.
- gibbonsrcool 4y agoI hadn’t heard this term before so that will help my search.
- girvo 4y agoMirageOS is not Rust, but in the ballpark! https://mirage.io/ https://mirage.io/
- gibbonsrcool 4y agoI am also interested in OCaml, so thanks for sharing this. I’ve heard it’s higher level than Rust and has GC. I’ve spent most of my career using Java and after a few weeks of Rust I love many things about it but feel managing lifecycles and ownership might be too much for me.
- speed_spread 4y ago
- anta40 4y agois there any updated xv6-rust (prefer X64, but at least X86)? I already tried: - https://github.com/connorkuehl/xv6-rust https://github.com/connorkuehl/xv6-rust - https://github.com/tiqwab/xv6-rust https://github.com/tiqwab/xv6-rust Cannot build both succesfully with latest Rust on Mac. Maybe I need to use Linux for this purpose...
- snvzz 4y agoAfter a look at RISC-V's privileged spec, I doubt I'd pick an ISA other than RISC-V to target, if I felt like writing a new OS. Beautiful and simple, as everything else RISC-V.
- xixixao 4y agoI took an OS class but wonder nonetheless: If you aim for 0 compatibility, perhaps even can design your cpu, could an OS be much simpler? Could some standard components (paging etc.) be avoided altogether? Or is what we have a minimal set already?
- a1369209993 4y ago> If you aim for 0 compatibility, perhaps even can design your cpu, could an OS be much simpler? Yes, but a lot (possibly most) of the benefit is likely to be from dropping bug-for-bug compatibility with defects of preexisting operating systems and instruction set architectures individually. (Like x86 segmentation or unix tcsetattr/termios, for a pair of very obvious examples.) For paging specifically, you'll have something functionally equivalent unless your system is deficient in the same ways a 6502 is (no virtual memory or memory protection). You might come up with something functionally equivalent but better, but if so that would be a novel discovery (either in the scientific sense, or in the sense of "I discovered a research paper from 1960 that solves this problem trivially as long as you have larger than 6-bit bytes and more than 64K[0] of RAM."). 0: Something like four rooms of vacuum tubes in 1960 money, which is why it never caught on. (/not-even-all-that-s)
- pocketarc 4y agoThe last part of this comment is brilliant. There may be large swathes of outstanding old research that had requirements that back then were out of reach but are now entirely mundane and easily achievable (even if they still wouldn't have a hope of reaching mainstream because of compatibility).
- t-3 4y agoYeah, you can make an OS incredibly simply. Virtual memory is usually supported in hardware, but files, forking, IPC aren't necessary at all to make a functional OS. The most minimal useful system you could get would probably be something like a bare-metal forth.
- drpixie 4y agoCheck out some microkernel systems (eg. Hurd). They have their problems, but the core OS is very minimal. The microkernel idea is to move everything that can be moved, out to user-space. The minimal OS services is typically thread management, address space management, and some kind of IPC. And if you choose not to provide virtual memory or memory protection, you have a very tiny core OS. Everything else, all the "normal" OS services can be provided by application level programs - device handlers, file systems, network stacks, time services, graphics,... almost everything.
- DesiLurker 4y agoI wonder if old BeOS source code can be opensourced. it would be a great alternative to port on risc-v from the getgo.
- rvense 4y agoDo you know haiku-os.org? Some people did this but the long way round...
- detrites 4y agoWas curious to understand the original Japanese version of this line, emphasis added to the relevant portion: > it's a clean, modern RISC architecture without all the legacy crud of MIPS Particularly the word "crud", and its implied denigration. Indeed, amusingly, the original Japanese translates: > but I think it is a sophisticated RISC ISA with some negative legacies gone
- renox 4y agoWhat's funny is that one man 'crud' is another regret: MIPS had arithmetic operations which detected integer overflow..
- azubinski 4y agoWriting an OS in Rust to run on RISC-V in a blockchain-based flying car
- lfmunoz4 4y ago[dead]