10 ms·
A completely-from-scratch hobby operating system
- rurban 5y agoThen why in the heck is he going for POSIX compatibility, when he can afford the luxury of not having to deal with blocking syscalls and all this crap? Much easier and safer multithreading. Also faster. And why we are there, why not a safer microkernel, keeping everything in userspace? Questions over questions.
- craigmart 5y ago>why in the heck is he going for POSIX compatibility Because existing desktop applications can be ported to ToaruOS >why not a safer microkernel, keeping everything in userspace? This is a design choice, microkernels aren't necessarily better than hybrid, they're slower, harder to debug and process management can be complicated
- fouc 5y agoIsn't QNX a microkernel? I remember it being known for being quite fast?
- guenthert 5y agoAs a real-time OS it is known for deterministic response times. If it were exceptionally fast (and licenses cheap enough), you'd see hosts in the TOP500 using it.
- eggy 5y agoI agree 100%. QNX is lurking in products you may use. It was the OS for the show control system that is used throughout entertainment where real-time is necessary for the safety of the devices it controls, which also have hardware safety at the lower level. I would drop into the QNX terminal for certain tasks. Unfortunately, you used to be able to download the show control software and play with it, but it has since been bought buy a company that sells it with the equipment they rent, so you need to buy trainging and it is behind their wall now. Not QNX, but the show control software that runs on QNX.
- p_l 5y agoTOP500 is chock full of microkernels though, even if the "I/O nodes" would run Linux
- guenthert 5y agoI'd like to see more information about that. I remember that Penguin Computing offered such some 15 years ago, but don't know where it was deployed or still is. Cray and IBM had also such a concept for their superclusers in the past, but are they still using such? The one HPC environment I worked on (a major car manufacturer in Europe) used plain RH Linux on all nodes as recently as three years ago. The current #1 (Fugaku) uses IHK/McKernel as kernel for the actual payload. The previous #1 (IBM Summit) seems to use RH Linux though. Perhaps, since the most performance critical part is run by and within the GPGPU(s), the actual OS doesn't matter all that much (for performance -- it matters of course for programmer's comfort/efficiency).
- p_l 5y agoThere used to be a lot of "special microkernel on compute RPCing to Linux on I/O" on Crays and the like. Hard to say how prevalent it is now, and most annoyingly I can't recall the names. (Charon?)
- bregma 5y agoNo, it's more of a nanokernel. It's very fast. Full disclosure: I maintain QNX toolchain.
- md8z 5y agoI'm very confused by this comment. There are a ton of other things you need to implement if you want to have desktop applications. POSIX does not specify any APIs for graphical applications. You might be thinking of something else. If you want to support the lion's share of desktop applications, it would actually be better to implement the Win32 API...
- craigmart 5y agoSorry, I meant software in general but wrote "desktop applications" instead. Anyway the sentence is still valid, even if you'll have to implement other things such as the graphical interface, the POSIX compliant code won't need modification
- md8z 5y agoIf you're taking an app built for Linux or GNU or BSD, then it probably will need modification, as those systems have various extensions on top of POSIX.
- eggy 5y agoJust curious how hard it would be to forego POSIX entirely if you were building an OS. I know TempleOS is entirely from scratch. I'd like to implement a small LISP like SectorLISP [1] (see yesterday's posts too on HN). I don't know much about building my own OS, so I'd like to start with something like MenuetOS (my first PL was asm), SerenityOS, TempleOS, or this one. I'd like it to be completely an 'island', i.e. POSIX not a requirement. I want to use it to hack on in isolation without any easy copy/paste shortcuts. I know Mezzano exists, and it has booted on bare metal, but I would like to start with the OS's above, implement my own LISP, and go from there. Any other OS recommendations base on my ignorant, but wishful, reqs above? I realize there are some others in Rust too. Thanks! [1] https://github.com/jart/sectorlisp https://github.com/jart/sectorlisp
- adrian_b 5y agoSomeone who would make a new OS, should define a completely new system call interface, as it is likely that now it is possible to conceive a better interface than 50 years ago and anyway if it would not be different there would be no reason to make a new OS, instead of modifying an existing one. Nevertheless, the first thing after defining a new OS interface must be writing a POSIX API translation layer, to be able to use without modifications the huge number of already existing programs. Writing a new OS is enough work, nobody would have time to also write file systems, compilers, a shell, a text editor, an Internet browser and so on. After having a usable environment, one can write whatever new program is desired, which would use the new native OS interface, but it would not be possible to replace everything at the same time. Besides having a POSIX translation layer, which can be written using as a starting point one of the standard C libraries, where the system calls must be replaced with the translation layer, some method must be found for reusing device drivers made for other operating systems, e.g. either for Linux or for one of the *BSD systems. Nobody would have time to also write all the needed device drivers. So there must exist some translation layer also for device drivers, maybe by running them in a virtual machine. The same as for user applications, if there is special interest in a certain device driver, it should be rewritten for the new OS, but rewriting all the device drivers that could be needed would take years, so it is important to implement a way to reuse the existing device drivers.
- yjftsjthsd-h 5y ago> microkernels aren't necessarily better than hybrid, they're slower, harder to debug and process management can be complicated I was basically on board, but how are they harder to debug? I'd think being able to run components in userspace would make debugging way easier.
- mjgerm 5y agoYou are now debugging a distributed system.
- jatone 5y agofun fact: you already are in linux. being a monolith doesn't change the nature of the problem.
- yjftsjthsd-h 5y agoAre you? I'm paddy pretty sure I can run a single Linux and point a single gdb at it[0] and debug it in a single memory space; I don't think you can do that with a microkernel. [0] possibly resorting to UML, but still
- yjftsjthsd-h 5y agoOh, good point; I was thinking at the component level
- VWWHFSfQ 5y agoAnd why in the heck didnt he write it in Rust! ...
- fouc 5y agoAt least he didn't write it in node.js /s Other interesting alternatives to c or rust might be vlang or zig.
- xvilka 5y agoFor V language there is Vinix[1], for Zig there's a kernel[2] only. [1] http://vinix-os.org/ http://vinix-os.org/ [2] https://github.com/jzck/kernel-zig https://github.com/jzck/kernel-zig
- tentacleuno 5y agoWell, that's already been done. https://node-os.com https://node-os.com
- turbinerneiter 5y agoYou can do all of this in your own hobby project.
- ape4 5y agoYou need POSIX to compile nearly any program
- rurban 5y agoThat his advantage. He doesn't want to, rather write everything anew from scratch. Also signals. With a microkernel you won't need signals
- mdp2021 5y agoSo, a preliminary problem for many OS-from-scratch projects intending to create "New Different Great OS" will be to have a language implementation which does not expect posix. It depends on the goal. If you want to compile pre-existing software for it, or if you want to really "start anew".
- spijdar 5y agoTrue, and as another commenter hinted at, you're free to do your own hobby project however you please, there's not a wrong way to do so if you find it fulfilling. But at the same time, it does make me sad that most hobby OSes end up seeking POSIX compatibility, because that means being destined to essentially either be another unix-variant or develop a unix-variant inside some subsystem of your OS. Yes, being unix-like means you gain access to a trove of software and libraries that makes porting applications much easier, but it also limits the potential be truly different and experiment, as your end result will look like "yet another unix" with misc. improvements. Since I think the enjoyment of building something like this comes from the satisfaction of building an OS from the ground up, I don't think it matters, but it would be cool to see more hobby OSes try more exotic ideas and runtimes.
- spicybright 5y agoFully agree. We need more Temple OS's!
- ranger_danger 5y agoFor *nix maybe, but there are many other non-POSIX operating systems in use today.
- the-dude 5y agoShallow dismissal of other people's work.
- klange 5y agoWhen I started the project a decade ago, my aim was specifically to understand POSIX/Unix and to "learn by doing", so of course there's some POSIX-like elements underpinning the design. Back before I decided that literally anything can be in scope and was using third-party components, this aided in porting software, as other commenters have pointed out. These days, it gives more purpose to things I build for the OS if they can also reasonably be built for Linux or macOS - things like my editor (which I wrote for the OS and now use as my daily driver in Linux), or my Python knock-off. As for the microkernel bit, this might sound like circular reasoning but I didn't go for a microkernel because no one really uses microkernels. It's not that I think microkernels are a bad idea, ToaruOS does push plenty of stuff into userspace. Rather, my main goal at the moment is to provide an educational resource that more accurately models the way "real" OSes work than the typical academic OS projects.
- amitport 5y agoHow this compares with xv6 for first OS course purpose? Is it viable to use this to teach the same things? (Paging, inter process communication, threading...)
- bitcoinmoney 5y agoAlso interested in this question. Never finished the MIT os course utilizing xv6.
- klange 5y agoI really like xv6 for its simplicity and restricted scope. It feels like a better Minix in the sense of being a codebase intended for university coursework. If your goal is learning the very basics of what an OS does, xv6 is a good place to start. ToaruOS can cover the same ground - of course there's paging and IPC and threads - but I don't have a cache of PDFs sitting around describing it, and there's no intentionally missing bits waiting for a student to fill them in for homework. If xv6 is "Operating Systems 101", then ToaruOS is a 300-level seminar/workshop. My intended audience is hobbyists who are looking for a reference for writing their own OS - the sort of people we often find on osdev.org, or the #osdev channel on Libera, who have stumbled through an old tutorial on x86 bring-up and want some example of how "the next steps" work.
- Semaphor 5y agoBack in 2014 this was posted before, so before the first "release" version and when there were external dependencies: https://news.ycombinator.com/item?id=8566217 https://news.ycombinator.com/item?id=8566217 (26 comments).
- kevincox 5y agoAt that time the repo probably looked like this: https://github.com/klange/toaruos/tree/9f34619078f92bd7d9815c541a3a01ff9a86d9d1 https://github.com/klange/toaruos/tree/9f34619078f92bd7d9815...
- behnamoh 5y agoI like that the main website looks like the OS itself! https://toaruos.org/ https://toaruos.org/
- shiftoutbox 5y agohttps://web.archive.org/web/20210423232715/https://toaruos.org/ https://web.archive.org/web/20210423232715/https://toaruos.o...
- _tom_ 5y agoLooks like the website has been hugged to death, for now. I wonder if the web server is hosted on toarus?
- bogomipz 5y agoUnder features, the first bullet point is "Dynamically linked userspace." Can someone say what this means? How is this different than a userspace made up of dynamically linked utilities and shell?
- afr0ck 5y agoLibraries are shared by programs and refrences to library routines are automatically resolved at program load time.
- kenferry 5y agoIt’s not different, but it’s a feature that this is included vs only supporting static linking.
- bogomipz 5y agoThat makes sense. I misinterpreted that point. That is indeed an important feature and one that's easy to take for granted. Cheers.
- jstx1 5y agoI'm very impressed any time I see people pouring so much work over long periods of time into hobby projects.
- deleted 5y ago[deleted]
- bluecatswim 5y agoNo inbuilt documentation system called Index?
- picture 5y agoOh that would be awesome
- fouc 5y agoNice. This came before SerenityOS and yet seems similar in approach.
- picture 5y agoI love that the OS seems to be named after Toaru Majutsu no Index and Toaru Kagaku no Railgun. Misaka the kernel and Kuroko the interpreter are named after iconic characters from the series.
- junon 5y agoThis is by Klange, the owner of the #osdev channel on Libera chat, associated with osdev.org. Super nice individual and always posts updates and interesting tidbits about ToaruOS. Strange, but welcome, to see it on the frontpage! :)
- klange 5y agoI wouldn't say I "own" that channel in any respect, I'm just responsible for dragging it over from Freenode during the exodus... we've got the same same staff as we did before the move and I'm not even the "founder" according to ChanServ.
- CSSer 5y agoJust to clarify: are you active/a mod in that channel? Now I’m curious how the GP got the idea you own it.
- klange 5y agoI am one of the more active channel ops, so it's not an unreasonable mistake to make.
- junon 5y agoBecause I created the channel during the Freenode exodus to prevent it being poached and transferred ownership to Klange when a few people mentioned he was the person that should have it.
- mst 5y agoThere are quite a lot of channels like that - the week I spent running around warning as many big projects as I could get hold of that there was clearly no way out seemed to help a lot, but of course there was still the problem of "who's actually online when the hammer drops to -execute- the planning people had done". It was kinda exhilerating watching practically the entire userbase move as one, but I'd really really rather not have to do it again :D Still kind of amazes me that given the number of signatories I got nobody thought "hey, maybe this isn't going to work" but it turned into an excellent mutual support resource for those of us moving instead: https://gist.github.com/shadowcat-mst/998cea12794768bdb3da2daeff31baad https://gist.github.com/shadowcat-mst/998cea12794768bdb3da2d...
- spicybright 5y agoI really like the UI! Most hobby OS's just do the same bland patterns most of the time.
- fieryskiff11 5y agoIs it written in HolyC?
- black_13 5y agoThis is why i come to this site.
- saladuh 5y agoA certain (weeb named, from scratch, hobby) OS.