6 ms·

Then why in the heck is he going for POSIX compatibility, when he can afford the luxury of not having to deal with blocking syscalls and all this crap? Much ea

by rurban 5y ago

Then why in the heck is he going for POSIX compatibility, when he can afford the luxury of not having to deal with blocking syscalls and all this crap? Much easier and safer multithreading. Also faster. And why we are there, why not a safer microkernel, keeping everything in userspace? Questions over questions.

craigmart 5y ago
>why in the heck is he going for POSIX compatibility Because existing desktop applications can be ported to ToaruOS >why not a safer microkernel, keeping everything in userspace? This is a design choice, microkernels aren't necessarily better than hybrid, they're slower, harder to debug and process management can be complicated
- fouc 5y ago
  Isn't QNX a microkernel? I remember it being known for being quite fast?
  - guenthert 5y ago
    As a real-time OS it is known for deterministic response times. If it were exceptionally fast (and licenses cheap enough), you'd see hosts in the TOP500 using it.
    - eggy 5y ago
      I agree 100%. QNX is lurking in products you may use. It was the OS for the show control system that is used throughout entertainment where real-time is necessary for the safety of the devices it controls, which also have hardware safety at the lower level. I would drop into the QNX terminal for certain tasks. Unfortunately, you used to be able to download the show control software and play with it, but it has since been bought buy a company that sells it with the equipment they rent, so you need to buy trainging and it is behind their wall now. Not QNX, but the show control software that runs on QNX.
    - p_l 5y ago
      TOP500 is chock full of microkernels though, even if the "I/O nodes" would run Linux
      - guenthert 5y ago
        I'd like to see more information about that. I remember that Penguin Computing offered such some 15 years ago, but don't know where it was deployed or still is. Cray and IBM had also such a concept for their superclusers in the past, but are they still using such? The one HPC environment I worked on (a major car manufacturer in Europe) used plain RH Linux on all nodes as recently as three years ago. The current #1 (Fugaku) uses IHK/McKernel as kernel for the actual payload. The previous #1 (IBM Summit) seems to use RH Linux though. Perhaps, since the most performance critical part is run by and within the GPGPU(s), the actual OS doesn't matter all that much (for performance -- it matters of course for programmer's comfort/efficiency).
        p_l 5y ago
        There used to be a lot of "special microkernel on compute RPCing to Linux on I/O" on Crays and the like. Hard to say how prevalent it is now, and most annoyingly I can't recall the names. (Charon?)
  - bregma 5y ago
    No, it's more of a nanokernel. It's very fast. Full disclosure: I maintain QNX toolchain.
- md8z 5y ago
  I'm very confused by this comment. There are a ton of other things you need to implement if you want to have desktop applications. POSIX does not specify any APIs for graphical applications. You might be thinking of something else. If you want to support the lion's share of desktop applications, it would actually be better to implement the Win32 API...
  - craigmart 5y ago
    Sorry, I meant software in general but wrote "desktop applications" instead. Anyway the sentence is still valid, even if you'll have to implement other things such as the graphical interface, the POSIX compliant code won't need modification
    - md8z 5y ago
      If you're taking an app built for Linux or GNU or BSD, then it probably will need modification, as those systems have various extensions on top of POSIX.
- eggy 5y ago
  Just curious how hard it would be to forego POSIX entirely if you were building an OS. I know TempleOS is entirely from scratch. I'd like to implement a small LISP like SectorLISP [1] (see yesterday's posts too on HN). I don't know much about building my own OS, so I'd like to start with something like MenuetOS (my first PL was asm), SerenityOS, TempleOS, or this one. I'd like it to be completely an 'island', i.e. POSIX not a requirement. I want to use it to hack on in isolation without any easy copy/paste shortcuts. I know Mezzano exists, and it has booted on bare metal, but I would like to start with the OS's above, implement my own LISP, and go from there. Any other OS recommendations base on my ignorant, but wishful, reqs above? I realize there are some others in Rust too. Thanks! [1] https://github.com/jart/sectorlisp https://github.com/jart/sectorlisp
  - adrian_b 5y ago
    Someone who would make a new OS, should define a completely new system call interface, as it is likely that now it is possible to conceive a better interface than 50 years ago and anyway if it would not be different there would be no reason to make a new OS, instead of modifying an existing one. Nevertheless, the first thing after defining a new OS interface must be writing a POSIX API translation layer, to be able to use without modifications the huge number of already existing programs. Writing a new OS is enough work, nobody would have time to also write file systems, compilers, a shell, a text editor, an Internet browser and so on. After having a usable environment, one can write whatever new program is desired, which would use the new native OS interface, but it would not be possible to replace everything at the same time. Besides having a POSIX translation layer, which can be written using as a starting point one of the standard C libraries, where the system calls must be replaced with the translation layer, some method must be found for reusing device drivers made for other operating systems, e.g. either for Linux or for one of the *BSD systems. Nobody would have time to also write all the needed device drivers. So there must exist some translation layer also for device drivers, maybe by running them in a virtual machine. The same as for user applications, if there is special interest in a certain device driver, it should be rewritten for the new OS, but rewriting all the device drivers that could be needed would take years, so it is important to implement a way to reuse the existing device drivers.
- yjftsjthsd-h 5y ago
  > microkernels aren't necessarily better than hybrid, they're slower, harder to debug and process management can be complicated I was basically on board, but how are they harder to debug? I'd think being able to run components in userspace would make debugging way easier.
  - mjgerm 5y ago
    You are now debugging a distributed system.
    - jatone 5y ago
      fun fact: you already are in linux. being a monolith doesn't change the nature of the problem.
      - yjftsjthsd-h 5y ago
        Are you? I'm paddy pretty sure I can run a single Linux and point a single gdb at it[0] and debug it in a single memory space; I don't think you can do that with a microkernel. [0] possibly resorting to UML, but still
    - yjftsjthsd-h 5y ago
      Oh, good point; I was thinking at the component level
VWWHFSfQ 5y ago
And why in the heck didnt he write it in Rust! ...
- fouc 5y ago
  At least he didn't write it in node.js /s Other interesting alternatives to c or rust might be vlang or zig.
  - xvilka 5y ago
    For V language there is Vinix[1], for Zig there's a kernel[2] only. [1] http://vinix-os.org/ http://vinix-os.org/ [2] https://github.com/jzck/kernel-zig https://github.com/jzck/kernel-zig
  - tentacleuno 5y ago
    Well, that's already been done. https://node-os.com https://node-os.com
turbinerneiter 5y ago
You can do all of this in your own hobby project.
ape4 5y ago
You need POSIX to compile nearly any program
- rurban 5y ago
  That his advantage. He doesn't want to, rather write everything anew from scratch. Also signals. With a microkernel you won't need signals
- mdp2021 5y ago
  So, a preliminary problem for many OS-from-scratch projects intending to create "New Different Great OS" will be to have a language implementation which does not expect posix. It depends on the goal. If you want to compile pre-existing software for it, or if you want to really "start anew".
- spijdar 5y ago
  True, and as another commenter hinted at, you're free to do your own hobby project however you please, there's not a wrong way to do so if you find it fulfilling. But at the same time, it does make me sad that most hobby OSes end up seeking POSIX compatibility, because that means being destined to essentially either be another unix-variant or develop a unix-variant inside some subsystem of your OS. Yes, being unix-like means you gain access to a trove of software and libraries that makes porting applications much easier, but it also limits the potential be truly different and experiment, as your end result will look like "yet another unix" with misc. improvements. Since I think the enjoyment of building something like this comes from the satisfaction of building an OS from the ground up, I don't think it matters, but it would be cool to see more hobby OSes try more exotic ideas and runtimes.
  - spicybright 5y ago
    Fully agree. We need more Temple OS's!
- ranger_danger 5y ago
  For *nix maybe, but there are many other non-POSIX operating systems in use today.
the-dude 5y ago
Shallow dismissal of other people's work.
- teddyh 5y ago
  Shallow dismissal of other people’s comments.
  - the-dude 5y ago
    Fine, I will provide the link again : https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
klange 5y ago
When I started the project a decade ago, my aim was specifically to understand POSIX/Unix and to "learn by doing", so of course there's some POSIX-like elements underpinning the design. Back before I decided that literally anything can be in scope and was using third-party components, this aided in porting software, as other commenters have pointed out. These days, it gives more purpose to things I build for the OS if they can also reasonably be built for Linux or macOS - things like my editor (which I wrote for the OS and now use as my daily driver in Linux), or my Python knock-off. As for the microkernel bit, this might sound like circular reasoning but I didn't go for a microkernel because no one really uses microkernels. It's not that I think microkernels are a bad idea, ToaruOS does push plenty of stuff into userspace. Rather, my main goal at the moment is to provide an educational resource that more accurately models the way "real" OSes work than the typical academic OS projects.

by rurban 5y ago

craigmart 5y ago

>why in the heck is he going for POSIX compatibility Because existing desktop applications can be ported to ToaruOS >why not a safer microkernel, keeping everything in userspace? This is a design choice, microkernels aren't necessarily better than hybrid, they're slower, harder to debug and process management can be complicated

fouc 5y ago
Isn't QNX a microkernel? I remember it being known for being quite fast?
- guenthert 5y ago
  As a real-time OS it is known for deterministic response times. If it were exceptionally fast (and licenses cheap enough), you'd see hosts in the TOP500 using it.
  - eggy 5y ago
    I agree 100%. QNX is lurking in products you may use. It was the OS for the show control system that is used throughout entertainment where real-time is necessary for the safety of the devices it controls, which also have hardware safety at the lower level. I would drop into the QNX terminal for certain tasks. Unfortunately, you used to be able to download the show control software and play with it, but it has since been bought buy a company that sells it with the equipment they rent, so you need to buy trainging and it is behind their wall now. Not QNX, but the show control software that runs on QNX.
  - p_l 5y ago
    TOP500 is chock full of microkernels though, even if the "I/O nodes" would run Linux
    - guenthert 5y ago
      I'd like to see more information about that. I remember that Penguin Computing offered such some 15 years ago, but don't know where it was deployed or still is. Cray and IBM had also such a concept for their superclusers in the past, but are they still using such? The one HPC environment I worked on (a major car manufacturer in Europe) used plain RH Linux on all nodes as recently as three years ago. The current #1 (Fugaku) uses IHK/McKernel as kernel for the actual payload. The previous #1 (IBM Summit) seems to use RH Linux though. Perhaps, since the most performance critical part is run by and within the GPGPU(s), the actual OS doesn't matter all that much (for performance -- it matters of course for programmer's comfort/efficiency).
      - p_l 5y ago
        There used to be a lot of "special microkernel on compute RPCing to Linux on I/O" on Crays and the like. Hard to say how prevalent it is now, and most annoyingly I can't recall the names. (Charon?)
- bregma 5y ago
  No, it's more of a nanokernel. It's very fast. Full disclosure: I maintain QNX toolchain.
md8z 5y ago
I'm very confused by this comment. There are a ton of other things you need to implement if you want to have desktop applications. POSIX does not specify any APIs for graphical applications. You might be thinking of something else. If you want to support the lion's share of desktop applications, it would actually be better to implement the Win32 API...
- craigmart 5y ago
  Sorry, I meant software in general but wrote "desktop applications" instead. Anyway the sentence is still valid, even if you'll have to implement other things such as the graphical interface, the POSIX compliant code won't need modification
  - md8z 5y ago
    If you're taking an app built for Linux or GNU or BSD, then it probably will need modification, as those systems have various extensions on top of POSIX.
eggy 5y ago
Just curious how hard it would be to forego POSIX entirely if you were building an OS. I know TempleOS is entirely from scratch. I'd like to implement a small LISP like SectorLISP [1] (see yesterday's posts too on HN). I don't know much about building my own OS, so I'd like to start with something like MenuetOS (my first PL was asm), SerenityOS, TempleOS, or this one. I'd like it to be completely an 'island', i.e. POSIX not a requirement. I want to use it to hack on in isolation without any easy copy/paste shortcuts. I know Mezzano exists, and it has booted on bare metal, but I would like to start with the OS's above, implement my own LISP, and go from there. Any other OS recommendations base on my ignorant, but wishful, reqs above? I realize there are some others in Rust too. Thanks! [1] https://github.com/jart/sectorlisp https://github.com/jart/sectorlisp
- adrian_b 5y ago
  Someone who would make a new OS, should define a completely new system call interface, as it is likely that now it is possible to conceive a better interface than 50 years ago and anyway if it would not be different there would be no reason to make a new OS, instead of modifying an existing one. Nevertheless, the first thing after defining a new OS interface must be writing a POSIX API translation layer, to be able to use without modifications the huge number of already existing programs. Writing a new OS is enough work, nobody would have time to also write file systems, compilers, a shell, a text editor, an Internet browser and so on. After having a usable environment, one can write whatever new program is desired, which would use the new native OS interface, but it would not be possible to replace everything at the same time. Besides having a POSIX translation layer, which can be written using as a starting point one of the standard C libraries, where the system calls must be replaced with the translation layer, some method must be found for reusing device drivers made for other operating systems, e.g. either for Linux or for one of the *BSD systems. Nobody would have time to also write all the needed device drivers. So there must exist some translation layer also for device drivers, maybe by running them in a virtual machine. The same as for user applications, if there is special interest in a certain device driver, it should be rewritten for the new OS, but rewriting all the device drivers that could be needed would take years, so it is important to implement a way to reuse the existing device drivers.
yjftsjthsd-h 5y ago
> microkernels aren't necessarily better than hybrid, they're slower, harder to debug and process management can be complicated I was basically on board, but how are they harder to debug? I'd think being able to run components in userspace would make debugging way easier.
- mjgerm 5y ago
  You are now debugging a distributed system.
  - jatone 5y ago
    fun fact: you already are in linux. being a monolith doesn't change the nature of the problem.
    - yjftsjthsd-h 5y ago
      Are you? I'm paddy pretty sure I can run a single Linux and point a single gdb at it[0] and debug it in a single memory space; I don't think you can do that with a microkernel. [0] possibly resorting to UML, but still
  - yjftsjthsd-h 5y ago
    Oh, good point; I was thinking at the component level

VWWHFSfQ 5y ago

And why in the heck didnt he write it in Rust! ...

fouc 5y ago
At least he didn't write it in node.js /s Other interesting alternatives to c or rust might be vlang or zig.
- xvilka 5y ago
  For V language there is Vinix[1], for Zig there's a kernel[2] only. [1] http://vinix-os.org/ http://vinix-os.org/ [2] https://github.com/jzck/kernel-zig https://github.com/jzck/kernel-zig
- tentacleuno 5y ago
  Well, that's already been done. https://node-os.com https://node-os.com

turbinerneiter 5y ago

You can do all of this in your own hobby project.

ape4 5y ago

You need POSIX to compile nearly any program

rurban 5y ago
That his advantage. He doesn't want to, rather write everything anew from scratch. Also signals. With a microkernel you won't need signals
mdp2021 5y ago
So, a preliminary problem for many OS-from-scratch projects intending to create "New Different Great OS" will be to have a language implementation which does not expect posix. It depends on the goal. If you want to compile pre-existing software for it, or if you want to really "start anew".
spijdar 5y ago
True, and as another commenter hinted at, you're free to do your own hobby project however you please, there's not a wrong way to do so if you find it fulfilling. But at the same time, it does make me sad that most hobby OSes end up seeking POSIX compatibility, because that means being destined to essentially either be another unix-variant or develop a unix-variant inside some subsystem of your OS. Yes, being unix-like means you gain access to a trove of software and libraries that makes porting applications much easier, but it also limits the potential be truly different and experiment, as your end result will look like "yet another unix" with misc. improvements. Since I think the enjoyment of building something like this comes from the satisfaction of building an OS from the ground up, I don't think it matters, but it would be cool to see more hobby OSes try more exotic ideas and runtimes.
- spicybright 5y ago
  Fully agree. We need more Temple OS's!
ranger_danger 5y ago
For *nix maybe, but there are many other non-POSIX operating systems in use today.

the-dude 5y ago

Shallow dismissal of other people's work.

teddyh 5y ago
Shallow dismissal of other people’s comments.
- the-dude 5y ago
  Fine, I will provide the link again : https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

klange 5y ago

When I started the project a decade ago, my aim was specifically to understand POSIX/Unix and to "learn by doing", so of course there's some POSIX-like elements underpinning the design. Back before I decided that literally anything can be in scope and was using third-party components, this aided in porting software, as other commenters have pointed out. These days, it gives more purpose to things I build for the OS if they can also reasonably be built for Linux or macOS - things like my editor (which I wrote for the OS and now use as my daily driver in Linux), or my Python knock-off. As for the microkernel bit, this might sound like circular reasoning but I didn't go for a microkernel because no one really uses microkernels. It's not that I think microkernels are a bad idea, ToaruOS does push plenty of stuff into userspace. Rather, my main goal at the moment is to provide an educational resource that more accurately models the way "real" OSes work than the typical academic OS projects.