4 ms·

I hate to sound dismissive but I feel like a lot of these projects put too much emphasis on booting and too little on something like say process semantics. I've

I hate to sound dismissive but I feel like a lot of these projects put too much emphasis on booting and too little on something like say process semantics. I've written bootloaders before and it was some of the most uninspired code I've ever written.

deleted 8y ago
[deleted]
jarfil 8y ago
I think it's because the boot sequence is always pretty much the same on a given architecture, but process management is where things start to depend less on the hardware and more on the OS design.
dschuetz 8y ago
Less is more, right? Also, there are two kinds of bootloaders: simple + reliable, and bloated + buggy.
- jrockway 8y ago
  grub is pretty bloated but also reliable. Finding the kernel is not as easy as it used to be. You need to read the partition table on the disk. You need to be able to understand the filesystem that the kernel is stored on. This adds a fair bit of complexity.
  - zaarn 8y ago
    On a modern UEFI system you actually get provided with an API that can do all that for you, though only FAT32 partitions are supported for the FS out of the box (you can load your own drivers for more). If your boot partition is FAT32, UEFI makes things much much simpler.
    - jrockway 8y ago
      True. And I think that's more than adequate for your homebrew OS. (Which can just have a second stage if you really need to boot off of an encrypted RAID-5 partition connected to a custom storage controller.)
zaarn 8y ago
Process management is hard. And that's if you throw out all the important stuff like "I want to run my programs outside the kernel" and "It sure would be nice if programs didn't have to share address space". I recommend the OSDev wiki for a deeper dive but ultimately, implementing processes involves a lot of different moving pieces that need to work. The most basic process manager would be cooperative, otherwise you'd have to write drivers for some external driver to regularly drive an interrupt. Cooperative means the program will either print A or B and then call an interrupt into the kernel. The kernel would save the registers and instruction pointer, restore the one of the other program (which prints the opposite of what the first printed; A or B) and do an interrupt return. This doesn't scale beyond a few processes that you have written yourself very carefully. Any more competent process management will need to rely on regular timer interrupts, will have to keep track of processes and how much CPU time they got, have a thread to run if nothing else can run, manage processes when they die and has to be able to switch in and out of ring3 for running the actual process. On top of that it has to be performant and efficient, you have only a few hundred opcodes before the latency of your process management becomes too large.
- desertrider12 8y ago
  If anyone wants to try this, on x86 getting a regular interrupt for preemptive multitasking is pretty easy, you can write an interrupt handler for the [real-time clock](https://wiki.osdev.org/RTC https://wiki.osdev.org/RTC). This will work on any PC and takes about 15 lines of code to set up. Every 1ms, whatever's running will be paused and your scheduling code can run. Actually writing the scheduler is the hard part ;)
  - 3chelon 8y ago
    I second that. My degree project was a 68K-based RTOS kernel with a simple preemptive task scheduler, and the scheduler was one of the easiest and smallest parts of the whole project.
    - zaarn 8y ago
      Realtime Schedulers have it a bit more easy IMO than non-realtime. For RT you can write a scheduler that at any point knows if all processes will meet their deadlines and you can easily design an algorithm to get to the most efficient scheduling order. In non-RT it gets harder because a process might not need to run at all and is just eternally paused on reading some dead socket, it might have higher priority than other processes, you might have 2000 processes to run but only 1000 timeslots left for the current timeslice. A process might begin to eat up CPU time and you have to somehow preserve the interactivity of the system, ie prevent other processes from starving without starving the big process in turn. And once you enter multi-CPU it gets even harder; coordinating multiple concurrent schedulers to run from the same task queue, which cannot ever take a simple spinlock without the system suddenly becoming dead. The parent comment simply noted that setting up the timer interrupt is fairly easy (though if you want something precise and faster than 1ms and multicore it might get more complicated), it's the stuff after that were you are forced to solve NP-hard problems in a fast way.
deleted 8y ago
[deleted]
dschuetz 8y ago
You know, once a machine is running, you can do everything else that's limited by your own imagination. I've been trying to figure out how a powerpc machine is being booted so that I can insert my own 'hello world' OS. Sure there is a BIOS equivalent somewhere in there, but how to figure out how and when to talk to it? Booting x86 might seem like nothing, but booting some old SPARC for instance seems like black magic to me.
- pjc50 8y ago
  I guess it all depends on the documentation. Sun's is excellent e.g.: http://www.bitsavers.org/pdf/sun/sun3/Sun-3_Customer_Maintenance_Training_Sep88.pdf http://www.bitsavers.org/pdf/sun/sun3/Sun-3_Customer_Mainten... - on the Sun 3/60 you have a rather nice boot ROM which does similar things to UEFI but without the drama. It will let you netboot it over tftp, for example. Which sort of PPC machine is it? Have you already managed to get e.g. NetBSD running on it?
  - dschuetz 8y ago
    Powermac g5 and/or power6. I got a CLFS kernel version 3.16 to boot and run (insanely fast) on the g5, but power6 is completely mysterious to me. There is some sort of RHEL linux based firmware running on it, but it seems there is no way to make it boot anything other than what's officially supported.

deleted 8y ago

[deleted]

jarfil 8y ago

I think it's because the boot sequence is always pretty much the same on a given architecture, but process management is where things start to depend less on the hardware and more on the OS design.

dschuetz 8y ago

Less is more, right? Also, there are two kinds of bootloaders: simple + reliable, and bloated + buggy.

jrockway 8y ago
grub is pretty bloated but also reliable. Finding the kernel is not as easy as it used to be. You need to read the partition table on the disk. You need to be able to understand the filesystem that the kernel is stored on. This adds a fair bit of complexity.
- zaarn 8y ago
  On a modern UEFI system you actually get provided with an API that can do all that for you, though only FAT32 partitions are supported for the FS out of the box (you can load your own drivers for more). If your boot partition is FAT32, UEFI makes things much much simpler.
  - jrockway 8y ago
    True. And I think that's more than adequate for your homebrew OS. (Which can just have a second stage if you really need to boot off of an encrypted RAID-5 partition connected to a custom storage controller.)

zaarn 8y ago

Process management is hard. And that's if you throw out all the important stuff like "I want to run my programs outside the kernel" and "It sure would be nice if programs didn't have to share address space". I recommend the OSDev wiki for a deeper dive but ultimately, implementing processes involves a lot of different moving pieces that need to work. The most basic process manager would be cooperative, otherwise you'd have to write drivers for some external driver to regularly drive an interrupt. Cooperative means the program will either print A or B and then call an interrupt into the kernel. The kernel would save the registers and instruction pointer, restore the one of the other program (which prints the opposite of what the first printed; A or B) and do an interrupt return. This doesn't scale beyond a few processes that you have written yourself very carefully. Any more competent process management will need to rely on regular timer interrupts, will have to keep track of processes and how much CPU time they got, have a thread to run if nothing else can run, manage processes when they die and has to be able to switch in and out of ring3 for running the actual process. On top of that it has to be performant and efficient, you have only a few hundred opcodes before the latency of your process management becomes too large.

desertrider12 8y ago
If anyone wants to try this, on x86 getting a regular interrupt for preemptive multitasking is pretty easy, you can write an interrupt handler for the [real-time clock](https://wiki.osdev.org/RTC https://wiki.osdev.org/RTC). This will work on any PC and takes about 15 lines of code to set up. Every 1ms, whatever's running will be paused and your scheduling code can run. Actually writing the scheduler is the hard part ;)
- 3chelon 8y ago
  I second that. My degree project was a 68K-based RTOS kernel with a simple preemptive task scheduler, and the scheduler was one of the easiest and smallest parts of the whole project.
  - zaarn 8y ago
    Realtime Schedulers have it a bit more easy IMO than non-realtime. For RT you can write a scheduler that at any point knows if all processes will meet their deadlines and you can easily design an algorithm to get to the most efficient scheduling order. In non-RT it gets harder because a process might not need to run at all and is just eternally paused on reading some dead socket, it might have higher priority than other processes, you might have 2000 processes to run but only 1000 timeslots left for the current timeslice. A process might begin to eat up CPU time and you have to somehow preserve the interactivity of the system, ie prevent other processes from starving without starving the big process in turn. And once you enter multi-CPU it gets even harder; coordinating multiple concurrent schedulers to run from the same task queue, which cannot ever take a simple spinlock without the system suddenly becoming dead. The parent comment simply noted that setting up the timer interrupt is fairly easy (though if you want something precise and faster than 1ms and multicore it might get more complicated), it's the stuff after that were you are forced to solve NP-hard problems in a fast way.

deleted 8y ago

[deleted]

dschuetz 8y ago

You know, once a machine is running, you can do everything else that's limited by your own imagination. I've been trying to figure out how a powerpc machine is being booted so that I can insert my own 'hello world' OS. Sure there is a BIOS equivalent somewhere in there, but how to figure out how and when to talk to it? Booting x86 might seem like nothing, but booting some old SPARC for instance seems like black magic to me.

pjc50 8y ago
I guess it all depends on the documentation. Sun's is excellent e.g.: http://www.bitsavers.org/pdf/sun/sun3/Sun-3_Customer_Maintenance_Training_Sep88.pdf http://www.bitsavers.org/pdf/sun/sun3/Sun-3_Customer_Mainten... - on the Sun 3/60 you have a rather nice boot ROM which does similar things to UEFI but without the drama. It will let you netboot it over tftp, for example. Which sort of PPC machine is it? Have you already managed to get e.g. NetBSD running on it?
- dschuetz 8y ago
  Powermac g5 and/or power6. I got a CLFS kernel version 3.16 to boot and run (insanely fast) on the g5, but power6 is completely mysterious to me. There is some sort of RHEL linux based firmware running on it, but it seems there is no way to make it boot anything other than what's officially supported.