9 ms·

Let’s write a simple Kernel (2014)

350 points by mmphosis 9y ago

TempleOS 9y ago
i proved i am divine king
TempleOS 9y ago
why dont you try to make a toilet. i dont think you are ready
exDM69 9y ago
Bare metal projects are always a fun hobby. I've done some projects like this in the past [0], let me share some things I learned the hard way. If the author is reading, here are a few corrections. > char * vidptr = (char * )0xb8000; This needs "volatile" qualifier or all the writes to video memory will be dropped when optimization is enabled. Memory mapped I/O like this is one of the few use cases where you need "volatile" in C. > gcc -m32 -c kernel.c -o kc.o You need to build a cross-compiler for bare metal projects. Using the system gcc will not work in the long run. The compiler packaged with your operating system is intended for building binaries for use with your operating system. It may have downstream patches or configured to a target in a way that will cause problems. The worst part is that it may seem to work for a while, until it doesn't. I ran into some hard-to-debug issues with my past projects and learned this the hard way. If I recall correctly, the issues were related to redzones and ABI conventions. You need to build GNU binutils and GCC for the "i686-pc-elf" target. I documented this process for someone else's bare metal project here [1]. It really pays off to do this right from the start. Once you have a cross-compiler and a build system that can produce debuggable elf images (you also need to build gdb for the target), things get much easier. Using the built-in debuggers in QEMU or Bochs can only get you so far. Having a proper debugger with symbols and source view will make working much easier. For a build system, plain old Makefiles work best, CMake and other high level build systems are a pain in the ass with bare metal projects that require linker scripts, etc. [0] https://github.com/rikusalminen/danjeros https://github.com/rikusalminen/danjeros [1] https://github.com/Overv/MineAssemble/blob/master/README.md https://github.com/Overv/MineAssemble/blob/master/README.md
- delta1 9y ago
  Great post. Thanks for sharing!
- pgeorgi 9y ago
  > You need to build GNU binutils and GCC for the "i686-pc-elf" target. coreboot has similar issues, and so we maintain our cross compiler building script for 10 years now[0] (although historically using i386-elf instead of i686-*-elf). I sometimes wonder if there's value in packaging our compilers (all 8 architectures) for other users. [0] https://review.coreboot.org/cgit/coreboot.git/tree/util/crossgcc https://review.coreboot.org/cgit/coreboot.git/tree/util/cros...
  - deleted 9y ago
    [deleted]
- m00dy 9y ago
  what is a redzone in your terminology ?
  - exDM69 9y ago
    You know what, I'm not actually sure. All I know is that it screws up your stack frames. I think I had to provide -mno-redzone on GCC command line. It's something in the ABI (I was on x86_64, the ABI is rather complex) related to padding the stack frames when calling functions. I don't remember if this applied to all functions or was it something special needed for interrupt handlers.
    - gpderetta 9y ago
      the unix x86_64 userspace abi guarantees that the 128 bytes (known as the red zone) above the stack pointer are guaranteed not to be clobbered by signal handlers. This an optimization that avoid the need for leaf functions to manipulate the stack pointer to allocate their stack frame. The linux kernel does not follow this part of the ABI, so the compiler needs to be told that there is no red zone or functions will randomly see their stack frame clobbered by interrupt handlers.
varunagrawal 9y ago
Am I the only one who's pretty much done with reading these sort of rinse and repeat articles about just writing a bootloader in the name of a kernel?
- Tepix 9y ago
  The article uses the grub bootloader so that part is missing. But I was expecting more when I read "kernel" as well.
- vidarh 9y ago
  Nobody is forcing you. I wish we had more of these, not least because of the first comment: "great post. Didn't know it was this easy ;)" People assume a lot of low level stuff like this is near magic not for mere mortals, and it's a great shame, because there's a lot of this kind of low level stuff that more people would benefit from playing with. And because demystifying it is a great path to get people into kernel hacking even if they don't end up writing their own complete kernel.
  - gmueckl 9y ago
    This stuff is not magic. But it requires a lot of reading and hunting for proper documentation on a platform as convoluted as x86. An ARM board would probably be way easier to play with. I never looked at the Pi. How well does it lend itself to this type of hacking?
    - vidarh 9y ago
      Specific ARM boards might very well be easier, in that a lot of them are clean slate designs, so you're not dragging along nearly 40 years of PC history. But ARM CPU's themselves is plagued by the fact that OEMs can put pretty much whatever they want in there, so there is a proliferation of really weird beasts.
- shakna 9y ago
  You could jump into a project like MikeOS, which has guides for modifying and extending [0], but the know-how from a bootloader should be enough to get you going. [0] http://mikeos.sourceforge.net/handbook-sysdev.html http://mikeos.sourceforge.net/handbook-sysdev.html
visarga 9y ago
Oh, that kind of kernel. There are many kernels - GPU kernels, SVM kernels, etc.
- mar77i 9y ago
  ...what Colonel?
magnat 9y ago
It's not much of a kernel. More like a self-contained bare-metal application. Main goal of a kernel is to provide services to applications. Although starting coding right away and getting tangible results is somewhat rewarding, it would be nice to see core concepts of designing a kernel mentioned in a "Kernel 101" article: * How to choose between monolithic/micro/nano architecture? * What are primary goals and use cases? * What hardware should it abstract away in HAL? * How to design userland-kernel and drivers-kernel API/ABI? * How much isolation is needed and what are ways to provide it?
d99kris 9y ago
My favourite kernel development tutorial is bkerndev [0]. It provides easy-to-read sources that can be used as a good base for simple projects. Shameless plug: I used bkerndev verbatim for my bare metal project - Nope OS [1] - a C64-like system that I built for my son when he was born, so that he could get to know computers the same way I did. :) [0] http://www.osdever.net/bkerndev/Docs/intro.htm http://www.osdever.net/bkerndev/Docs/intro.htm [1] https://github.com/d99kris/nopeos https://github.com/d99kris/nopeos
billpg 9y ago
Decades ago, I wrote a very simple operating system. It was an MSDOS EXE that when run, took over all available memory and rewrote all the interrupt vectors. You couldn't "exit" back to DOS because by then it had gone - you could only power-cycle the machine and let MSDOS boot up normally. Because it launched from DOS, many insisted to me it wasn't really an operating system. What was it if it wasn't an operating system?
- jacquesm 9y ago
  An operating system exports abstract facilities such as hardware, storage and other useful bits to other software we call applications. How you get there doesn't matter, what you do once you are there makes the difference. So I guess both you and your friends were wrong.
- gpderetta 9y ago
  You could think of MSDOS as a glorified bootloader...
  - laumars 9y ago
    To be fair, Windows used MS DOS as a bootloader for decades.
    - a-priori 9y ago
      Only the consumer line that ended at Windows ME. The Windows NT line, which subsumed the consumer line starting at Windows 2000, always used its own bootloader. So has Windows CE.
      - laumars 9y ago
        Well yes, clearly. That's why I used past tense rather than saying "still uses" and why I didn't mention NT nor CE (I also didn't mention Microsoft BASIC nor Xenix - just so you know they didn't use DOS as a bootloader either :P). I figured the context was obvious given the audience of HN, but clearly not. Anyhow, my point was just to reinforce the GP's point that it's ok to think about DOS as a bootloader because Microsoft have done exactly this themselves.
bogomipz 9y ago
I had a question about some of these passages, as they appear contradictory and or incorrect. >"Most registers of the x86 CPU have well defined values after power-on. The Instruction Pointer (EIP) register holds the memory address for the instruction being executed by the processor. EIP is hardcoded to the value 0xFFFFFFF0. Thus, the x86 CPU is hardwired to begin execution at the physical address 0xFFFFFFF0. It is in fact, the last 16 bytes of the 32-bit address space. This memory address is called reset vector. This says that the EIP is doing a JMP to an address in RAM not a memory-mapped IO address which points to ROM where the BIOS is stored. >"Now, the chipset’s memory map makes sure that 0xFFFFFFF0 is mapped to a certain part of the BIOS, not to the RAM. Meanwhile, the BIOS copies itself to the RAM for faster access. This is called shadowing. The address 0xFFFFFFF0 will contain just a jump instruction to the address in memory where BIOS has copied itself." The second says that the CPU is doing a JMP to an IO mapped-memory address which points to ROM. So the first passage says that CPU is just doing a JMP to 0xFFFFFFF0 in RAM. The second passage say that the CPU is doing a JMP to memory-mapped IO address which points to ROM. Are these not completely contradictory or am I reading this wrong? It's always been my understanding that that reset vector always pointed to a memory-mapped address which was located in ROM. Since the BIOS' POST routines contain code to initialiZe and test memory(the BIOs will actually emit beep codes if no memory is present or memory is faulty.), this would be a chicken and egg problem. Also the post mentions the chipset loads the BIOS into RAM as a process called "Shadowing" which was done because ROM used to be slow. But since BIOS ROM these days in generally NAND flash I don't believe this is the case any longer. Also these two statements appear to contradict each other: >"All x86 processors begin in a simplistic 16-bit mode called real mode. The GRUB bootloader makes the switch to 32-bit protected mode by setting the lowest bit of CR0 register to 1. Thus the kernel loads in 32-bit protected mode." >"Do note that in case of linux kernel, GRUB detects linux boot protocol and loads linux kernel in real mode. Linux kernel itself makes the switch to protected mode." The first states that the kernel loads in protected mode and then the following states that kernel loads in real mode.
- arkj 9y ago
  The reset vector must be a mapped ROM address otherwise how will the jmp instruction be loaded. But I also seen in some places where they say it can be configured to DRAM as well [0]. I am not sure how it will work then. [0] (page 5)- https://firmware.intel.com/sites/default/files/resources/A_Tour_Beyond_BIOS_Memory_Map_in%20UEFI_BIOS.pdf https://firmware.intel.com/sites/default/files/resources/A_T...
drngdds 9y ago
"This programming tutorial is great, but it could really use some giant meme images," said no one ever.

350 points by mmphosis 9y ago

TempleOS 9y ago

i proved i am divine king

TempleOS 9y ago

why dont you try to make a toilet. i dont think you are ready

exDM69 9y ago

Bare metal projects are always a fun hobby. I've done some projects like this in the past [0], let me share some things I learned the hard way. If the author is reading, here are a few corrections. > char * vidptr = (char * )0xb8000; This needs "volatile" qualifier or all the writes to video memory will be dropped when optimization is enabled. Memory mapped I/O like this is one of the few use cases where you need "volatile" in C. > gcc -m32 -c kernel.c -o kc.o You need to build a cross-compiler for bare metal projects. Using the system gcc will not work in the long run. The compiler packaged with your operating system is intended for building binaries for use with your operating system. It may have downstream patches or configured to a target in a way that will cause problems. The worst part is that it may seem to work for a while, until it doesn't. I ran into some hard-to-debug issues with my past projects and learned this the hard way. If I recall correctly, the issues were related to redzones and ABI conventions. You need to build GNU binutils and GCC for the "i686-pc-elf" target. I documented this process for someone else's bare metal project here [1]. It really pays off to do this right from the start. Once you have a cross-compiler and a build system that can produce debuggable elf images (you also need to build gdb for the target), things get much easier. Using the built-in debuggers in QEMU or Bochs can only get you so far. Having a proper debugger with symbols and source view will make working much easier. For a build system, plain old Makefiles work best, CMake and other high level build systems are a pain in the ass with bare metal projects that require linker scripts, etc. [0] https://github.com/rikusalminen/danjeros https://github.com/rikusalminen/danjeros [1] https://github.com/Overv/MineAssemble/blob/master/README.md https://github.com/Overv/MineAssemble/blob/master/README.md

delta1 9y ago
Great post. Thanks for sharing!
pgeorgi 9y ago
> You need to build GNU binutils and GCC for the "i686-pc-elf" target. coreboot has similar issues, and so we maintain our cross compiler building script for 10 years now[0] (although historically using i386-elf instead of i686-*-elf). I sometimes wonder if there's value in packaging our compilers (all 8 architectures) for other users. [0] https://review.coreboot.org/cgit/coreboot.git/tree/util/crossgcc https://review.coreboot.org/cgit/coreboot.git/tree/util/cros...
- deleted 9y ago
  [deleted]
m00dy 9y ago
what is a redzone in your terminology ?
- exDM69 9y ago
  You know what, I'm not actually sure. All I know is that it screws up your stack frames. I think I had to provide -mno-redzone on GCC command line. It's something in the ABI (I was on x86_64, the ABI is rather complex) related to padding the stack frames when calling functions. I don't remember if this applied to all functions or was it something special needed for interrupt handlers.
  - gpderetta 9y ago
    the unix x86_64 userspace abi guarantees that the 128 bytes (known as the red zone) above the stack pointer are guaranteed not to be clobbered by signal handlers. This an optimization that avoid the need for leaf functions to manipulate the stack pointer to allocate their stack frame. The linux kernel does not follow this part of the ABI, so the compiler needs to be told that there is no red zone or functions will randomly see their stack frame clobbered by interrupt handlers.

varunagrawal 9y ago

Am I the only one who's pretty much done with reading these sort of rinse and repeat articles about just writing a bootloader in the name of a kernel?

Tepix 9y ago
The article uses the grub bootloader so that part is missing. But I was expecting more when I read "kernel" as well.
vidarh 9y ago
Nobody is forcing you. I wish we had more of these, not least because of the first comment: "great post. Didn't know it was this easy ;)" People assume a lot of low level stuff like this is near magic not for mere mortals, and it's a great shame, because there's a lot of this kind of low level stuff that more people would benefit from playing with. And because demystifying it is a great path to get people into kernel hacking even if they don't end up writing their own complete kernel.
- gmueckl 9y ago
  This stuff is not magic. But it requires a lot of reading and hunting for proper documentation on a platform as convoluted as x86. An ARM board would probably be way easier to play with. I never looked at the Pi. How well does it lend itself to this type of hacking?
  - vidarh 9y ago
    Specific ARM boards might very well be easier, in that a lot of them are clean slate designs, so you're not dragging along nearly 40 years of PC history. But ARM CPU's themselves is plagued by the fact that OEMs can put pretty much whatever they want in there, so there is a proliferation of really weird beasts.
shakna 9y ago
You could jump into a project like MikeOS, which has guides for modifying and extending [0], but the know-how from a bootloader should be enough to get you going. [0] http://mikeos.sourceforge.net/handbook-sysdev.html http://mikeos.sourceforge.net/handbook-sysdev.html

visarga 9y ago

Oh, that kind of kernel. There are many kernels - GPU kernels, SVM kernels, etc.

mar77i 9y ago
...what Colonel?

magnat 9y ago

It's not much of a kernel. More like a self-contained bare-metal application. Main goal of a kernel is to provide services to applications. Although starting coding right away and getting tangible results is somewhat rewarding, it would be nice to see core concepts of designing a kernel mentioned in a "Kernel 101" article: * How to choose between monolithic/micro/nano architecture? * What are primary goals and use cases? * What hardware should it abstract away in HAL? * How to design userland-kernel and drivers-kernel API/ABI? * How much isolation is needed and what are ways to provide it?

d99kris 9y ago

My favourite kernel development tutorial is bkerndev [0]. It provides easy-to-read sources that can be used as a good base for simple projects. Shameless plug: I used bkerndev verbatim for my bare metal project - Nope OS [1] - a C64-like system that I built for my son when he was born, so that he could get to know computers the same way I did. :) [0] http://www.osdever.net/bkerndev/Docs/intro.htm http://www.osdever.net/bkerndev/Docs/intro.htm [1] https://github.com/d99kris/nopeos https://github.com/d99kris/nopeos

billpg 9y ago

Decades ago, I wrote a very simple operating system. It was an MSDOS EXE that when run, took over all available memory and rewrote all the interrupt vectors. You couldn't "exit" back to DOS because by then it had gone - you could only power-cycle the machine and let MSDOS boot up normally. Because it launched from DOS, many insisted to me it wasn't really an operating system. What was it if it wasn't an operating system?

jacquesm 9y ago
An operating system exports abstract facilities such as hardware, storage and other useful bits to other software we call applications. How you get there doesn't matter, what you do once you are there makes the difference. So I guess both you and your friends were wrong.
gpderetta 9y ago
You could think of MSDOS as a glorified bootloader...
- laumars 9y ago
  To be fair, Windows used MS DOS as a bootloader for decades.
  - a-priori 9y ago
    Only the consumer line that ended at Windows ME. The Windows NT line, which subsumed the consumer line starting at Windows 2000, always used its own bootloader. So has Windows CE.
    - laumars 9y ago
      Well yes, clearly. That's why I used past tense rather than saying "still uses" and why I didn't mention NT nor CE (I also didn't mention Microsoft BASIC nor Xenix - just so you know they didn't use DOS as a bootloader either :P). I figured the context was obvious given the audience of HN, but clearly not. Anyhow, my point was just to reinforce the GP's point that it's ok to think about DOS as a bootloader because Microsoft have done exactly this themselves.

bogomipz 9y ago

I had a question about some of these passages, as they appear contradictory and or incorrect. >"Most registers of the x86 CPU have well defined values after power-on. The Instruction Pointer (EIP) register holds the memory address for the instruction being executed by the processor. EIP is hardcoded to the value 0xFFFFFFF0. Thus, the x86 CPU is hardwired to begin execution at the physical address 0xFFFFFFF0. It is in fact, the last 16 bytes of the 32-bit address space. This memory address is called reset vector. This says that the EIP is doing a JMP to an address in RAM not a memory-mapped IO address which points to ROM where the BIOS is stored. >"Now, the chipset’s memory map makes sure that 0xFFFFFFF0 is mapped to a certain part of the BIOS, not to the RAM. Meanwhile, the BIOS copies itself to the RAM for faster access. This is called shadowing. The address 0xFFFFFFF0 will contain just a jump instruction to the address in memory where BIOS has copied itself." The second says that the CPU is doing a JMP to an IO mapped-memory address which points to ROM. So the first passage says that CPU is just doing a JMP to 0xFFFFFFF0 in RAM. The second passage say that the CPU is doing a JMP to memory-mapped IO address which points to ROM. Are these not completely contradictory or am I reading this wrong? It's always been my understanding that that reset vector always pointed to a memory-mapped address which was located in ROM. Since the BIOS' POST routines contain code to initialiZe and test memory(the BIOs will actually emit beep codes if no memory is present or memory is faulty.), this would be a chicken and egg problem. Also the post mentions the chipset loads the BIOS into RAM as a process called "Shadowing" which was done because ROM used to be slow. But since BIOS ROM these days in generally NAND flash I don't believe this is the case any longer. Also these two statements appear to contradict each other: >"All x86 processors begin in a simplistic 16-bit mode called real mode. The GRUB bootloader makes the switch to 32-bit protected mode by setting the lowest bit of CR0 register to 1. Thus the kernel loads in 32-bit protected mode." >"Do note that in case of linux kernel, GRUB detects linux boot protocol and loads linux kernel in real mode. Linux kernel itself makes the switch to protected mode." The first states that the kernel loads in protected mode and then the following states that kernel loads in real mode.

arkj 9y ago
The reset vector must be a mapped ROM address otherwise how will the jmp instruction be loaded. But I also seen in some places where they say it can be configured to DRAM as well [0]. I am not sure how it will work then. [0] (page 5)- https://firmware.intel.com/sites/default/files/resources/A_Tour_Beyond_BIOS_Memory_Map_in%20UEFI_BIOS.pdf https://firmware.intel.com/sites/default/files/resources/A_T...

drngdds 9y ago

"This programming tutorial is great, but it could really use some giant meme images," said no one ever.