5 ms·

Bare metal projects are always a fun hobby. I've done some projects like this in the past [0], let me share some things I learned the hard way. If the author i

by exDM69 9y ago

Bare metal projects are always a fun hobby. I've done some projects like this in the past [0], let me share some things I learned the hard way. If the author is reading, here are a few corrections. > char * vidptr = (char * )0xb8000; This needs "volatile" qualifier or all the writes to video memory will be dropped when optimization is enabled. Memory mapped I/O like this is one of the few use cases where you need "volatile" in C. > gcc -m32 -c kernel.c -o kc.o You need to build a cross-compiler for bare metal projects. Using the system gcc will not work in the long run. The compiler packaged with your operating system is intended for building binaries for use with your operating system. It may have downstream patches or configured to a target in a way that will cause problems. The worst part is that it may seem to work for a while, until it doesn't. I ran into some hard-to-debug issues with my past projects and learned this the hard way. If I recall correctly, the issues were related to redzones and ABI conventions. You need to build GNU binutils and GCC for the "i686-pc-elf" target. I documented this process for someone else's bare metal project here [1]. It really pays off to do this right from the start. Once you have a cross-compiler and a build system that can produce debuggable elf images (you also need to build gdb for the target), things get much easier. Using the built-in debuggers in QEMU or Bochs can only get you so far. Having a proper debugger with symbols and source view will make working much easier. For a build system, plain old Makefiles work best, CMake and other high level build systems are a pain in the ass with bare metal projects that require linker scripts, etc. [0] https://github.com/rikusalminen/danjeros https://github.com/rikusalminen/danjeros [1] https://github.com/Overv/MineAssemble/blob/master/README.md https://github.com/Overv/MineAssemble/blob/master/README.md

delta1 9y ago
Great post. Thanks for sharing!
pgeorgi 9y ago
> You need to build GNU binutils and GCC for the "i686-pc-elf" target. coreboot has similar issues, and so we maintain our cross compiler building script for 10 years now[0] (although historically using i386-elf instead of i686-*-elf). I sometimes wonder if there's value in packaging our compilers (all 8 architectures) for other users. [0] https://review.coreboot.org/cgit/coreboot.git/tree/util/crossgcc https://review.coreboot.org/cgit/coreboot.git/tree/util/cros...
- deleted 9y ago
  [deleted]
m00dy 9y ago
what is a redzone in your terminology ?
- exDM69 9y ago
  You know what, I'm not actually sure. All I know is that it screws up your stack frames. I think I had to provide -mno-redzone on GCC command line. It's something in the ABI (I was on x86_64, the ABI is rather complex) related to padding the stack frames when calling functions. I don't remember if this applied to all functions or was it something special needed for interrupt handlers.
  - gpderetta 9y ago
    the unix x86_64 userspace abi guarantees that the 128 bytes (known as the red zone) above the stack pointer are guaranteed not to be clobbered by signal handlers. This an optimization that avoid the need for leaf functions to manipulate the stack pointer to allocate their stack frame. The linux kernel does not follow this part of the ABI, so the compiler needs to be told that there is no red zone or functions will randomly see their stack frame clobbered by interrupt handlers.
    - DSMan195276 9y ago
      > The linux kernel does not follow this part of the ABI, so the compiler needs to be told that there is no red zone or functions will randomly see their stack frame clobbered by interrupt handlers. You're absolute right, but I would add a qualifier that it is impossible for a kernel to use the redzone optimization. Certain architectures (Like x86) will push data onto the stack when an interrupt happens and there is no way to tell them to put that data after the first 128 bytes on the stack.
      - convolvatron 9y ago
        you could use some register other than %esp for your stack :)
    - 9y ago
bogomipz 9y ago
>"You need to build a cross-compiler for bare metal projects." Might you have any useful links or documentation on this that you could share? Thanks.
- mw6621 9y ago
  I found this a while back, it works great for compiling a cross compiler for Linux and Windows. https://github.com/lordmilko/i686-elf-tools https://github.com/lordmilko/i686-elf-tools There's also a bit of information on OSDEV if you are wondering why you need a cross compiler: http://wiki.osdev.org/GCC_Cross-Compiler http://wiki.osdev.org/GCC_Cross-Compiler
  - carussell 9y ago
    FWIW, the need to bootstrap a cross-compiler from source is not inherent to cross-compilers; it's possible to implement a compiler in such a way that every version of the compiler is automatically a cross-compiler. I remember a few years back when I was trying to play with MINIX. Tanenbaum got several million EUR and hired some grad students to work on the thing. They promptly replaced much of the system with NetBSD. ("Perhaps too much", you can hear Tanenbaum say in one of his talks.) As a result of this the system compiler ACK was switched out for LLVM/clang. I complained on the mailing list about this because a full system build from source is something that used to be doable in <10 minutes—something Tanenbaum used to boast about—and it was now taking 3 hours if you decided to blow away your source/build directory and do a from-scratch build. The worst part is that the MINIX core, i.e., all the interesting parts, still only accounted for ~10 minutes of that build time, and virtually all the rest was spent compiling and then recompiling LLVM. The response I got from one of the aforementioned grad students was that this is "just how cross-compilers work". No, pal; that's how the compiler that you chose works. Later, the Go folks fixed their compiler to be a cross-compiler by default. See https://dave.cheney.net/2015/03/03/cross-compilation-just-got-a-whole-lot-better-in-go-1-5 https://dave.cheney.net/2015/03/03/cross-compilation-just-go... IMO, it's unforgivable that any given mainstream toolset wouldn't make this a baseline project goal.
    - 9y ago
justbaker 9y ago
Comments like this are why I still use HN. Thanks a bunch!

by exDM69 9y ago

delta1 9y ago

Great post. Thanks for sharing!

pgeorgi 9y ago

> You need to build GNU binutils and GCC for the "i686-pc-elf" target. coreboot has similar issues, and so we maintain our cross compiler building script for 10 years now[0] (although historically using i386-elf instead of i686-*-elf). I sometimes wonder if there's value in packaging our compilers (all 8 architectures) for other users. [0] https://review.coreboot.org/cgit/coreboot.git/tree/util/crossgcc https://review.coreboot.org/cgit/coreboot.git/tree/util/cros...

deleted 9y ago
[deleted]

m00dy 9y ago

what is a redzone in your terminology ?

exDM69 9y ago
You know what, I'm not actually sure. All I know is that it screws up your stack frames. I think I had to provide -mno-redzone on GCC command line. It's something in the ABI (I was on x86_64, the ABI is rather complex) related to padding the stack frames when calling functions. I don't remember if this applied to all functions or was it something special needed for interrupt handlers.
- gpderetta 9y ago
  the unix x86_64 userspace abi guarantees that the 128 bytes (known as the red zone) above the stack pointer are guaranteed not to be clobbered by signal handlers. This an optimization that avoid the need for leaf functions to manipulate the stack pointer to allocate their stack frame. The linux kernel does not follow this part of the ABI, so the compiler needs to be told that there is no red zone or functions will randomly see their stack frame clobbered by interrupt handlers.
  - DSMan195276 9y ago
    > The linux kernel does not follow this part of the ABI, so the compiler needs to be told that there is no red zone or functions will randomly see their stack frame clobbered by interrupt handlers. You're absolute right, but I would add a qualifier that it is impossible for a kernel to use the redzone optimization. Certain architectures (Like x86) will push data onto the stack when an interrupt happens and there is no way to tell them to put that data after the first 128 bytes on the stack.
    - convolvatron 9y ago
      you could use some register other than %esp for your stack :)
  - 9y ago

bogomipz 9y ago

>"You need to build a cross-compiler for bare metal projects." Might you have any useful links or documentation on this that you could share? Thanks.

mw6621 9y ago
I found this a while back, it works great for compiling a cross compiler for Linux and Windows. https://github.com/lordmilko/i686-elf-tools https://github.com/lordmilko/i686-elf-tools There's also a bit of information on OSDEV if you are wondering why you need a cross compiler: http://wiki.osdev.org/GCC_Cross-Compiler http://wiki.osdev.org/GCC_Cross-Compiler
- carussell 9y ago
  FWIW, the need to bootstrap a cross-compiler from source is not inherent to cross-compilers; it's possible to implement a compiler in such a way that every version of the compiler is automatically a cross-compiler. I remember a few years back when I was trying to play with MINIX. Tanenbaum got several million EUR and hired some grad students to work on the thing. They promptly replaced much of the system with NetBSD. ("Perhaps too much", you can hear Tanenbaum say in one of his talks.) As a result of this the system compiler ACK was switched out for LLVM/clang. I complained on the mailing list about this because a full system build from source is something that used to be doable in <10 minutes—something Tanenbaum used to boast about—and it was now taking 3 hours if you decided to blow away your source/build directory and do a from-scratch build. The worst part is that the MINIX core, i.e., all the interesting parts, still only accounted for ~10 minutes of that build time, and virtually all the rest was spent compiling and then recompiling LLVM. The response I got from one of the aforementioned grad students was that this is "just how cross-compilers work". No, pal; that's how the compiler that you chose works. Later, the Go folks fixed their compiler to be a cross-compiler by default. See https://dave.cheney.net/2015/03/03/cross-compilation-just-got-a-whole-lot-better-in-go-1-5 https://dave.cheney.net/2015/03/03/cross-compilation-just-go... IMO, it's unforgivable that any given mainstream toolset wouldn't make this a baseline project goal.
  - 9y ago

justbaker 9y ago

Comments like this are why I still use HN. Thanks a bunch!