13 ms·
Writing a BIOS bootloader for 64-bit mode from scratch
- AstralStorm 2y agoHow old is UEFI now? Pity nobody deprecated BIOS alongside long mode.
- deaddodo 2y agoBIOS is deprecated. All of its functionality on new motherboards is basically emulated via the UEFI; and it's certainly not being extended upon. Deprecated doesn't mean deleted, it just means "no longer updated/developed with a goal towards removal".
- livrem 2y agoThis killed FreeDOS (and presumably all the other *DOS as well) on modern hardware unfortunately. It was fun as long as it lasted. I do not know what the next-best single-user, single-process, non-bloated OS would be to run on modern hardware that still has some reasonably modern software and can be used for distraction-free (hobby) development the way FreeDOS could.
- trueismywork 2y agoLinux in single user mode
- mschuster91 2y agoThat's still multi-process though, there's an awful lot of background tasks running in pretty much every non-fossil kernel version, not to mention userspace daemons (udev, dbus, dhcp) without which most normal userspace stuff doesn't even work.
- Brian_K_White 2y agoNone of that exists in single user. When you say init=/bin/foo, then that's it, the only process is /bin/foo.
- ryandrake 2y ago/bin/foo is the initial process. It can fork and/or exec other processes, right?
- Brian_K_White 2y agoSure the facility to fork still exists. So what? Observing that the kernel still provides fork() is like observing that the cpu still provides JMP. It won't fork random processes you don't explicitly tell it to. I thought it was obvious that if you don't want unsolicited processes, then don't specify /bin/init as /bin/foo. The practical example is /bin/sh, but it could be any other executable. Up to you to specify a binary that does what you want, and doesn't require a bunch of other processes like gdbus to function itself. init=/bin/sh is more or less like ms-dos loading command.com
- gtirloni 2y agoIt's obvious but many people here seem to be confusing the Linux kernel with kernel+systemd and complaining Linux has many processes like it's not customizable.
- p_l 2y agoAs much as DOS allowed multiple processes, even if only one was executed at the time and there was no multitasking outside of ill-fated MS-DOS 4.0 and various Concurrent DOS products.
- ForOldHack 2y agoYou were not there. Multitasking? Xenix. Multitasking DOS? DOS Merge. Pre-emptive multitasking? AmigaOS. DOS 4.0M was task switching. OS/2 2.0? I'm being extremely facetious. Give me a date, and I will tell you what existed. Nothing mainstream existed until windows 3.1. at least on x86/32 There also was OLEC on windows 2, that did run in real mode, but the only thing that took advantage of it was the demos, and samples: no commercial product used it.
- jasaldivara 2y ago> I do not know what the next-best single-user, single-process, non-bloated OS would be to run on modern hardware that still has some reasonably modern software and can be used for distraction-free (hobby) development the way FreeDOS could. Not sure why would you want a single-process OS on modern hardware, but there are some alternatives that run much less things on the background than regular Linux: Haiku, FreeBSD, NetBSD, OpenBSD, or some lightweight non-glibc, non-systemd Linux-based like Adelie or Alpine.
- nineteen999 2y agoOr, you know, just booting the Linux kernel with init=/bin/sh with /bin/sh being a statically linked binary. You're overthinking things.
- rwmj 2y agoWhat's the reason why FreeDOS can't use the CSM (the BIOS compatibility mode of UEFI)?
- EvanAnderson 2y agoAFAIK it can. I believe some UEFI implementations don't have CSM.
- p_l 2y agoa Type 3 UEFI implementation has no CSM, Type 2 has CSM available, Type 1 enforces booting into CSM (what many "BIOS"es actually was in later days)
- EvanAnderson 2y agoThanks for that. That sent me down an enjoyable rabbit hole. I got started with PCs back in the 80s and became fairly familiar with how boot worked back then. UEFI happened while I was paying attention to other things and I've never become as familiar with it as I should be. This was a good excuse to do some reading.
- deaddodo 2y agoJust for clarification's sake, the proper terminology is "UEFI class" not "type". Otherwise, this is accurate.
- cl91 2y ago> the next-best single-user, single-process, non-bloated OS is UEFI.
- 5- 2y agonote that you can switch to long mode directly, without going into protected mode first, with way less code: https://wiki.osdev.org/Entering_Long_Mode_Directly https://wiki.osdev.org/Entering_Long_Mode_Directly i've had a bootloader for a small 64-bit kernel based on this that fit comfortably into the bootsector, including loading the kernel from disk and setting up vesa modes, no stage2 required.
- D4ckard 2y agoYes, you can do that too
- dataflow 2y ago> i've had a bootloader for a small 64-bit kernel based on this that fit comfortably into the bootsector, including loading the kernel from disk and setting up vesa modes, no stage2 required. How in the world do you fit all that in 512 bytes? I'm guessing you don't have a real-world filesystem (that allows the kernel to be anywhere on the disk just as a normal file)? Because just dealing with file fragmentation should bump you way over 512 bytes I would imagine.
- 5- 2y agoyes, the kernel was in a known location on disk (directly after the bootsector). the whole boot disk image was generated during build, which is common for small systems.
- rep_lodsb 2y agoI've been thinking about how to structure a filesystem such that code complexity - in the boot loader and elsewhere - can be minimized. You'd want something similar to NTFS, except that it should be possible to refer to a file (MFT entry) directly by starting sector number. So they would either need to always remain at a fixed position, or have a link pointing back to any reference so it can be updated. Somewhere in the first sector (aligned to a 64 bit boundary), there would be a small structure that just contains a unique signature (not ASCII but some random value; 64 bits seems like more than enough as opposed to a GUID), and a pointer to the "FS info block". Leaving all remaining space in the sector available for boot code or possibly another "overlayed" filesystem. That info block in turn would point to the MFT entry for the stage 2 boot code. An MFT entry would contain at a minimum an easy to locate list of (start sector,count) extents. Maybe there could be a flag that tells the OS that a certain file should never be fragmented? File names would be entirely optional and for human consumption; for direct links within a filesystem, sector numbers would be used, and some kind of unique numeric identifier for anything "higher-level". I'm genuinely wondering if some expert here sees any problem with this scheme, other than it not conforming to how mainstream OSes do things?
- blankx32 2y agohttps://wiki.osdev.org/A20_Line https://wiki.osdev.org/A20_Line
- ruslan 2y agoDoes this boot procedure work with EFI/UEFI ? If so, does UEFI supervisor emulate swithing real/protected/long modes or does it go in real hardware ?
- khaledh 2y agoNo. UEFI firmware creates a completely different environment for a UEFI bootloader than the legacy BIOS environment (real-address mode). The UEFI firmware enters 64-bit long mode directly on modern systems, and sets up a flat memory model GDT, as well as identity-mapped paging. I've written about creating a UEFI bootloader (for my hobby OS) here: https://0xc0ffee.netlify.app/osdev/05-bootloader-p1.html https://0xc0ffee.netlify.app/osdev/05-bootloader-p1.html
- surajrmal 2y agoI thought many UEFI implementations support legacy bios mode as well. Or well they used to.
- the_panopticon 2y agoThere is still support for CSM in the open source https://github.com/tianocore/tianocore.github.io/wiki/Compatibility-Support-Module https://github.com/tianocore/tianocore.github.io/wiki/Compat... and even nice projects like https://github.com/coreboot/seabiosto https://github.com/coreboot/seabiosto fabricate the CSM16 binary, but many vendors have stopped validating this path, including production of legacy BIOS option roms for adapters (net, gfx, etc) https://www.phoronix.com/news/Intel-Legacy-BIOS-EOL-2020 https://www.phoronix.com/news/Intel-Legacy-BIOS-EOL-2020. I still believe CSP's maintain some of this support in their hypervisors' guest firmware for legacy OS binaries/ISO boot support? Also since Windows requires UEFI Secure boot enabled by default and CSM has to be disabled for the UEFI secure boot path, this is another reason legacy BIOS boot isn't exercised so much these days. We could have added legacy oroms hashes to UEFI Secure boot implementations https://patents.google.com/patent/US8694761B2/en https://patents.google.com/patent/US8694761B2/en, too, but again folks pushed back in their zeal to remove legacy BIOS overall. We didn't add the CSM spec https://www.intel.com/content/dam/www/public/us/en/documents/reference-guides/efi-compatibility-support-module-specification-v098.pdf https://www.intel.com/content/dam/www/public/us/en/documents... to the PI https://uefi.org/specs/PI/1.8A/ https://uefi.org/specs/PI/1.8A/ since folks were hoping UEFI would remove the need for CSM. I still remember being challenged in the early 2000's by a long-time BIOS mgr at Intel "Is removing legacy a good idea with EFI? You know, we're really at legacy."
- amelius 2y agoIs this any simpler on ARM?
- rwmj 2y agoOnly in the sense that every board vendor does their own random thing, which makes it simpler for the board vendors and horribly complicated for everyone else.
- surajrmal 2y agoYes. Bootloaders are still complex, but there is less legacy setup that is required. That said, if you're targeting UEFI instead of BIOS, it's a great deal simpler on x86 as well.
- gtirloni 2y agoNot sure, I wouldn't count on it. Currently deep in RISC-V and it seems there's hope.
- zokier 2y agoI don't see how riscv can be anything but worse than Arm in this regard. With Arm at least Arm Holdings has some nominal power to steer towards sanity (devicetrees, systemready etc), with riscv it's again full freedom for vendors to make their own bespoke crap.
- rwmj 2y agoUnfortunately this is true in general, but for servers there is some hope: https://github.com/riscv-non-isa/riscv-server-platform https://github.com/riscv-non-isa/riscv-server-platform
- gtirloni 2y ago> With Arm at least Arm Holdings has some nominal power to steer towards sanity How's that working? RISC-V at least can have a formal spec that companies can choose to follow. The profile mentioned in another comment is one way. Companies can say they're compliant with this or that profile and software can target it.
- ThinkBeat 2y agoAll to me entirely unnecessary steps required to get the CPU into the correct mode is astounding. They all seem to be steps needed for backwards compatibility. Could Intel just provide a flag, command, to start in the right mode from the beginning. Or just remove all the backwards compatibility. I think I remember doing some research and ARM64 has some of the same issues. Are there any CPUs that are designed from scratch as 64 bit it will not have any need for backwards compatibility and would enter the required state by default? I guess sthat was the goal / design of Itanium? are made to start in the desired 64 bit state from th
- LiamPowell 2y agoUEFI exists. You just put a Windows-like binary in a folder on a partition and it runs in a hosted environment in 64-bit mode. And of course there's countless bootloaders that can take care of all this for you too.
- rep_lodsb 2y agoAnd then you're free from dealing with the somewhat convoluted processor init stuff, but instead depend on the Windows PE format, FAT filesystem, and an overcomplicated API. Seems like a bad tradeoff, and part of a slippery slope towards a completely locked down system, where writing your own code and getting it to run on the 'bare metal' is flat out impossible.
- immibis 2y agoWhat's wrong with depending on the Windows PE format, FAT filesystem and UEFI? You're always going to have some dependencies. FAT32 is better than having the first sector load some magic reserved sectors. Windows PE is better than a fixed memory address.
- rep_lodsb 2y agoIt's adding pointless complexity, and baking assumptions about how an OS should work into the firmware. Loading a sector at a fixed memory address and jumping to it (with some function provided so that your code can go on to load other sectors) is both easier to understand, and doesn't require you to use some multi-megabyte toolchain.
- hyperman1 2y agoThe 80286 has the Machine Status Word (MSW), a 16 bit register. The 80386 expands this to CR0, a 32 bits register. Then 64 bit long mode adds the EFER MSR and expands CR0 to 64 bits. But even today only 11 bits of CR0 are in use and EFER has 8 active bits. I wonder why intel/AMD did not simply use the free bits of the existing register, and made that decision twice? https://wiki.osdev.org/CPU_Registers_x86-64#CR0 https://wiki.osdev.org/CPU_Registers_x86-64#CR0.
- rcxdude 2y agoProbably for more robust backwards compatibility with software that might assume a given value for or write to the reserved bits. The assignment of bits to registers like this in the hardware is pretty arbitrary, there's not really any cost to using the higher bits
- monocasa 2y agoParticularly AMD made the 64 bit extension without any real input from Intel and didn't want to use any bits that would later conflict with a bit Intel might use in CR0. So a brand new register was in order.
- rep_lodsb 2y agoThe flag register layout is another case of extreme backwards compatibility - its lower bits have the same definitions they had on the 8-bit 8080, even the same fixed values: Sign : Zero : always '0' : AuxCarry : always '0' : Parity : always '1' : Carry (the parity flag came all the way from the 8008 / Datapoint 2200[1], and is the inverted XOR of the result's lower 8 bits; aux carry is the carry out of bit 3, used for BCD arithmetic) Flag bit 15 has also stayed reserved, except at one time it was used by the NEC Vxx chips for their 8080 compatibility mode. That feature had to be first unlocked by executing a special instruction, because there is code out there that loads the entire (16 bit) flag register with 0000 or FFFF. With the mode bit unlocked, that would inadvertently switch the CPU to running a completely different set of opcodes! [1] https://www.righto.com/2023/08/datapoint-to-8086.html https://www.righto.com/2023/08/datapoint-to-8086.html
- 2y ago
- rep_lodsb 2y agoThe most unnecessarily complicated thing in this article to me is the Makefile and linker script. NASM supports generating flat binary output, but apparently using it would be too "hacky"?
- darby_nine 2y agoI find linker scripts much easier to read and reason about than flat nasm but that's just me. Especially with multiple source files.
- sim7c00 2y agoFrom how I view linker scripts: U can use a linker script to create file layout. If you want a flat binary file... u dont want a file layout. So linkerscript is really useless for a flat binary blob. Even if you make it with multiple files. It'd be just the same as saying: cat blob1 blob2 blob2 > finalyblob. If you'd say have multiple blobs, and use linker directives to align them, the position dependent code within the assembled files will be wrong, unless you specifically define that using for example ORG directive in NASM. If you use the ORG directive in NASM, u will need to keep that synchronized with the linker script in order for all the labels etc. to keep the right offsets calculated. So essentially.. this linker script might even add complexity and issues when working with multiple binary blobs. u can't align them or use nice linker features.... If you use more structured files, which allow for example for relocation... then ur already using ELF or PE and can simple produce those files. They can be more masterfuly linked with nice features, and linker scripts are then essential. You can add your binary blob in the right location in the output file of a linking run, using a linker script. This is useful to add data into your files but for an MBR it's not particularly useful. People do it, but it adds no benefit over just sticking it on the front if your disk using 'dd' for example. ------ That being my views, I am wondering what you see the benefit here? Are there some linker features i am unaware of that are particularly useful here? (I really know only alignment stuff in there... and include blobs or put stuff into /discard/ and some basic define sections/segments etc.). I am not familiar with perhaps more advanced linking features.
- sim7c00 2y ago
- cf100clunk 2y agoA laudable project. UEFI proponents here wondering why the person bothered to create a new bootloader approach might be missing the point of why people undertake such tasks as this. As the writer ends: > Cool if you actually came along this far. Cool indeed.
- ForOldHack 2y agoThis seems both cool, and a good exercise, but is it useful? Does it have a UX like a fisher/price toy that you can verify/change your settings on the fly? Booting is the process of going from mini-me mode/single user/recovery mode to flying. I have been running Unix along side a Microsoft product since Xenix/dos. ( Looks like 40 years...) How much have we advanced? I also have been using Linux since the swedish version came out ( first release ) and GNU 0.1. My apologies about calling Xenix, Unix, It is a has-been wanna-be me-too square-excrament from shortly after release until it's languishing demise. Microsoft does not release products, they empty their cat boxes onto customers. ( The most recent example is both co-pilot And 22H2. ) If you look at how F1 cars have evolved, and pencils as well as pocket calculators - how close are we to the usable ideal? Why isn't the bootloader a static kernel mode? It used to be. Someone recently suggested it should be, and I agreed.