10 ms·

A x64 OS #1: UEFI

224 points by based2 10y ago

TempleOSV409 10y ago
UEFI needs to support God's compiler Binaries. God's compiler does not generate elf. Multicore starts in real mode. You might as well learn how to do real-to-long anyway.
webkike 10y ago
I had no idea UEFI booted directly into long mode... I feel like a moron who hasn't kept up with the times, but admittedly I'm not super into kernel dev. Can I just ignore segmentation entirely? It has always been my least favorite part of programming on x86
- yuhong 10y ago
  You still need a GDT, but most of the values are ignored in long mode. There is still limited support in the form of changing the FS/GS segment bases, which is useful for the SYSCALL/SYSRET instructions.
- pgeorgi 10y ago
  It depends. Cheap Intel boards come with a 32bit UEFI (for whatever reason, it's not like UEFI is paid by the address bit), leading to all kinds of issues. But even then, segmentation is not an issue.
  - jdietrich 10y ago
    >(for whatever reason, it's not like UEFI is paid by the address bit) 32 bit Windows 10 is 4GB smaller than the 64 bit version. This matters on cheap tablets and netbooks that come with <32gb of MMC storage. I've heard that 32 bit OEM licenses are also slightly cheaper, but I can't find confirmation of that. https://blogs.intel.com/evangelists/2015/07/22/why-cheap-systems-run-32-bit-uefi-on-x64-systems/ https://blogs.intel.com/evangelists/2015/07/22/why-cheap-sys...
    - yuhong 10y ago
      In retrospect, it probably would have been better for x64 OSes to run the EFI runtime services in compatibility mode.
    - byuu 10y ago
      I know we're probably ten years or more out from it (because it'd break way too much software ... I have an application I want to use but can't due to a 16-bit installer ... in 2016); but I really hope Microsoft eventually puts out a 64-bit only operating system. It would be lovely to be rid of SYSWOW64, Program Files (x86), the alternate shadow registry, ~3GB of wasted space for all the 32-bit code, etc. I intentionally leave off the 32-bit compatibility with my FreeBSD systems, and it's lovely. Of course, it's so much easier there with everything being open source.
      - MarkSweep 10y ago
        Since at least Windows Server 2008 R2 in Server Core mode you can uninstall 32-bit compatibility: https://msdn.microsoft.com/en-us/library/windows/desktop/dd371790(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/dd3... Additionally Nano server does not have it. Not sure if it is yet possible to remove from the desktop yet, but I'm doubtful.
        Sanddancer 10y ago
        Not yet. There are still way too many programs that are 32 bit only.
      - prodigal_erik 10y ago
        I feel just the opposite. I have a pile of the most powerful general-purpose commodity hardware ever built, and it's a damning indictment of the industry that it can no longer run software that used to work fine on tiny machines.
        Sanddancer 10y ago
        I disagree. Things that made sense on a tiny machine, like the original 8086's wraparound at 1 MB led to hacks like the A20 gate [1]. Or timing loops that were dependent on a processor running at 4.77 mhz, requiring things like the turbo button. Compatibility is a balancing game, because you end up needing to keep bugs lying around that otherwise would be fixed at some point in a newer architecture. I'm more of the opinion that we're much better off explicitly emulating those areas, so they are run in their own context and it's a lot easier to mitigate damage. While compatibility is a positive thing, improvement is a much more positive thing. [1] https://en.wikipedia.org/wiki/A20_line#A20_gate https://en.wikipedia.org/wiki/A20_line#A20_gate
  - amluto 10y ago
    64-bit Linux kernels can be configured to so that they can be booted by 32-bit UEFI. It's messy but it works. Even runtime services like UEFI variables work. I think the idea is that long mode can be fused off. IIRC there's some MS requirement that boxes that have 64-bit CPUs must use 64-bit Windows, so keeping 32-bit firmware and adding a fuse works around it for users who care.
- byuu 10y ago
  > I had no idea UEFI booted directly into long mode I was really surprised to learn that it requires PE executables and Windows' custom x64 calling convention. I know it's just a small technical detail for booting ... but it's kind of scary that Microsoft had so much sway in the design of how our future PCs will boot =(
  - yuhong 10y ago
    MS even had to add ARM64 to the PE spec in order to support ARM SBSA. At least I think the FAT LFN patents has expired by now.
    - DiabloD3 10y ago
      Microsoft has granted a patent license for FAT patents for anyone implementing stuff required for UEFI.
    - slipstream- 10y ago
      There is lots of evidence that a port of NT to ARM64 exists.
    - cocotino 10y ago
      Do you even need LFN for UEFI? I don't think it should be a requirement for a FS that is not exposed to the user...
  - gizmo686 10y ago
    Another small technical detail. Windows has a mechanism to sign executables. The certificate in this mechanism is defined with the following structure [0]: typedef struct _WIN_CERTIFICATE { DWORD dwLength; WORD wRevision; WORD wCertificateType; BYTE bCertificate[ANYSIZE_ARRAY]; } WIN_CERTIFICATE, *PWIN_CERTIFICATE; UEFI has a shockingly simmilar structure [1, page 1812]: typedef struct _WIN_CERTIFICATE { UINT32 dwLength; UINT16 wRevision; UINT16 wCertificateType; //UINT8 bCertificate[ANYSIZE_ARRAY]; } WIN_CERTIFICATE; [0] https://msdn.microsoft.com/en-us/library/windows/desktop/dn582059(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/dn5... [1] http://www.uefi.org/sites/default/files/resources/UEFI%20Spec%202_6.pdf http://www.uefi.org/sites/default/files/resources/UEFI%20Spe...
gp7 10y ago
huehuehuehuehue is a wonderful name for a kernel
- somethingsimple 10y ago
  I was surprised the author is not Brazilian.
  - Sylos 10y ago
    Huehuehue has kind of become a meme around the internet...
- gayprogrammer 10y ago
  Can you please explain why the author uses that name?
  - Sylos 10y ago
    "Huehuehue" is the Brazillian equivalent of "hahaha". But since "huehuehue" sounds like a really crazy laughter to English speakers, when English internet culture clashes with Brazillian internet culture, this "huehuehue" really stands out to English speakers and seems hilariously overused. As a result, English speakers have started jokingly immitating it and it turned into a meme of sorts.
digi_owl 10y ago
Huh, just the other day i was pondering delivering the OS on a card/chip with a bit of flash space for storing patches and drivers. Right now it seems we get into a whole lot of problems because we have the OS sitting on the same RW space as user files and programs.
- sigcode 10y ago
  Would your idea include small form factor external media as the "card/chip"? What would be the negatives to this approach? I have been doing this for many years and have still not found any. Currently I prefer RO space embedded in the kernel for the "core" OS and mfs/tmpfs overlay for a RW, larger "additional" OS. If RW portion crashes, the system falls back to the RO portion.
  - digi_owl 10y ago
    Well my initial thought was a PCIe card, but it could likely just as well be an SD or such if set up right. And yeah, you can get an approximation of it using _nix and clever mounts. But i was thinking about having the initial OS sit on an actual ROM chip with a flash chip next to it on the same board/die.
qwertyuiop924 10y ago
You still can't just ignore BIOS. It might be on the deathrow, but it's still got plenty of life in it. Hell, knowing Intel, x86 BIOS will be supported until x86 itself keels over. Is it okay that I fantasize about that?
- userbinator 10y ago
  Linus Torvalds agrees with me and you, (U)EFI is an overengineered mess that is complex enough to be an OS itself: http://yarchive.net/comp/linux/efi.html http://yarchive.net/comp/linux/efi.html
  - qwertyuiop924 10y ago
    Huh. Well, if stories are to be believed, it's STILL better than BIOS.
    - ori_b 10y ago
      Bios is wonderful because it's limited; Nobody wants to try shoving more functionality into it. No system management, no drivers, nothing. The bios loads your kernel, and then you get out and do your own thing. And that's the advantage of it.
      - cfallin 10y ago
        > The bios loads your kernel Not even that! It loads the first 512-byte sector of your boot disk and jumps to it in 16-bit mode. Usually that bit of code loads in a few more sectors (using BIOS disk IO calls), enough to load a real bootloader's image (like GRUB), which subsequently understands your filesystem and loads the real kernel. It's kind of a hack but I completely agree, much better to leave it in the hands of the systems developer -- firmware that tries to do too much often just gets in the way.
      - jacquesm 10y ago
        It may also initialize some of your hardware by calling the firmware supplied with that hardware in ROM (for instance: video cards, network adapters and RAID controllers).
dom96 10y ago
That's awesome. I hope that I will get to do something similar as part of my university degree.
shmerl 10y ago
> Despite my strong attachment to the Rust community and Rust’s perfect suitability for kernels1, I’ll be writing the kernel in C. Mostly because it is unlikely people inside the university will be familiar with Rust, but also because GNU-EFI is a C library and I cannot be bothered to bind it. I’d surely be writing it in Rust were I more serious about the project. I saw an example of booting from UEFI in Rust: http://blog.theincredibleholk.org/blog/2013/11/18/booting-to-rust/ http://blog.theincredibleholk.org/blog/2013/11/18/booting-to...
- hornetblack 10y ago
  This is quite an old version of rust[1]. I tried this a little while ago (but got frustrated trying to link it) [1] For example: * `u16` would now be: `const u16` * `int` is no longer a type. * `slice::as_ptr` is now a method, so using transmute to extract the pointer from a slice is probably not ideal. * `core` is a crate that includes things like transmute so you don't need to manually include rust intrinsics etc.
  - shmerl 10y ago
    Yes, it's obviously somewhat outdated, since it's from 2013, which was before the first release.
e12e 10y ago
In case others find it interesting, I tried to look around for any relationship between linuxbios/seabios and uefi - and found some references tianocore and edk/edk II: https://github.com/tianocore/tianocore.github.io/wiki/EDK_II_Overview https://github.com/tianocore/tianocore.github.io/wiki/EDK_II... Looks like seabios is (mostly) happy being a bios (with possible plans to be packaged as an "csm application" (I'm guessing that means being loaded from efi)). But, s far as I can tell, for a free/open uefi toolkit, tianocore is the project to look at. I'm a little disappointed that I couldn't easily find a ready-to-run uefi space invader or tetris clone... ;) [ed: eh, I see tfa links to tianocore's ovmf code for vms... Oh, well... ]
AstralStorm 10y ago
Sadly, UEFI implementations are as buggy as anything else. Missing start of Option ROMs? Invalid GPU initialization? I have hit this and more with both Phoenix and AMI. And also security issues thanks to magic buggy Intel blobs. At least there is a valid, but hard to compile and deploy reference implementation. It is funny that CSM mode is so much more polished and tested than pure UEFI one. And that so many cards still do not provide compatible firmware.

224 points by based2 10y ago

UEFI needs to support God's compiler Binaries. God's compiler does not generate elf. Multicore starts in real mode. You might as well learn how to do real-to-long anyway.

webkike 10y ago

I had no idea UEFI booted directly into long mode... I feel like a moron who hasn't kept up with the times, but admittedly I'm not super into kernel dev. Can I just ignore segmentation entirely? It has always been my least favorite part of programming on x86

yuhong 10y ago
You still need a GDT, but most of the values are ignored in long mode. There is still limited support in the form of changing the FS/GS segment bases, which is useful for the SYSCALL/SYSRET instructions.
pgeorgi 10y ago
It depends. Cheap Intel boards come with a 32bit UEFI (for whatever reason, it's not like UEFI is paid by the address bit), leading to all kinds of issues. But even then, segmentation is not an issue.
- jdietrich 10y ago
  >(for whatever reason, it's not like UEFI is paid by the address bit) 32 bit Windows 10 is 4GB smaller than the 64 bit version. This matters on cheap tablets and netbooks that come with <32gb of MMC storage. I've heard that 32 bit OEM licenses are also slightly cheaper, but I can't find confirmation of that. https://blogs.intel.com/evangelists/2015/07/22/why-cheap-systems-run-32-bit-uefi-on-x64-systems/ https://blogs.intel.com/evangelists/2015/07/22/why-cheap-sys...
  - yuhong 10y ago
    In retrospect, it probably would have been better for x64 OSes to run the EFI runtime services in compatibility mode.
  - byuu 10y ago
    I know we're probably ten years or more out from it (because it'd break way too much software ... I have an application I want to use but can't due to a 16-bit installer ... in 2016); but I really hope Microsoft eventually puts out a 64-bit only operating system. It would be lovely to be rid of SYSWOW64, Program Files (x86), the alternate shadow registry, ~3GB of wasted space for all the 32-bit code, etc. I intentionally leave off the 32-bit compatibility with my FreeBSD systems, and it's lovely. Of course, it's so much easier there with everything being open source.
    - MarkSweep 10y ago
      Since at least Windows Server 2008 R2 in Server Core mode you can uninstall 32-bit compatibility: https://msdn.microsoft.com/en-us/library/windows/desktop/dd371790(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/dd3... Additionally Nano server does not have it. Not sure if it is yet possible to remove from the desktop yet, but I'm doubtful.
      - Sanddancer 10y ago
        Not yet. There are still way too many programs that are 32 bit only.
    - prodigal_erik 10y ago
      I feel just the opposite. I have a pile of the most powerful general-purpose commodity hardware ever built, and it's a damning indictment of the industry that it can no longer run software that used to work fine on tiny machines.
      - Sanddancer 10y ago
        I disagree. Things that made sense on a tiny machine, like the original 8086's wraparound at 1 MB led to hacks like the A20 gate [1]. Or timing loops that were dependent on a processor running at 4.77 mhz, requiring things like the turbo button. Compatibility is a balancing game, because you end up needing to keep bugs lying around that otherwise would be fixed at some point in a newer architecture. I'm more of the opinion that we're much better off explicitly emulating those areas, so they are run in their own context and it's a lot easier to mitigate damage. While compatibility is a positive thing, improvement is a much more positive thing. [1] https://en.wikipedia.org/wiki/A20_line#A20_gate https://en.wikipedia.org/wiki/A20_line#A20_gate
- amluto 10y ago
  64-bit Linux kernels can be configured to so that they can be booted by 32-bit UEFI. It's messy but it works. Even runtime services like UEFI variables work. I think the idea is that long mode can be fused off. IIRC there's some MS requirement that boxes that have 64-bit CPUs must use 64-bit Windows, so keeping 32-bit firmware and adding a fuse works around it for users who care.
byuu 10y ago
> I had no idea UEFI booted directly into long mode I was really surprised to learn that it requires PE executables and Windows' custom x64 calling convention. I know it's just a small technical detail for booting ... but it's kind of scary that Microsoft had so much sway in the design of how our future PCs will boot =(
- yuhong 10y ago
  MS even had to add ARM64 to the PE spec in order to support ARM SBSA. At least I think the FAT LFN patents has expired by now.
  - DiabloD3 10y ago
    Microsoft has granted a patent license for FAT patents for anyone implementing stuff required for UEFI.
  - slipstream- 10y ago
    There is lots of evidence that a port of NT to ARM64 exists.
  - cocotino 10y ago
    Do you even need LFN for UEFI? I don't think it should be a requirement for a FS that is not exposed to the user...
- gizmo686 10y ago
  Another small technical detail. Windows has a mechanism to sign executables. The certificate in this mechanism is defined with the following structure [0]: typedef struct _WIN_CERTIFICATE { DWORD dwLength; WORD wRevision; WORD wCertificateType; BYTE bCertificate[ANYSIZE_ARRAY]; } WIN_CERTIFICATE, *PWIN_CERTIFICATE; UEFI has a shockingly simmilar structure [1, page 1812]: typedef struct _WIN_CERTIFICATE { UINT32 dwLength; UINT16 wRevision; UINT16 wCertificateType; //UINT8 bCertificate[ANYSIZE_ARRAY]; } WIN_CERTIFICATE; [0] https://msdn.microsoft.com/en-us/library/windows/desktop/dn582059(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/dn5... [1] http://www.uefi.org/sites/default/files/resources/UEFI%20Spec%202_6.pdf http://www.uefi.org/sites/default/files/resources/UEFI%20Spe...

gp7 10y ago

huehuehuehuehue is a wonderful name for a kernel

somethingsimple 10y ago
I was surprised the author is not Brazilian.
- Sylos 10y ago
  Huehuehue has kind of become a meme around the internet...
gayprogrammer 10y ago
Can you please explain why the author uses that name?
- Sylos 10y ago
  "Huehuehue" is the Brazillian equivalent of "hahaha". But since "huehuehue" sounds like a really crazy laughter to English speakers, when English internet culture clashes with Brazillian internet culture, this "huehuehue" really stands out to English speakers and seems hilariously overused. As a result, English speakers have started jokingly immitating it and it turned into a meme of sorts.

digi_owl 10y ago

Huh, just the other day i was pondering delivering the OS on a card/chip with a bit of flash space for storing patches and drivers. Right now it seems we get into a whole lot of problems because we have the OS sitting on the same RW space as user files and programs.

sigcode 10y ago
Would your idea include small form factor external media as the "card/chip"? What would be the negatives to this approach? I have been doing this for many years and have still not found any. Currently I prefer RO space embedded in the kernel for the "core" OS and mfs/tmpfs overlay for a RW, larger "additional" OS. If RW portion crashes, the system falls back to the RO portion.
- digi_owl 10y ago
  Well my initial thought was a PCIe card, but it could likely just as well be an SD or such if set up right. And yeah, you can get an approximation of it using _nix and clever mounts. But i was thinking about having the initial OS sit on an actual ROM chip with a flash chip next to it on the same board/die.

qwertyuiop924 10y ago

You still can't just ignore BIOS. It might be on the deathrow, but it's still got plenty of life in it. Hell, knowing Intel, x86 BIOS will be supported until x86 itself keels over. Is it okay that I fantasize about that?

userbinator 10y ago
Linus Torvalds agrees with me and you, (U)EFI is an overengineered mess that is complex enough to be an OS itself: http://yarchive.net/comp/linux/efi.html http://yarchive.net/comp/linux/efi.html
- qwertyuiop924 10y ago
  Huh. Well, if stories are to be believed, it's STILL better than BIOS.
  - ori_b 10y ago
    Bios is wonderful because it's limited; Nobody wants to try shoving more functionality into it. No system management, no drivers, nothing. The bios loads your kernel, and then you get out and do your own thing. And that's the advantage of it.
    - cfallin 10y ago
      > The bios loads your kernel Not even that! It loads the first 512-byte sector of your boot disk and jumps to it in 16-bit mode. Usually that bit of code loads in a few more sectors (using BIOS disk IO calls), enough to load a real bootloader's image (like GRUB), which subsequently understands your filesystem and loads the real kernel. It's kind of a hack but I completely agree, much better to leave it in the hands of the systems developer -- firmware that tries to do too much often just gets in the way.
    - jacquesm 10y ago
      It may also initialize some of your hardware by calling the firmware supplied with that hardware in ROM (for instance: video cards, network adapters and RAID controllers).

dom96 10y ago

That's awesome. I hope that I will get to do something similar as part of my university degree.

shmerl 10y ago

> Despite my strong attachment to the Rust community and Rust’s perfect suitability for kernels1, I’ll be writing the kernel in C. Mostly because it is unlikely people inside the university will be familiar with Rust, but also because GNU-EFI is a C library and I cannot be bothered to bind it. I’d surely be writing it in Rust were I more serious about the project. I saw an example of booting from UEFI in Rust: http://blog.theincredibleholk.org/blog/2013/11/18/booting-to-rust/ http://blog.theincredibleholk.org/blog/2013/11/18/booting-to...

hornetblack 10y ago
This is quite an old version of rust[1]. I tried this a little while ago (but got frustrated trying to link it) [1] For example: * `u16` would now be: `const u16` * `int` is no longer a type. * `slice::as_ptr` is now a method, so using transmute to extract the pointer from a slice is probably not ideal. * `core` is a crate that includes things like transmute so you don't need to manually include rust intrinsics etc.
- shmerl 10y ago
  Yes, it's obviously somewhat outdated, since it's from 2013, which was before the first release.

e12e 10y ago

In case others find it interesting, I tried to look around for any relationship between linuxbios/seabios and uefi - and found some references tianocore and edk/edk II: https://github.com/tianocore/tianocore.github.io/wiki/EDK_II_Overview https://github.com/tianocore/tianocore.github.io/wiki/EDK_II... Looks like seabios is (mostly) happy being a bios (with possible plans to be packaged as an "csm application" (I'm guessing that means being loaded from efi)). But, s far as I can tell, for a free/open uefi toolkit, tianocore is the project to look at. I'm a little disappointed that I couldn't easily find a ready-to-run uefi space invader or tetris clone... ;) [ed: eh, I see tfa links to tianocore's ovmf code for vms... Oh, well... ]

AstralStorm 10y ago

Sadly, UEFI implementations are as buggy as anything else. Missing start of Option ROMs? Invalid GPU initialization? I have hit this and more with both Phoenix and AMI. And also security issues thanks to magic buggy Intel blobs. At least there is a valid, but hard to compile and deploy reference implementation. It is funny that CSM mode is so much more polished and tested than pure UEFI one. And that so many cards still do not provide compatible firmware.