7 ms·

Building XNU for macOS 11.2 (Intel and Apple Silicon)

67 points by mrpippy 6y ago

loosetypes 6y ago
Xmxinx | Xmxinx means XNU is not Xinu | Xmxinx means Xinu is not XNU https://en.m.wikipedia.org/wiki/Xinu https://en.m.wikipedia.org/wiki/Xinu
remexre 6y ago
This is pretty exciting! I've been wanting to mess around with Darwin without having to own a Mac; FreeBSD features on a microkernel sounds potentially really nice. Trying to bootstrap up from the last available ISOs is terrible though; they're from over a decade ago, for i686...
- justaguy88 6y ago
  It's not really being used as a microkernel though
  - remexre 6y ago
    My understanding (as someone who doesn't run nor code for Darwin, yet at least) is that it's totally possible to use the microkernely features in userspace right up until you need to do a syscall, and even then you can have a task that translates port messages to traditional syscalls. I guess I care less about a microkernel's advantages for kernel development, than it having non-broken IPC :P
    - skissane 6y ago
      > it's totally possible to use the microkernely features in userspace right up until you need to do a syscall XNU has two sets of syscalls – Mach syscalls (negative syscall numbers) and BSD syscalls (positive syscall numbers). The Mach ones expose the classic Mach "microkernely" API, and the BSD ones provide the traditional POSIX API The term "syscall" is a bit ambiguous – sometimes it is said that the BSD side has syscalls but the Mach side has traps, other times it is said that BSD syscalls and Mach traps are the two different types of XNU syscalls. Definitely the BSD syscalls include a bunch invented by Apple themselves (not inherited from *BSD), and I think the same is the case for the Mach traps as well
- scruffyherder 6y ago
  It’s always been “possible “ but it’s always been a massive pain in the butt as there is a lot of inside track stuff, and the public source always has a habit of depending on itself. It was a massive chore to get Darwin 0.1 to build for x86 but it’s possible. And there is 2 things only available as a binary blob, the intel PCMCIA code was never released nor the EISA/PCI adaption layer. I have some qemu images here: https://sourceforge.net/projects/aapl-darwin/ https://sourceforge.net/projects/aapl-darwin/
- asveikau 6y ago
  In the early 2000s some people tried to build distros around it. Here it is running Gnome in 2004: https://commons.m.wikimedia.org/wiki/File:GNOME_2_running_on_openDarwin_(2004).png https://commons.m.wikimedia.org/wiki/File:GNOME_2_running_on... In those days there was a bit more optimism about such things but it was sort of like "why not just use Linux"? I think, iirc from reading tech news back then, there were also complaints about the quality and quantity of what Apple was releasing in source form.
ogre_codes 6y ago
Apparently there a decent chunk of the Apple Silicon files come in the form of binary files which seem to fall under the APSL. From the sounds of it, the binaries may be quite useful for reverse engineering and putting to use in the Linux port. From Marcan (who is porting Linux to the M1) > Wait no, hold on, WTF. They do supoprt M1. Just not in source code. They freaking bundled some .a files with uncensored binaries of the parts of xnu that need to support M1. More details here: https://twitter.com/marcan42/status/1355907973776723971 https://twitter.com/marcan42/status/1355907973776723971 While the binaries aren't quite as nice as source, its a good basis for reverse engineering some of the M1 specific code needed.
- marcan_42 6y ago
  The 11.1 source included the blobs as .a files, ostensibly APSL licensed. The 11.2 source moved those to the KDK (this is what the article is talking about), so they are no longer APSL, and further censored the source in the actual XNU open source release. It is quite evident at this stage that this is somehow related to ARM. They must not like Apple adding nonstandard extensions to the ISA, and they want them to hide them all. The good news is we don't have to care, and the (accidentally or not) "APSL" blob release gives everyone license to reverse engineer them, since it's not under an EULA.
  - ogre_codes 6y ago
    Love to know how/ why this got pushed out. If it's a rogue engineer, a mistake, or what exactly. Regardless, you can't put the cat back in the bag.
    - drvdevd 6y ago
      Sounds like Apple wants to promote or enable development on its new platform (even of 3rd party kernels), but can’t just distribute source due to some ARM licensing agreements or other legal/contractual restrictions. This is probably a middle ground through legal.
    - manwe150 6y ago

67 points by mrpippy 6y ago

Xmxinx | Xmxinx means XNU is not Xinu | Xmxinx means Xinu is not XNU https://en.m.wikipedia.org/wiki/Xinu https://en.m.wikipedia.org/wiki/Xinu

remexre 6y ago

This is pretty exciting! I've been wanting to mess around with Darwin without having to own a Mac; FreeBSD features on a microkernel sounds potentially really nice. Trying to bootstrap up from the last available ISOs is terrible though; they're from over a decade ago, for i686...

justaguy88 6y ago
It's not really being used as a microkernel though
- remexre 6y ago
  My understanding (as someone who doesn't run nor code for Darwin, yet at least) is that it's totally possible to use the microkernely features in userspace right up until you need to do a syscall, and even then you can have a task that translates port messages to traditional syscalls. I guess I care less about a microkernel's advantages for kernel development, than it having non-broken IPC :P
  - skissane 6y ago
    > it's totally possible to use the microkernely features in userspace right up until you need to do a syscall XNU has two sets of syscalls – Mach syscalls (negative syscall numbers) and BSD syscalls (positive syscall numbers). The Mach ones expose the classic Mach "microkernely" API, and the BSD ones provide the traditional POSIX API The term "syscall" is a bit ambiguous – sometimes it is said that the BSD side has syscalls but the Mach side has traps, other times it is said that BSD syscalls and Mach traps are the two different types of XNU syscalls. Definitely the BSD syscalls include a bunch invented by Apple themselves (not inherited from *BSD), and I think the same is the case for the Mach traps as well
scruffyherder 6y ago
It’s always been “possible “ but it’s always been a massive pain in the butt as there is a lot of inside track stuff, and the public source always has a habit of depending on itself. It was a massive chore to get Darwin 0.1 to build for x86 but it’s possible. And there is 2 things only available as a binary blob, the intel PCMCIA code was never released nor the EISA/PCI adaption layer. I have some qemu images here: https://sourceforge.net/projects/aapl-darwin/ https://sourceforge.net/projects/aapl-darwin/
asveikau 6y ago
In the early 2000s some people tried to build distros around it. Here it is running Gnome in 2004: https://commons.m.wikimedia.org/wiki/File:GNOME_2_running_on_openDarwin_(2004).png https://commons.m.wikimedia.org/wiki/File:GNOME_2_running_on... In those days there was a bit more optimism about such things but it was sort of like "why not just use Linux"? I think, iirc from reading tech news back then, there were also complaints about the quality and quantity of what Apple was releasing in source form.

ogre_codes 6y ago

Apparently there a decent chunk of the Apple Silicon files come in the form of binary files which seem to fall under the APSL. From the sounds of it, the binaries may be quite useful for reverse engineering and putting to use in the Linux port. From Marcan (who is porting Linux to the M1) > Wait no, hold on, WTF. They do supoprt M1. Just not in source code. They freaking bundled some .a files with uncensored binaries of the parts of xnu that need to support M1. More details here: https://twitter.com/marcan42/status/1355907973776723971 https://twitter.com/marcan42/status/1355907973776723971 While the binaries aren't quite as nice as source, its a good basis for reverse engineering some of the M1 specific code needed.

marcan_42 6y ago
The 11.1 source included the blobs as .a files, ostensibly APSL licensed. The 11.2 source moved those to the KDK (this is what the article is talking about), so they are no longer APSL, and further censored the source in the actual XNU open source release. It is quite evident at this stage that this is somehow related to ARM. They must not like Apple adding nonstandard extensions to the ISA, and they want them to hide them all. The good news is we don't have to care, and the (accidentally or not) "APSL" blob release gives everyone license to reverse engineer them, since it's not under an EULA.
- ogre_codes 6y ago
  Love to know how/ why this got pushed out. If it's a rogue engineer, a mistake, or what exactly. Regardless, you can't put the cat back in the bag.
  - drvdevd 6y ago
    Sounds like Apple wants to promote or enable development on its new platform (even of 3rd party kernels), but can’t just distribute source due to some ARM licensing agreements or other legal/contractual restrictions. This is probably a middle ground through legal.
  - manwe150 6y ago