10 ms·

Mach kernel

139 points by thatguyagain 7y ago

blank_pattern 7y ago
Apple even open sources their XNU kernel: https://opensource.apple.com/ https://opensource.apple.com/ There's some delay between when a new macOS version comes out and when sources get published, but it's great to see how they use the Mach kernel in practice.
- ksec 7y ago
  Why do they update the macOS XNU Kernel and not iOS?
  - deleted 7y ago
    [deleted]
  - shakna 7y ago
    IIRC they share the exact same kernel, so they don't need to.
    - saagarjha 7y ago
      They’re not quite the same, which is why the lack of iOS sources was a minor annoyance for a while.
  - blank_pattern 7y ago
    They seem really big on keeping iOS things secret-ish, but the XNU kernel for both iOS and macOS seem to be built from the same code base. Up until a couple years ago, they'd strip out ARM specific things from the releases macOS XNU code. Then they started leaving that stuff in!
    - vbezhenar 7y ago
      What about their embedded OS? Like one they're running on T2 chip or inside Airpods? Is it still based on XNU kernel?
      - antoinealb 7y ago
        I would be surprised if that was the case. Low-power micro-controllers, such as ARM's Cortex-M series usually do not have support for virtual memory, which is required for running XNU.
      - cnasc 7y ago
        They use L4 in the Secure Enclave. Perhaps they do the same for other embedded devices
      - my123 7y ago
        T2 is a variant of A10 and runs XNU for the main CPU. Firmware cores and AirPods run Apple RTKit.
- keehun 7y ago
  It's also mirrored on Github! [1] [1]: http://github.com/apple/darwin-xnu http://github.com/apple/darwin-xnu
- sudomakeup 7y ago
  I dont recall if it was Apple that did this, but theres trick one can pull with an "open source"codebase thats part of a proprietary product. One can have said codebase effectively outsource functionality to function calls on closed source libraries. Google does something similar with android except in that case its moreso coupling functionality to their services: https://arstechnica.com/gadgets/2018/07/googles-iron-grip-on-android-controlling-open-source-by-any-means-necessary/ https://arstechnica.com/gadgets/2018/07/googles-iron-grip-on...
  - 693471 7y ago
    Apple does not need to do that because of the BSD license, but I've done that with AGPL3 software before. It was the suggested solution by the original developer of the project, whom we hired
self_awareness 7y ago
After being integrated into XNU through the OSFMK kernel, it's not microkernel anymore.
- pjmlp 7y ago
  It will become one again with the new user space drivers being introduced in Catalina. As announced at WWDC, it will be a progressive transition, for every new driver model being supported as user space driver, the related kernel space APIs will be automatically deprecated and removed in the following OS release the year after.
  - valleyer 7y ago
    That still leaves lots of functionality in the kernel, most notably the BSD code (which implements Unix syscalls). So even with the new userspace driver support, most people would not consider xnu a microkernel.
    - pjmlp 7y ago
      It will be very hard to be a monolithic kernel without any sort of kernel space drivers. Also many BSD syscalls have been deprecated along the years, including POSIX features like networking stack, now replaced by Objective-C APIs. I would advise spending some time reading "Mac OS X Internals: A Systems Approach" and "Mac OS X and iOS Internals" books, to learn that having everything on kernel space alone, doesn't rewrite the original microkernel code into a huge monolith.
      - gpderetta 7y ago
        Userspace drivers do not make a microkernel.
        pjmlp 7y ago
        They surely do when there isn't anything else available. Apple showed their long term roadmap at WWDC how they plan to purge all kernel drivers.
drewg123 7y ago
Given a syscall that does nothing, a full round-trip under BSD would require about 40μs, whereas on a user-space Mach system it would take just under 500μs This is still true, but to a lesser extent, on MacOSX (or at least was, a decade ago). I was writing HPC drivers for a cluster interconnect. So performance was critical. We had been using the BSD ioctl system to communicate with our drivers because we used ioctls in all our other drivers (Linux, Solaris, FreeBSD, Windows). I did some microbenchmarks and noticed that it was far slower than FreeBSD or Linux ioctls & complained. Apple suggested that I re-write the app/driver communication using IOKit, which is Mach based. The result was something that was twice as slow.
- big_chungus 7y ago
  So the obvious question is how it has changed over the years. What has apple done to pull latency down to a comparable time?
  - drewg123 7y ago
    I doubt that they care. In the period where I worked on Mac drivers and kept close track of the Darwin sources (roughly 2003->2013) I saw very little in the way of performance improvements. AFAIK, they still don't support 15 year old technologies like MSI-X that permit efficient multi-queue network drivers. Have you ever built a project by hand on MacOS and then on Linux (or FreeBSD)? Have you noticed how absurdly, painfully slow it is running autoconf on MacOSX? That's because MacOS system calls are horrifically slow compared to Linux / BSD.
    - SomeOldThrow 7y ago
      What does “horrifically slow” mean compared to a monolithic syscall? I haven’t run autoconf on linux in years.
      - drewg123 7y ago
        Small number of minutes vs less than a minute. The last time I built something by hand in both places, it seemed to take 2x to 3x as long to run all the autoconf checks on MacOS. That's basically a fork / exec / open /close sort of benchmark.
Austin_Conlon 7y ago
Oral History of Avie Tevanian starting with the Mach segment: https://youtu.be/vwCdKU9uYnE?t=3995 https://youtu.be/vwCdKU9uYnE?t=3995.
wolfspider 7y ago
Actually stumbled across this the other day- had no idea the Mach kernel was being used on Apple hardware before the Intel transition. MachTen was a paid-for Mach kernel based around BSD4.4 https://en.wikipedia.org/wiki/MachTen https://en.wikipedia.org/wiki/MachTen
- pram 7y ago
  Technically they were anyway, since XNU/OSX were on PowerPC ;P
- kijiki 7y ago
  MkLinux ran Linux as the sole server process on Mach on Macs. It was noticeably slower than the direct PowerPC port. It was done by Apple and OSF, and for a while was the only way to run Linux on NuBus Macs.
insulanian 7y ago
> Mach's name Mach evolved in a euphemization spiral: While the developers, once during the naming phase, had to bike to lunch through rainy Pittsburgh's mud puddles, Tevanian joked the word muck could serve as a backronym for their Multi-User [or Multiprocessor Universal] Communication Kernel. Italian CMU engineer Dario Giuse later asked project leader Rick Rashid about the project's current title and received "MUCK" as the answer, though not spelled out but just pronounced as IPA: [mʌk] which he, according to the Italian alphabet, wrote as Mach. Rashid liked Giuse's spelling "Mach" so much that it prevailed. It's fascinating how some things get their names :)
- Hitton 7y ago
  I always thought it was named after Mach number because of its speed. Being named after muck couldn't be farther from that.
  - mcguire 7y ago
    Speed was not one of Mach's attributes.
  - cafard 7y ago
    I had supposed it was named after Ernst Mach, after whom the Mach number is named.
- qubex 7y ago
  I’m Italian and I’m not sure I’d transliterate that way, but...
  - aidenn0 7y ago
    Depends who is speaking; I've heard the vowel sound in "muck" spoken as any of ʌ, ə, or u by native English speakers from different regions.
mcguire 7y ago
I worked at IBM Austin on the performance team when the Mach-based IBM Microkernel/Workplace OS was ongoing. (Ask me anything! :-)) Performance was the big problem. (At one point, a disk read was CPU-bound.) 1. The Mach Interface Generator (mig) generated code with wildly different performance, with no obvious relationship to the mig spec. 2. Context switching is always the big topic, but it's a red herring. Our problem was primarily data transfer; copy-on-write took so long to set up that it was frequently cheaper to just do the copy. 3. Making a syscall to get the current PID is a stupid idea.
- mlinksva 7y ago
  > At one point, a disk read was CPU-bound. That sounds impressive. Could you say more about how/why?
  - mcguire 7y ago
    1. Keep in mind that the programmers had swallowed gallons of the multi-server microkernel koolaid, so the flow was user code -> mk -> file server -> mk -> device driver and back. 2. Someone used the wrong magic keywords in the mig spec, causing poor message send-receive structure and code. 3. The same someone, IIRC, set up the copy on write memory management for each page, rather than one big buffer. The latter would still be slower than just copying, but geeze. 4. There was something wrong with the driver at the time; I never heard what. 5. In fairness, there was some overhead from our monitoring.
- fc_barnes 7y ago
  Is there some relationship between Mach's poor performance and the ability of macOS to remain functional in low-memory situations? Linux has been called out recently for becoming unusable when free memory is low. Using both, I can say that the Mac is hands-down a better OS for desktop use where having the system not freeze is way more important than a 70% slowdown in file transfer rate. Why is the Mac so much better under low memory conditions than Linux? Is it the kernel, and if so, is there an inherent trade-off between low memory performance and other kinds of performance? If there is a trade-off, would reworking the Linux kernel to function better under low memory conditions also create a way forward for a non-dbus, non-systemd, yet modern Linux?
cpeterso 7y ago
Google is developing Fuchsia as a possible replacement for Android's Linux kernel. Are there any reports or rumors about Apple developing a next-generation kernel (perhaps with a safe language like Swift and optimized for mobile devices) to eventually replace Mach/XNU and its historical baggage?
- pjmlp 7y ago
  They are moving all drivers to user space. All the ones corresponding to the former IO Kit do require C++, all the remaining categories are going to be supported from Swift as well. This is planned to take place across several releases, at the end of which no kernel drivers will be any longer allowed.
  - andrekandre 7y ago
    > They are moving all drivers to user space. probably you mean 3-rd party drivers? or even apple internally? [edit] i couldn’t tell that from the slides mentioned below, but maybe i missed something > All the ones corresponding to the former IO Kit do require C++ yea, which leads me to believe if apple was to rewrite the kernel, they probably would go with c++ ... or maybe it’s just that swift doesn’t have its embedded chops up to snuff yet... relevant slides: https://devstreaming-cdn.apple.com/videos/wwdc/2019/702vygott3n041/702/702_system_extensions_and_driverkit.pdf?dl=1 https://devstreaming-cdn.apple.com/videos/wwdc/2019/702vygot...
    - pjmlp 7y ago
      Watch the presentation, as it contains more information as the slides. The long term roadmap is as follows: 1 - surface kernel APIs for a specific driver model as userspace API 2 - deprecate for the respective OS release the kernel entry points related to the newly surfaced driver model 3 - remove the kernel api on the following OS release 4 - rinse and repeat untill there aren't any kernel driver APIs left The ones being released with Catalina are just the first wave.
      - 7y ago

139 points by thatguyagain 7y ago

Apple even open sources their XNU kernel: https://opensource.apple.com/ https://opensource.apple.com/ There's some delay between when a new macOS version comes out and when sources get published, but it's great to see how they use the Mach kernel in practice.

ksec 7y ago
Why do they update the macOS XNU Kernel and not iOS?
- deleted 7y ago
  [deleted]
- shakna 7y ago
  IIRC they share the exact same kernel, so they don't need to.
  - saagarjha 7y ago
    They’re not quite the same, which is why the lack of iOS sources was a minor annoyance for a while.
- blank_pattern 7y ago
  They seem really big on keeping iOS things secret-ish, but the XNU kernel for both iOS and macOS seem to be built from the same code base. Up until a couple years ago, they'd strip out ARM specific things from the releases macOS XNU code. Then they started leaving that stuff in!
  - vbezhenar 7y ago
    What about their embedded OS? Like one they're running on T2 chip or inside Airpods? Is it still based on XNU kernel?
    - antoinealb 7y ago
      I would be surprised if that was the case. Low-power micro-controllers, such as ARM's Cortex-M series usually do not have support for virtual memory, which is required for running XNU.
    - cnasc 7y ago
      They use L4 in the Secure Enclave. Perhaps they do the same for other embedded devices
    - my123 7y ago
      T2 is a variant of A10 and runs XNU for the main CPU. Firmware cores and AirPods run Apple RTKit.
keehun 7y ago
It's also mirrored on Github! [1] [1]: http://github.com/apple/darwin-xnu http://github.com/apple/darwin-xnu
sudomakeup 7y ago
I dont recall if it was Apple that did this, but theres trick one can pull with an "open source"codebase thats part of a proprietary product. One can have said codebase effectively outsource functionality to function calls on closed source libraries. Google does something similar with android except in that case its moreso coupling functionality to their services: https://arstechnica.com/gadgets/2018/07/googles-iron-grip-on-android-controlling-open-source-by-any-means-necessary/ https://arstechnica.com/gadgets/2018/07/googles-iron-grip-on...
- 693471 7y ago
  Apple does not need to do that because of the BSD license, but I've done that with AGPL3 software before. It was the suggested solution by the original developer of the project, whom we hired

self_awareness 7y ago

After being integrated into XNU through the OSFMK kernel, it's not microkernel anymore.

pjmlp 7y ago
It will become one again with the new user space drivers being introduced in Catalina. As announced at WWDC, it will be a progressive transition, for every new driver model being supported as user space driver, the related kernel space APIs will be automatically deprecated and removed in the following OS release the year after.
- valleyer 7y ago
  That still leaves lots of functionality in the kernel, most notably the BSD code (which implements Unix syscalls). So even with the new userspace driver support, most people would not consider xnu a microkernel.
  - pjmlp 7y ago
    It will be very hard to be a monolithic kernel without any sort of kernel space drivers. Also many BSD syscalls have been deprecated along the years, including POSIX features like networking stack, now replaced by Objective-C APIs. I would advise spending some time reading "Mac OS X Internals: A Systems Approach" and "Mac OS X and iOS Internals" books, to learn that having everything on kernel space alone, doesn't rewrite the original microkernel code into a huge monolith.
    - gpderetta 7y ago
      Userspace drivers do not make a microkernel.
      - pjmlp 7y ago
        They surely do when there isn't anything else available. Apple showed their long term roadmap at WWDC how they plan to purge all kernel drivers.

drewg123 7y ago

Given a syscall that does nothing, a full round-trip under BSD would require about 40μs, whereas on a user-space Mach system it would take just under 500μs This is still true, but to a lesser extent, on MacOSX (or at least was, a decade ago). I was writing HPC drivers for a cluster interconnect. So performance was critical. We had been using the BSD ioctl system to communicate with our drivers because we used ioctls in all our other drivers (Linux, Solaris, FreeBSD, Windows). I did some microbenchmarks and noticed that it was far slower than FreeBSD or Linux ioctls & complained. Apple suggested that I re-write the app/driver communication using IOKit, which is Mach based. The result was something that was twice as slow.

big_chungus 7y ago
So the obvious question is how it has changed over the years. What has apple done to pull latency down to a comparable time?
- drewg123 7y ago
  I doubt that they care. In the period where I worked on Mac drivers and kept close track of the Darwin sources (roughly 2003->2013) I saw very little in the way of performance improvements. AFAIK, they still don't support 15 year old technologies like MSI-X that permit efficient multi-queue network drivers. Have you ever built a project by hand on MacOS and then on Linux (or FreeBSD)? Have you noticed how absurdly, painfully slow it is running autoconf on MacOSX? That's because MacOS system calls are horrifically slow compared to Linux / BSD.
  - SomeOldThrow 7y ago
    What does “horrifically slow” mean compared to a monolithic syscall? I haven’t run autoconf on linux in years.
    - drewg123 7y ago
      Small number of minutes vs less than a minute. The last time I built something by hand in both places, it seemed to take 2x to 3x as long to run all the autoconf checks on MacOS. That's basically a fork / exec / open /close sort of benchmark.

Austin_Conlon 7y ago

Oral History of Avie Tevanian starting with the Mach segment: https://youtu.be/vwCdKU9uYnE?t=3995 https://youtu.be/vwCdKU9uYnE?t=3995.

wolfspider 7y ago

Actually stumbled across this the other day- had no idea the Mach kernel was being used on Apple hardware before the Intel transition. MachTen was a paid-for Mach kernel based around BSD4.4 https://en.wikipedia.org/wiki/MachTen https://en.wikipedia.org/wiki/MachTen

pram 7y ago
Technically they were anyway, since XNU/OSX were on PowerPC ;P
kijiki 7y ago
MkLinux ran Linux as the sole server process on Mach on Macs. It was noticeably slower than the direct PowerPC port. It was done by Apple and OSF, and for a while was the only way to run Linux on NuBus Macs.

insulanian 7y ago

> Mach's name Mach evolved in a euphemization spiral: While the developers, once during the naming phase, had to bike to lunch through rainy Pittsburgh's mud puddles, Tevanian joked the word muck could serve as a backronym for their Multi-User [or Multiprocessor Universal] Communication Kernel. Italian CMU engineer Dario Giuse later asked project leader Rick Rashid about the project's current title and received "MUCK" as the answer, though not spelled out but just pronounced as IPA: [mʌk] which he, according to the Italian alphabet, wrote as Mach. Rashid liked Giuse's spelling "Mach" so much that it prevailed. It's fascinating how some things get their names :)

Hitton 7y ago
I always thought it was named after Mach number because of its speed. Being named after muck couldn't be farther from that.
- mcguire 7y ago
  Speed was not one of Mach's attributes.
- cafard 7y ago
  I had supposed it was named after Ernst Mach, after whom the Mach number is named.
qubex 7y ago
I’m Italian and I’m not sure I’d transliterate that way, but...
- aidenn0 7y ago
  Depends who is speaking; I've heard the vowel sound in "muck" spoken as any of ʌ, ə, or u by native English speakers from different regions.

mcguire 7y ago

I worked at IBM Austin on the performance team when the Mach-based IBM Microkernel/Workplace OS was ongoing. (Ask me anything! :-)) Performance was the big problem. (At one point, a disk read was CPU-bound.) 1. The Mach Interface Generator (mig) generated code with wildly different performance, with no obvious relationship to the mig spec. 2. Context switching is always the big topic, but it's a red herring. Our problem was primarily data transfer; copy-on-write took so long to set up that it was frequently cheaper to just do the copy. 3. Making a syscall to get the current PID is a stupid idea.

mlinksva 7y ago
> At one point, a disk read was CPU-bound. That sounds impressive. Could you say more about how/why?
- mcguire 7y ago
  1. Keep in mind that the programmers had swallowed gallons of the multi-server microkernel koolaid, so the flow was user code -> mk -> file server -> mk -> device driver and back. 2. Someone used the wrong magic keywords in the mig spec, causing poor message send-receive structure and code. 3. The same someone, IIRC, set up the copy on write memory management for each page, rather than one big buffer. The latter would still be slower than just copying, but geeze. 4. There was something wrong with the driver at the time; I never heard what. 5. In fairness, there was some overhead from our monitoring.
fc_barnes 7y ago
Is there some relationship between Mach's poor performance and the ability of macOS to remain functional in low-memory situations? Linux has been called out recently for becoming unusable when free memory is low. Using both, I can say that the Mac is hands-down a better OS for desktop use where having the system not freeze is way more important than a 70% slowdown in file transfer rate. Why is the Mac so much better under low memory conditions than Linux? Is it the kernel, and if so, is there an inherent trade-off between low memory performance and other kinds of performance? If there is a trade-off, would reworking the Linux kernel to function better under low memory conditions also create a way forward for a non-dbus, non-systemd, yet modern Linux?

cpeterso 7y ago

Google is developing Fuchsia as a possible replacement for Android's Linux kernel. Are there any reports or rumors about Apple developing a next-generation kernel (perhaps with a safe language like Swift and optimized for mobile devices) to eventually replace Mach/XNU and its historical baggage?

pjmlp 7y ago
They are moving all drivers to user space. All the ones corresponding to the former IO Kit do require C++, all the remaining categories are going to be supported from Swift as well. This is planned to take place across several releases, at the end of which no kernel drivers will be any longer allowed.
- andrekandre 7y ago
  > They are moving all drivers to user space. probably you mean 3-rd party drivers? or even apple internally? [edit] i couldn’t tell that from the slides mentioned below, but maybe i missed something > All the ones corresponding to the former IO Kit do require C++ yea, which leads me to believe if apple was to rewrite the kernel, they probably would go with c++ ... or maybe it’s just that swift doesn’t have its embedded chops up to snuff yet... relevant slides: https://devstreaming-cdn.apple.com/videos/wwdc/2019/702vygott3n041/702/702_system_extensions_and_driverkit.pdf?dl=1 https://devstreaming-cdn.apple.com/videos/wwdc/2019/702vygot...
  - pjmlp 7y ago
    Watch the presentation, as it contains more information as the slides. The long term roadmap is as follows: 1 - surface kernel APIs for a specific driver model as userspace API 2 - deprecate for the respective OS release the kernel entry points related to the newly surfaced driver model 3 - remove the kernel api on the following OS release 4 - rinse and repeat untill there aren't any kernel driver APIs left The ones being released with Catalina are just the first wave.
    - 7y ago