9 ms·

This article does a good job of explaining how kernel extensions worked under Intel and how they will work under M1, and the blessed replacement for kernel exte

This article does a good job of explaining how kernel extensions worked under Intel and how they will work under M1, and the blessed replacement for kernel extensions by Apple under M1. What it's missing, and what I assume everyone's questions is: Are the capabilities of Apple's user-level APIs sufficient to replace most legitimate uses of kernel extensions?

pjmlp 5y ago
The roadmap as presented by Apple at WWDC 2019 was that after an userspace replacement for a kernel subsystem gets presented, there is one year deprecation until it gets removed from the kernel on the following OS release. So there is one year to try out what might still be missing and give respective feedback. The long term roadmap is that macOS will eventually be much more microkernel like.
- cpach 5y ago
  Cool! Very interesting development given how bad reputation microkernels had for a while. (I’m thinking of Linux vs Minix/Hurd.)
  - kzrdude 5y ago
    macOS (Mac OS X lineage) was always based on mach (a microkernel) which made it a hybrid kernel: https://en.wikipedia.org/wiki/XNU https://en.wikipedia.org/wiki/XNU
  - zibzab 5y ago
    FYI, Linux userspace drivers were introduced in 2006-2007 and improved multiple times over the years. Some of my filesystem & networking drivers run in usersapce. The monolithic vs microkernel debate was pretty clueless at times. Core Linux is smaller than dito hurd when configured with similar functionality. OSX mach has always been a hybrid kernel. In a sense Linux is too but you can also configure it as a mostly-monolithic thing, and most seem to prefer that.
    - creshal 5y ago
      Some of the userspace drivers are even shared between OSX and Linux, via FUSE.
      - roblabla 5y ago
        Another example would be userspace network drivers that directly talk to the network card. Those tend to be much more performant than using the kernel driver due to avoiding the need to context switch from userspace to kernelspace.
        zibzab 5y ago
        Is that really true in pratice (excluding unikernels, of course)?
        MayeulC 5y ago
        Aren't we making userspace/kernel space communication a lot more efficient with io-uring, though? Likewise, eBPF allows userspace programs to perform some work inside kernel space to reduce syscalls. I wonder if io-uring won't be a starting point for a "microkernelization" of Linux. It seems really effective as a message-passing interface.
        pjmlp 5y ago
        Look at Android' Project Treble, that is what Google has done, just with Android IPC instead of io-uring.
  - pjmlp 5y ago
    Since Android 8, Linux is a kind of pseudo-microkernel, as all drivers are required to be Treble based, and traditional Linux drivers are considered legacy on the platform. https://source.android.com/devices/architecture/hal-types https://source.android.com/devices/architecture/hal-types The Switch (yes the games console) uses a microkernel. QNX an OS for human critical scenarios is a microkernel. Finally, when arguing Linux vs Minix/Hurd, think of the irony of running Linux on top of a cloud hypervisor, with an endless load of Kubernetes or Docker loads.
    - SlowRobotAhead 5y ago
      > QNX an OS for human critical scenarios is a microkernel. Hmm, Never thought about a stated OS focus like that. Off to look it up. Good mention
      - pjmlp 5y ago
        Search for high integrity computing, that is the industry term.
  - tzs 5y ago
    At the time of the Linux vs. Minix debate, on pretty much every performance measure that Linus said monolithic kernels were better at the microkernel QNX was totally wiping the floor with Linux.
- AshamedCaptain 5y ago
  I am quite sure that we have been other this several times in HN. The fact that they're no longer going to allow loading of 3rd party kernel modules (a pure political decision) does not a make a microkernel (which is an architectural style). The kernel is still going to be big and monolithic (it contains the entire BSD kernel in the same address space!), and they're still going to be loading Apple-authorized kernel modules (500MiB of them, according to the article) in the same address space.
  - pjmlp 5y ago
    As you wish, I am no longer going to debate the pedantic meaning of pure microkernel ideology. I rather be happy for the improvements in the long term roadmap and overall outcome for the eco-system.
    - AshamedCaptain 5y ago
      "Pedantic"? If you just consider disallowing 3rd party modules to be a "microkernel" all while the kernel itself still loads most if not all (approved) drivers such as PCI bus, storage devices or filesystems in the same address space, then _all_ current major operating systems are "microkernels".
      - pjmlp 5y ago
        From all pratical purposes for a third party driver writer persona, the only path forward are user space drivers. What you call a kernel like that?
        Nursie 5y ago
        Tivoised?
        Bjartr 5y ago
        Non-extensible? Locked? "A kernel that only allows user-space third party drivers"? That this style doesn't have a nifty name isn't a good reason to weaken the established definition of microkernel just so it does.
webmobdev 5y ago
> Are the capabilities of Apple's user-level APIs sufficient to replace most legitimate uses of kernel extensions? From what I remember reading from some macOS Application Firewall makers, no. (And that's ofcourse, intentional.) While it is true that poorly coded Kernel extensions can make an OS unstable, this is just another example of Apple taking away more control from its users and further crippling macOS to make it more like ios.
- least 5y ago
  After feedback from aforementioned firewall developers, changes were made to network extensions that allowed them to do what they wanted with their applications (for the most part, at least). I still think that it’s a worse in-practice system than kexts but they are responding to feedback at least. You can still load kernel extensions in Big Sur by disabling SIP, though.
  - no_time 5y ago
    I wonder how far away are we from governments mandating SIP and equivalents to be turned on to connect to the internet.
- IfOnlyYouKnew 5y ago
  Well, I'm glad to see even the cynics can't find an evil motive here, and has to resort to just assuming stupidity. That, however, isn't easy to do, considering Apple has a rather impressive track record of creating products people enjoy and buy. And to jump two replies ahead: them trying to make it "easy" and "save" doesn't imply their users are idiots. Quite a few developers and scientists use Macs, and the lawyers aren't exactly dumb either. Spending hours trying to somehow get both sound and bluetooth to work at the same time, a favourite pastime on Linux, means lifetime wasted for something that shouldn't need doing. And considering nobody reads the source, there is no reason to believe some "expert" has some ability to avoid installing that one extension that soon starts encrypting their files. At least they tend to have better backups...
  - Hackbraten 5y ago
AshamedCaptain 5y ago
The entire "ring 1" thing is highly likely false. It's been decades since I last saw anyone using the x86 ring levels and I seriously doubt XNU is using them. In any case, extensions run in the same address space as the rest of the kernel making any such "rings" pointless.
- als0 5y ago
  Rings 1 and 2 don't even exist on AMD64/x86-64.
  - zvrba 5y ago
    Yes they do, but segmentation is practically non-existent and page tables distinguish only between rings 0 and rings 1-3. So no, they aren't very useful, but they do exist and, AFAIK, they're still checked in "far" control transfers.
    - tzs 5y ago
      It's too bad they have been made useless. I'm not fully sure about this, but I think that if you could have two rings for user code so that applications that want to run untrusted code in a sandbox could run the main application in ring 2 and the sandboxed code in ring 3, you could make the application safe from Spectre attacks from the sandboxed code without giving up performance.
      - bonzini 5y ago
        The point of Spectre attacks is that they use side channel to infer the target's data from communication between the target and the attacking userspace process. This is possible even over the network, so ring 2 wouldn't help much.
- zinekeller 5y ago
  I mean, recent developments in the Windows/NT architecture uses virtualization to create a three-level "ring" system, so while it's not strictly architectural "rings", it's effectively a tri-ring system. But then, I haven't delved into how macOS in M1 precisely works.
deregulateMed 5y ago
"blessed" replacement Why did you use the word Blessed? I'm curious what made your brain choose that word instead of something neutral. Reminds me of "Walled garden" marketing speak instead of "Walled Prison". Edit- is there some sensitive nerve I hit?
- stefanfisk 5y ago
  I can't speak for the author, but personally I'm enough of an atheist to not read "blessed" as more than "approved by the powers that be".
- andyzweb 5y ago
  I thought it might have been some kind of reference to the bless(8) command.
  - cyberge99 5y ago
    In context this is how it’s typically used.
- zepto 5y ago
  > "Walled Prison" I'm curious what made your brain choose that word instead of something neutral.
  - andrekandre 5y ago
    what would be a good term?
    - zepto 5y ago
      Why do you ask?
      - andrekandre 5y ago
        i was just curious people use metaphors with a lot of conotation (walled prison) so if there is a more neutral term, i was curious how to state it in either non-positive or non-negative way
smoldesu 5y ago
> Are the capabilities of Apple's user-level APIs sufficient to replace most legitimate uses of kernel extensions? I would have trusted literally anyone else to do a good job at this. Apple's in the business of brokering power though, I see no reason why they'd be motivated to make those APIs as rich as kexts.
- Razengan 5y ago
  Why?
- Someone 5y ago
  The whole reason for this change is to limit the damage drivers can do. Because of that, nobody would be stupid enough to make those APIs (in the wide sense, as in “what kexts can do, not what they are supposed to do) as rich as kexts.

pjmlp 5y ago

The roadmap as presented by Apple at WWDC 2019 was that after an userspace replacement for a kernel subsystem gets presented, there is one year deprecation until it gets removed from the kernel on the following OS release. So there is one year to try out what might still be missing and give respective feedback. The long term roadmap is that macOS will eventually be much more microkernel like.

cpach 5y ago
Cool! Very interesting development given how bad reputation microkernels had for a while. (I’m thinking of Linux vs Minix/Hurd.)
- kzrdude 5y ago
  macOS (Mac OS X lineage) was always based on mach (a microkernel) which made it a hybrid kernel: https://en.wikipedia.org/wiki/XNU https://en.wikipedia.org/wiki/XNU
- zibzab 5y ago
  FYI, Linux userspace drivers were introduced in 2006-2007 and improved multiple times over the years. Some of my filesystem & networking drivers run in usersapce. The monolithic vs microkernel debate was pretty clueless at times. Core Linux is smaller than dito hurd when configured with similar functionality. OSX mach has always been a hybrid kernel. In a sense Linux is too but you can also configure it as a mostly-monolithic thing, and most seem to prefer that.
  - creshal 5y ago
    Some of the userspace drivers are even shared between OSX and Linux, via FUSE.
    - roblabla 5y ago
      Another example would be userspace network drivers that directly talk to the network card. Those tend to be much more performant than using the kernel driver due to avoiding the need to context switch from userspace to kernelspace.
      - zibzab 5y ago
        Is that really true in pratice (excluding unikernels, of course)?
      - MayeulC 5y ago
        Aren't we making userspace/kernel space communication a lot more efficient with io-uring, though? Likewise, eBPF allows userspace programs to perform some work inside kernel space to reduce syscalls. I wonder if io-uring won't be a starting point for a "microkernelization" of Linux. It seems really effective as a message-passing interface.
        pjmlp 5y ago
        Look at Android' Project Treble, that is what Google has done, just with Android IPC instead of io-uring.
- pjmlp 5y ago
  Since Android 8, Linux is a kind of pseudo-microkernel, as all drivers are required to be Treble based, and traditional Linux drivers are considered legacy on the platform. https://source.android.com/devices/architecture/hal-types https://source.android.com/devices/architecture/hal-types The Switch (yes the games console) uses a microkernel. QNX an OS for human critical scenarios is a microkernel. Finally, when arguing Linux vs Minix/Hurd, think of the irony of running Linux on top of a cloud hypervisor, with an endless load of Kubernetes or Docker loads.
  - SlowRobotAhead 5y ago
    > QNX an OS for human critical scenarios is a microkernel. Hmm, Never thought about a stated OS focus like that. Off to look it up. Good mention
    - pjmlp 5y ago
      Search for high integrity computing, that is the industry term.
- tzs 5y ago
  At the time of the Linux vs. Minix debate, on pretty much every performance measure that Linus said monolithic kernels were better at the microkernel QNX was totally wiping the floor with Linux.
AshamedCaptain 5y ago
I am quite sure that we have been other this several times in HN. The fact that they're no longer going to allow loading of 3rd party kernel modules (a pure political decision) does not a make a microkernel (which is an architectural style). The kernel is still going to be big and monolithic (it contains the entire BSD kernel in the same address space!), and they're still going to be loading Apple-authorized kernel modules (500MiB of them, according to the article) in the same address space.
- pjmlp 5y ago
  As you wish, I am no longer going to debate the pedantic meaning of pure microkernel ideology. I rather be happy for the improvements in the long term roadmap and overall outcome for the eco-system.
  - AshamedCaptain 5y ago
    "Pedantic"? If you just consider disallowing 3rd party modules to be a "microkernel" all while the kernel itself still loads most if not all (approved) drivers such as PCI bus, storage devices or filesystems in the same address space, then _all_ current major operating systems are "microkernels".
    - pjmlp 5y ago
      From all pratical purposes for a third party driver writer persona, the only path forward are user space drivers. What you call a kernel like that?
      - Nursie 5y ago
        Tivoised?
      - Bjartr 5y ago
        Non-extensible? Locked? "A kernel that only allows user-space third party drivers"? That this style doesn't have a nifty name isn't a good reason to weaken the established definition of microkernel just so it does.

webmobdev 5y ago

> Are the capabilities of Apple's user-level APIs sufficient to replace most legitimate uses of kernel extensions? From what I remember reading from some macOS Application Firewall makers, no. (And that's ofcourse, intentional.) While it is true that poorly coded Kernel extensions can make an OS unstable, this is just another example of Apple taking away more control from its users and further crippling macOS to make it more like ios.

least 5y ago
After feedback from aforementioned firewall developers, changes were made to network extensions that allowed them to do what they wanted with their applications (for the most part, at least). I still think that it’s a worse in-practice system than kexts but they are responding to feedback at least. You can still load kernel extensions in Big Sur by disabling SIP, though.
- no_time 5y ago
  I wonder how far away are we from governments mandating SIP and equivalents to be turned on to connect to the internet.
IfOnlyYouKnew 5y ago
Well, I'm glad to see even the cynics can't find an evil motive here, and has to resort to just assuming stupidity. That, however, isn't easy to do, considering Apple has a rather impressive track record of creating products people enjoy and buy. And to jump two replies ahead: them trying to make it "easy" and "save" doesn't imply their users are idiots. Quite a few developers and scientists use Macs, and the lawyers aren't exactly dumb either. Spending hours trying to somehow get both sound and bluetooth to work at the same time, a favourite pastime on Linux, means lifetime wasted for something that shouldn't need doing. And considering nobody reads the source, there is no reason to believe some "expert" has some ability to avoid installing that one extension that soon starts encrypting their files. At least they tend to have better backups...
- Hackbraten 5y ago

AshamedCaptain 5y ago

The entire "ring 1" thing is highly likely false. It's been decades since I last saw anyone using the x86 ring levels and I seriously doubt XNU is using them. In any case, extensions run in the same address space as the rest of the kernel making any such "rings" pointless.

als0 5y ago
Rings 1 and 2 don't even exist on AMD64/x86-64.
- zvrba 5y ago
  Yes they do, but segmentation is practically non-existent and page tables distinguish only between rings 0 and rings 1-3. So no, they aren't very useful, but they do exist and, AFAIK, they're still checked in "far" control transfers.
  - tzs 5y ago
    It's too bad they have been made useless. I'm not fully sure about this, but I think that if you could have two rings for user code so that applications that want to run untrusted code in a sandbox could run the main application in ring 2 and the sandboxed code in ring 3, you could make the application safe from Spectre attacks from the sandboxed code without giving up performance.
    - bonzini 5y ago
      The point of Spectre attacks is that they use side channel to infer the target's data from communication between the target and the attacking userspace process. This is possible even over the network, so ring 2 wouldn't help much.
zinekeller 5y ago
I mean, recent developments in the Windows/NT architecture uses virtualization to create a three-level "ring" system, so while it's not strictly architectural "rings", it's effectively a tri-ring system. But then, I haven't delved into how macOS in M1 precisely works.

deregulateMed 5y ago

"blessed" replacement Why did you use the word Blessed? I'm curious what made your brain choose that word instead of something neutral. Reminds me of "Walled garden" marketing speak instead of "Walled Prison". Edit- is there some sensitive nerve I hit?

stefanfisk 5y ago
I can't speak for the author, but personally I'm enough of an atheist to not read "blessed" as more than "approved by the powers that be".
andyzweb 5y ago
I thought it might have been some kind of reference to the bless(8) command.
- cyberge99 5y ago
  In context this is how it’s typically used.
zepto 5y ago
> "Walled Prison" I'm curious what made your brain choose that word instead of something neutral.
- andrekandre 5y ago
  what would be a good term?
  - zepto 5y ago
    Why do you ask?
    - andrekandre 5y ago
      i was just curious people use metaphors with a lot of conotation (walled prison) so if there is a more neutral term, i was curious how to state it in either non-positive or non-negative way

smoldesu 5y ago

> Are the capabilities of Apple's user-level APIs sufficient to replace most legitimate uses of kernel extensions? I would have trusted literally anyone else to do a good job at this. Apple's in the business of brokering power though, I see no reason why they'd be motivated to make those APIs as rich as kexts.

Razengan 5y ago
Why?
Someone 5y ago
The whole reason for this change is to limit the damage drivers can do. Because of that, nobody would be stupid enough to make those APIs (in the wide sense, as in “what kexts can do, not what they are supposed to do) as rich as kexts.