3 ms·

The engineering world has long since moved on from CMU/OSFMK. Rewrite the internals outward using formally-verified, capabilities-secure (seL4 with MAC tagging

The engineering world has long since moved on from CMU/OSFMK. Rewrite the internals outward using formally-verified, capabilities-secure (seL4 with MAC tagging added) and zero-copy message passing RPC in thread-safe Rust. Take IOKit and make it entirely user-space. And make almost everything run in user-space. The need for worrying about placing and cleaning individual allocations goes by the wayside when there's fewer "cooks" throwing spinning fragile plates around and trusting that their aim is perfect. Apple could rewrite it with the same syscall ABI but make it immensely more secure, compartmentalized, and efficient. They won't because of the performance pressures and rewards their employees are under. This is how big companies fail repeatedly and are bested by startups.

conradev 4y ago
The kernel is massive, and rewriting the entire thing takes a serious investment of time (and therefore capital) Zircon is the closest attempt I’ve seen and it has taken years to be shipped on a single, battery-less device. It might be close to ready, but it was likely expensive for Google, which is now scrambling to rein in R&D spending. Meanwhile, the incremental security improvements that Apple and Google have made have driven up the cost of zero days sharply on their devices
- kergonath 4y ago
  Right. Don’t let the perfect be the enemy of the good. If anything, Apple has demonstrated on several occasions that they are able to commit to long-term changes and the sustained effort they may require. It depends on their analysis, but if they think they should do it, they probably already have a team working on it somewhere and a plan to bring it progressively over ~5 years, with a series of incremental steps.
  - pjmlp 4y ago
    The strategy that was communicated at WWDC was that every time a kext model has an userspace alternative, there will be one year transition, and then the following OS version won't support the former kext variant. The long term roadmap is to keep moving the infrastructure to userspace this way, until no one else besides Apple themselves can touch kernel memory space.
    - saagarjha 4y ago
      The problem here is that Apple themselves have problems with their kernel-space code.
      - pjmlp 4y ago
        True, the point is that they are taking multiple paths to make macOS/iOS more secure and (we already discussed this to death), more micro-kernel like, even if not pure. A kernel with Apple only code, is more stable and secure than letting everyone to the party. Every little piece of the puzzle helps.
      - 1letterunixname 4y ago
        Yep. It does too much and it's still stuck in he 1980's. If you want security: a formally-verified microkernel that doesn't do much but memory mapping, IPC, and context switching. Everything else needs to be running either in a VM like BEAM or formally-verifiable, statically-compiled, safe "userland"-ish binaries. Going beyond PNP and ACPI, per-OS hardware drivers could be made unnecessary if vendors provided "ACPI"-like APIs that exposed standard, introspectable functions and data structures to the OS in a unified way independent of bus and architecture, i.e., a GPU, a block device, a serial port, a sensor. A universal "IOKit" would required less integration and development effort for all to support, and every OS would get support for free.
- 1letterunixname 4y ago
  Fuchsia/Zircon is a fractional ray of hope but it's not formally-verified and it still uses methodologies from the 1990's. You misunderstand the approach to migrating: not rip-replace rewrite the world all at once, but refactoring internal areas section-by-section. There's no law that says syscalls have to change. There are few incentives in corporate land to rewrite but to keep ducktapping together the past without addressing the underlying antique, brittle software engineering underneath. seL4, Amoeba, Plan 9, MINIX 3, and DragonFly show what's possible. Neptune OS is interesting. Capabilities-secure, DAC, microkernels are the future. Ideally, kernel modules shouldn't exist and everything apart from memory mapping, security, and IPC should be a "userland-ish" process. Any complicated operations involving multiple components should be software transactional across multiple "processes". Monolithic kernels with file servers and every conceivable library built-in, written in C, with enormous wads of firmware blobs and millions of lines of fragile drivers without any internal protections are the past. Patching together piles of mud is still patching mud.
galad87 4y ago
They are slowly moving things to DriverKit, in each new macOS release they are moving more parts to user-space.
olliej 4y ago
… but that is what Apple is doing?
pjmlp 4y ago
Swift usage on iOS 16 already surpassed C++. Also as other have pointed out, Apple is migrating all kernel extensions into userspace, their long term roadmap is to transition into a micro-kernel like experience, even if not a pure one.

conradev 4y ago

The kernel is massive, and rewriting the entire thing takes a serious investment of time (and therefore capital) Zircon is the closest attempt I’ve seen and it has taken years to be shipped on a single, battery-less device. It might be close to ready, but it was likely expensive for Google, which is now scrambling to rein in R&D spending. Meanwhile, the incremental security improvements that Apple and Google have made have driven up the cost of zero days sharply on their devices

kergonath 4y ago
Right. Don’t let the perfect be the enemy of the good. If anything, Apple has demonstrated on several occasions that they are able to commit to long-term changes and the sustained effort they may require. It depends on their analysis, but if they think they should do it, they probably already have a team working on it somewhere and a plan to bring it progressively over ~5 years, with a series of incremental steps.
- pjmlp 4y ago
  The strategy that was communicated at WWDC was that every time a kext model has an userspace alternative, there will be one year transition, and then the following OS version won't support the former kext variant. The long term roadmap is to keep moving the infrastructure to userspace this way, until no one else besides Apple themselves can touch kernel memory space.
  - saagarjha 4y ago
    The problem here is that Apple themselves have problems with their kernel-space code.
    - pjmlp 4y ago
      True, the point is that they are taking multiple paths to make macOS/iOS more secure and (we already discussed this to death), more micro-kernel like, even if not pure. A kernel with Apple only code, is more stable and secure than letting everyone to the party. Every little piece of the puzzle helps.
    - 1letterunixname 4y ago
      Yep. It does too much and it's still stuck in he 1980's. If you want security: a formally-verified microkernel that doesn't do much but memory mapping, IPC, and context switching. Everything else needs to be running either in a VM like BEAM or formally-verifiable, statically-compiled, safe "userland"-ish binaries. Going beyond PNP and ACPI, per-OS hardware drivers could be made unnecessary if vendors provided "ACPI"-like APIs that exposed standard, introspectable functions and data structures to the OS in a unified way independent of bus and architecture, i.e., a GPU, a block device, a serial port, a sensor. A universal "IOKit" would required less integration and development effort for all to support, and every OS would get support for free.
1letterunixname 4y ago
Fuchsia/Zircon is a fractional ray of hope but it's not formally-verified and it still uses methodologies from the 1990's. You misunderstand the approach to migrating: not rip-replace rewrite the world all at once, but refactoring internal areas section-by-section. There's no law that says syscalls have to change. There are few incentives in corporate land to rewrite but to keep ducktapping together the past without addressing the underlying antique, brittle software engineering underneath. seL4, Amoeba, Plan 9, MINIX 3, and DragonFly show what's possible. Neptune OS is interesting. Capabilities-secure, DAC, microkernels are the future. Ideally, kernel modules shouldn't exist and everything apart from memory mapping, security, and IPC should be a "userland-ish" process. Any complicated operations involving multiple components should be software transactional across multiple "processes". Monolithic kernels with file servers and every conceivable library built-in, written in C, with enormous wads of firmware blobs and millions of lines of fragile drivers without any internal protections are the past. Patching together piles of mud is still patching mud.

galad87 4y ago

They are slowly moving things to DriverKit, in each new macOS release they are moving more parts to user-space.

olliej 4y ago

… but that is what Apple is doing?

pjmlp 4y ago

Swift usage on iOS 16 already surpassed C++. Also as other have pointed out, Apple is migrating all kernel extensions into userspace, their long term roadmap is to transition into a micro-kernel like experience, even if not a pure one.