6 ms·

It's not about clustering, "multiserver" in this context means that parts that are stuffed together in the kernel on Linux or BSD run in separate userspace proc

by krylon 4y ago

It's not about clustering, "multiserver" in this context means that parts that are stuffed together in the kernel on Linux or BSD run in separate userspace processes (i.e. networking, file system, device drivers, ...). This approach gives better isolation of those components, so a bug in one is less likely to affect others, and if one crashes, it can potentially be restarted, while the rest of the system keeps running. If done right, it can also make these individual parts simpler. GNU/Hurd tries the same, but with POSIX compatibility and so far with mixed success. Other multiserver microkernel systems I know off are Minix and QNX. I'm sure there's more.

forgotpwd16 4y ago
Redox, Fuchsia, Genode, Helios/Ares, are some more recent examples.
mike_hearn 4y ago
Modern operating systems are all microkernel-ish to some extent. Linux can run filesystems in usermode processes, Windows does run the graphics subsystem in a separate process, macOS consists of tons of components running in various servers connected by Mach ports and is similar to Android in that they're moving drivers progressively out of the kernel into processes. The two classic examples of the networking stack and the filesystem generally don't move because they aren't buggy enough to be worthwhile, the performance hit is too great and because it's unreasonable to expect userland processes to cleanly handle a restart of things like that. Like, in 99.99% of real programs if every file descriptor is suddenly invalidated then that program will crash and the devs won't care if you report it as a bug because they'll say, yeah well, actually it's on you to fix your buggy OS not on us to code around it. Other platforms don't crash their network or FS driver every five minutes so why should we do extra work for yours? So the kernel vs microkernel distinction doesn't seem particularly relevant or interesting anymore, at least not to me.
- snvzz 4y ago
  >the performance hit is too great I see this myth referenced a lot due to misconceptions constructed around design issues in early, pre-Liedtke microkernels (particularly Mach, used in IOS/OSX) leaving a long-lasting impression. L4 changed everything, and seL4 takes it to the next level. Some references: - https://archive.ph/30hhi https://archive.ph/30hhi - https://youtu.be/INBEKSkAiTA?list=PLtoQeavghzr3atxNQig-sLGbwoKBSVxGV&t=870 https://youtu.be/INBEKSkAiTA?list=PLtoQeavghzr3atxNQig-sLGbw...
  - snvzz 4y ago
    I add: https://sel4.systems/About/seL4-whitepaper.pdf https://sel4.systems/About/seL4-whitepaper.pdf
- pjmlp 4y ago
  That is why Linux is run on top of hypervisors, alongside containers and kubernets orchestration engines, get to use that extra performance.
  - snvzz 4y ago
    With that extra insecurity due to huge TCB. The kernel is MLoCs of poorly structured, highly privileged code. A breeding field for bugs.
- still_grokking 4y ago
  > Modern operating systems are all microkernel-ish to some extent. This won't become true even it gets repeated more. A micro kernel requires a micro kernel design form the ground up. (Something like FDs would be only one example of something where you would need different APIs…) You can't make a micro kernel out of a monolith. That's one of the reasons something like Hurd failed, and something like macOS is a slow and buggy hell. The story about the graphics drivers on Windows is also not true, afik. I've heard they moved this back into kernel space (with some special encapsulation) long ago because it was way to slow otherwise. Because you can't move an existing design to be an micro kernel after the if your APIs are all designed for a monolith setting. Alone the fact that you need async "SysCalls" (or better said "ServerCalls") in a proper micro kernel design makes it impossible to make a micro kernel out of a monolith (where all SysCalls are usually sync, because it makes no sense to have an async layer for intra-process calls). Also micro kernels aren't slow. The canonical link was posted already in a sibling. The main point about micro kernels, and why they still more than matter, is not about bugs and crashes. The main point is about security. You can't have a secure monolith! The TCB would need to be way to large.
  - ngcc_hk 4y ago
    A better word is non-monolithic.
  - pjmlp 4y ago
    > The story about the graphics drivers on Windows is also not true, afik. I've heard they moved this back into kernel space (with some special encapsulation) long ago because it was way to slow otherwise. Yes, if you are stuck using a legacy installation of Windows NT 4 - Windows 7, instead of Windows Vista onwards. That move happened during Windows NT 3.51 to NT 4.0 transition, and was undone during Vista.
    - still_grokking 4y ago
      Thanks for clarifying.
- naasking 4y ago
  > and because it's unreasonable to expect userland processes to cleanly handle a restart of things like that Why would it be more reasonable to handle this in kernel space? Doing well anything in kernel space is always harder and less safe than user space.
  - mike_hearn 4y ago
    You don't handle it in kernel space. If your FS driver segfaults then you get a kernel panic and the entire machine restarts, including all apps, therefore the apps aren't expected to handle such failures.
    - naasking 4y ago
      The "apps" wouldn't be expected to handle this failure in a microkernel OS either, only the driver process would. Since it's outside of the kernel it has a much better chance at doing so. At the very least, you can just kill the driver and restart it, which is effectively like restarting the system just without interrupting the programs that don't depend on it. This actually has been done successfully so it's not just a hypothetical benefit.
      - mike_hearn 4y ago
        If the driver process crashes then presumably other processes have to be able to re-establish a connection and resync state. I think the question is how many programs don't depend on the filesystem. Given that they're loaded from it, you'd presumably need to give up mmap and pre-fault all the code for even very simple programs to survive a loss of connection to the fs driver. It seems like an extreme edge case.
        naasking 4y ago
        I suspect you're still thinking monolithically. Firstly the driver would not expose a filesystem interface, it might expose something like a block-level interface. The filesystem would be provided by yet another process that operates over that block interface. Secondly, mmap implies shared address space mappings, but note that the driver process doesn't have to be the owner of those mappings. If the driver restarts then the mmap process reissues any DMA commands to the faulted driver(s) to re-establish those mappings. So now block devices can be restarted in case of faulty hardware. This might seem like kicking the can down the road a bit, but the point is to partition the statefulness into smaller, more manageable pieces that can be reused. The filesystem process can preserve its caches and open file descriptors, and it should log mmap commands it issued in case of mmap faults as well. This is pretty robust and not so trivial to do at the kernel level where data structures are all intertwined.
- gnufx 4y ago
  The necessarily poor performance of micro-kernel-ish systems was a myth even for a 1970s design (not to diminish Liedkte's work). In the '80s I used fast (realtime, multi-access) GEC OS4000 systems, thankful not to need VMS often. Whether or not the Nucleus was strictly a micro-kernel, it was at least the moral equivalent, message-passing, and I think influenced by the RC4000 (which didn't work so well). I've been logged in when the filesystem was rebooted, not that it was often necessary.

by krylon 4y ago

forgotpwd16 4y ago

Redox, Fuchsia, Genode, Helios/Ares, are some more recent examples.

mike_hearn 4y ago

Modern operating systems are all microkernel-ish to some extent. Linux can run filesystems in usermode processes, Windows does run the graphics subsystem in a separate process, macOS consists of tons of components running in various servers connected by Mach ports and is similar to Android in that they're moving drivers progressively out of the kernel into processes. The two classic examples of the networking stack and the filesystem generally don't move because they aren't buggy enough to be worthwhile, the performance hit is too great and because it's unreasonable to expect userland processes to cleanly handle a restart of things like that. Like, in 99.99% of real programs if every file descriptor is suddenly invalidated then that program will crash and the devs won't care if you report it as a bug because they'll say, yeah well, actually it's on you to fix your buggy OS not on us to code around it. Other platforms don't crash their network or FS driver every five minutes so why should we do extra work for yours? So the kernel vs microkernel distinction doesn't seem particularly relevant or interesting anymore, at least not to me.

snvzz 4y ago
>the performance hit is too great I see this myth referenced a lot due to misconceptions constructed around design issues in early, pre-Liedtke microkernels (particularly Mach, used in IOS/OSX) leaving a long-lasting impression. L4 changed everything, and seL4 takes it to the next level. Some references: - https://archive.ph/30hhi https://archive.ph/30hhi - https://youtu.be/INBEKSkAiTA?list=PLtoQeavghzr3atxNQig-sLGbwoKBSVxGV&t=870 https://youtu.be/INBEKSkAiTA?list=PLtoQeavghzr3atxNQig-sLGbw...
- snvzz 4y ago
  I add: https://sel4.systems/About/seL4-whitepaper.pdf https://sel4.systems/About/seL4-whitepaper.pdf
pjmlp 4y ago
That is why Linux is run on top of hypervisors, alongside containers and kubernets orchestration engines, get to use that extra performance.
- snvzz 4y ago
  With that extra insecurity due to huge TCB. The kernel is MLoCs of poorly structured, highly privileged code. A breeding field for bugs.
still_grokking 4y ago
> Modern operating systems are all microkernel-ish to some extent. This won't become true even it gets repeated more. A micro kernel requires a micro kernel design form the ground up. (Something like FDs would be only one example of something where you would need different APIs…) You can't make a micro kernel out of a monolith. That's one of the reasons something like Hurd failed, and something like macOS is a slow and buggy hell. The story about the graphics drivers on Windows is also not true, afik. I've heard they moved this back into kernel space (with some special encapsulation) long ago because it was way to slow otherwise. Because you can't move an existing design to be an micro kernel after the if your APIs are all designed for a monolith setting. Alone the fact that you need async "SysCalls" (or better said "ServerCalls") in a proper micro kernel design makes it impossible to make a micro kernel out of a monolith (where all SysCalls are usually sync, because it makes no sense to have an async layer for intra-process calls). Also micro kernels aren't slow. The canonical link was posted already in a sibling. The main point about micro kernels, and why they still more than matter, is not about bugs and crashes. The main point is about security. You can't have a secure monolith! The TCB would need to be way to large.
- ngcc_hk 4y ago
  A better word is non-monolithic.
- pjmlp 4y ago
  > The story about the graphics drivers on Windows is also not true, afik. I've heard they moved this back into kernel space (with some special encapsulation) long ago because it was way to slow otherwise. Yes, if you are stuck using a legacy installation of Windows NT 4 - Windows 7, instead of Windows Vista onwards. That move happened during Windows NT 3.51 to NT 4.0 transition, and was undone during Vista.
  - still_grokking 4y ago
    Thanks for clarifying.
naasking 4y ago
> and because it's unreasonable to expect userland processes to cleanly handle a restart of things like that Why would it be more reasonable to handle this in kernel space? Doing well anything in kernel space is always harder and less safe than user space.
- mike_hearn 4y ago
  You don't handle it in kernel space. If your FS driver segfaults then you get a kernel panic and the entire machine restarts, including all apps, therefore the apps aren't expected to handle such failures.
  - naasking 4y ago
    The "apps" wouldn't be expected to handle this failure in a microkernel OS either, only the driver process would. Since it's outside of the kernel it has a much better chance at doing so. At the very least, you can just kill the driver and restart it, which is effectively like restarting the system just without interrupting the programs that don't depend on it. This actually has been done successfully so it's not just a hypothetical benefit.
    - mike_hearn 4y ago
      If the driver process crashes then presumably other processes have to be able to re-establish a connection and resync state. I think the question is how many programs don't depend on the filesystem. Given that they're loaded from it, you'd presumably need to give up mmap and pre-fault all the code for even very simple programs to survive a loss of connection to the fs driver. It seems like an extreme edge case.
      - naasking 4y ago
        I suspect you're still thinking monolithically. Firstly the driver would not expose a filesystem interface, it might expose something like a block-level interface. The filesystem would be provided by yet another process that operates over that block interface. Secondly, mmap implies shared address space mappings, but note that the driver process doesn't have to be the owner of those mappings. If the driver restarts then the mmap process reissues any DMA commands to the faulted driver(s) to re-establish those mappings. So now block devices can be restarted in case of faulty hardware. This might seem like kicking the can down the road a bit, but the point is to partition the statefulness into smaller, more manageable pieces that can be reused. The filesystem process can preserve its caches and open file descriptors, and it should log mmap commands it issued in case of mmap faults as well. This is pretty robust and not so trivial to do at the kernel level where data structures are all intertwined.
gnufx 4y ago
The necessarily poor performance of micro-kernel-ish systems was a myth even for a 1970s design (not to diminish Liedkte's work). In the '80s I used fast (realtime, multi-access) GEC OS4000 systems, thankful not to need VMS often. Whether or not the Nucleus was strictly a micro-kernel, it was at least the moral equivalent, message-passing, and I think influenced by the RC4000 (which didn't work so well). I've been logged in when the filesystem was rebooted, not that it was often necessary.