6 ms·

It's my understanding that the microkernel architecture is slow. Nearly unusably slow. And that's why nobody uses it. Am I off-base? I'm interested!

dijit 5y ago
Yes, it’s going to be slower All of those security checks between components and memory passing will cause it to be slower. But that doesn’t mean it’s a worthy trade off. People write software in python despite it being slower than C++.
- hutrdvnj 5y ago
  Except that anything that is actually performance critical is written as a C extension python module. I think that low level filesystem operations are very performance critical.
  - jeffbee 5y ago
    I doubt that mounting a filesystem is performance-critical. You could afford to fork an unprivileged user-space process written in Perl to parse these mount options and that would be fast enough for everyone.
    - YarickR2 5y ago
      Tell that to dockerd mounting images layer by layer , with k8s doing all kinds of emptyDir/PVC mounts on top. Pod start up speeds are abysmal now, they would be positively glacial with userspafe permissions validations
      - jeffbee 5y ago
        That's exactly my point. People who are doing lots of mounts are already demonstrably not performance-sensitive to a difference of a few milliseconds. They already waited 20 minutes for the stupid cluster autoscaler to provision a machine for them. They DGAF.
        Shorel 5y ago
        They just need some time off for leisure activities. https://xkcd.com/303/ https://xkcd.com/303/
      - stormbrew 5y ago
        ?? The layering unionfs (really aufs or overlayfs) mounts docker does are all in kernel. There's no userspace involvement in the actual file ops. I'm not at all sure what you think any of that has to do with the idea of usermode filesystem handling. Anyways, on linux userland filesystem stuff (ie. fuse) is slow because the kernel manages the vfs layer and has to mediate. In a system designed for non-privileged vfs it doesn't have to be anywhere near so involved. And beyond that, there's really nothing in particular that a filesystem driver does that needs to be privileged to 'run fast'. VFS is pretty much the posterchild case for how you can pull things out of a kernel and maintain performance, tbh. There's no reason at all it can't be fast except the kernel getting in the way rather than helping.
jeffbee 5y ago
As more and more things move out of the kernel, the perceived performance problems of microkernels look less important. If you are doing your network protocols in user space, and your thread scheduling is in user space, and you're not using a traditional filesystem much, then suddenly nobody cares how fast the kernel is.
- stormbrew 5y ago
  Yeah this, really. For the most part even monolithic kernels have kind of reversed trend in the last decade or so and there's a lot of push to move critical code out of the kernel. A lot of new kernel apis are built to avoid context switching, and part of that usually involves moving to a more asynchronous kind of communication between process and kernel. Often these APIs even look an awful lot like late generation microkernel shared memory buffer protocols. DRI and uring in linux for example. A lot of the "microkernels are inherently slow" meme is built on how earlier port-based message passing kernels like mach worked.
- athrowaway3z 5y ago
  I'm really not understanding what you're saying here. What the hell is a microkernel here? What kind of security are you talking about? User space filesystem and network implementations still need access to the hardware. Multiplexing that access is a kernels job. The more you want to separate and hide that between clients the higher the cost. As far as i understand your argument you are saying "If an application has a dedicated hard drive there will be little overhead"
  - deleted 5y ago
    [deleted]
  - jeffbee 5y ago
    People think a monolithic kernel can be faster because of high-level abstractions that make many calls within the kernel, for example if I write to a TCP socket and everything else is handled for me, in the kernel, by function calls only. People believe this is faster than having an isolated network stack that has to pass messages to an isolated network driver and all that. But increasingly people realize that the performance of writing to a TCP socket in a unikernel also pretty much sucks and you get a much better result as you move more and more of it into user space. For example you decide, correctly, that TCP is obsolete and you switch to QUIC. Now the existence of the Linux TCP stack is of no value to you. You furthermore discover that Linux firewall, traffic control, and routing also kinda sucks, so you start using raw networking. Then you discover that trying to get your frames processed on the right core at the right moment isn't great in Linux, so you just take over the whole net device with DPDK. Now, nothing in the whole Linux network stack is of any use to you at all. The same thing can happen with storage. Maybe you started with files on XFS but then eventually you were using disaggregated storage where a service takes over the whole device with SPDK, and all the storage users are talking to the service instead of the kernel.
nwmcsween 5y ago
This depends on the definition of the word microkernel, in the classical definition yes it will be much slower due to IPC, for something like an exokernel though it will be much faster than even a monolithic kernel.
snvzz 5y ago
There's more myth than truth[0]. In the early days, they were slow. Mach, used in OSX, is a representative of those early days. Liedtke's L4 proved that a performant microkernel is possible. Later, SMP changed the scenario considerably, as all of a sudden the microkernel multiserver fits SMP like a globe, while monolithic kernels need the complexity of locks to handle it. [0] https://news.ycombinator.com/item?id=10824382 https://news.ycombinator.com/item?id=10824382
- erichocean 5y ago
  > Mach, used in OSX, is a representative of those early days. Mach, the microkernel, is not using in OS X and never has been. Mach, the API, is used (mostly due to legacy reasons). OS X has a Linux-style monolithic kernel (with legacy Mach APIs in userspace).
  - dralley 5y ago
    I thought Windows and OSX were closer to hybrid kernels, at least with respect to running a lot of their drivers in userspace.
    - dippersauce 5y ago
      Almost every developer I have worked with considers NT and XNU to be hybrid at this point. Unfortunately, almost everything in the parent comment is incorrect.
      - erichocean 5y ago
        > Unfortunately, almost everything in the parent comment is incorrect. Avie Tevanian would disagree, but what does he know? P.s. Nice job regurgitating Wikipedia in your other comment, without understanding it. I guess with only 30 karma, you've got to get your HN points somehow… Next time, read the Wikipedia entry on Mach so you'll learn that Mach didn't even become an actual microkernel until version 3.0. OS X's XNU kernel branched at 2.5. Oops.
  - 5y ago
roblabla 5y ago
The Nintendo Switch operating system, Horizon/NX, is a capability-based microkernel architecture. It's nowhere near unusably slow. In fact, its performance are pretty good. It does "cheat" a bit in that the whole OS runs on a single core, dedicating the remaining three cores to the game. But that may actually be the most efficient use of the hardware.
01100011 5y ago
Generally yes. The microkernel architecture is theoretically better in many ways and this has led to many attempts over the years to switch to that architecture. Some attempts have been relatively successful. Over time, as hardware becomes more heterogeneous and more capable, we are likely going to see a move to microkernels. Like many engineering decisions, the choice is not always clear cut. Other commenters have suggested microkernels are clearly superior and not choosing them is stupid, but there are many facets to that decision.
pjmlp 5y ago
QNX is a microkernel and is used everywhere on embedded space, and it isn't the only one. Microkernels are the preferred choice in high integrity computing.

dijit 5y ago

Yes, it’s going to be slower All of those security checks between components and memory passing will cause it to be slower. But that doesn’t mean it’s a worthy trade off. People write software in python despite it being slower than C++.

hutrdvnj 5y ago
Except that anything that is actually performance critical is written as a C extension python module. I think that low level filesystem operations are very performance critical.
- jeffbee 5y ago
  I doubt that mounting a filesystem is performance-critical. You could afford to fork an unprivileged user-space process written in Perl to parse these mount options and that would be fast enough for everyone.
  - YarickR2 5y ago
    Tell that to dockerd mounting images layer by layer , with k8s doing all kinds of emptyDir/PVC mounts on top. Pod start up speeds are abysmal now, they would be positively glacial with userspafe permissions validations
    - jeffbee 5y ago
      That's exactly my point. People who are doing lots of mounts are already demonstrably not performance-sensitive to a difference of a few milliseconds. They already waited 20 minutes for the stupid cluster autoscaler to provision a machine for them. They DGAF.
      - Shorel 5y ago
        They just need some time off for leisure activities. https://xkcd.com/303/ https://xkcd.com/303/
    - stormbrew 5y ago
      ?? The layering unionfs (really aufs or overlayfs) mounts docker does are all in kernel. There's no userspace involvement in the actual file ops. I'm not at all sure what you think any of that has to do with the idea of usermode filesystem handling. Anyways, on linux userland filesystem stuff (ie. fuse) is slow because the kernel manages the vfs layer and has to mediate. In a system designed for non-privileged vfs it doesn't have to be anywhere near so involved. And beyond that, there's really nothing in particular that a filesystem driver does that needs to be privileged to 'run fast'. VFS is pretty much the posterchild case for how you can pull things out of a kernel and maintain performance, tbh. There's no reason at all it can't be fast except the kernel getting in the way rather than helping.

jeffbee 5y ago

As more and more things move out of the kernel, the perceived performance problems of microkernels look less important. If you are doing your network protocols in user space, and your thread scheduling is in user space, and you're not using a traditional filesystem much, then suddenly nobody cares how fast the kernel is.

stormbrew 5y ago
Yeah this, really. For the most part even monolithic kernels have kind of reversed trend in the last decade or so and there's a lot of push to move critical code out of the kernel. A lot of new kernel apis are built to avoid context switching, and part of that usually involves moving to a more asynchronous kind of communication between process and kernel. Often these APIs even look an awful lot like late generation microkernel shared memory buffer protocols. DRI and uring in linux for example. A lot of the "microkernels are inherently slow" meme is built on how earlier port-based message passing kernels like mach worked.
athrowaway3z 5y ago
I'm really not understanding what you're saying here. What the hell is a microkernel here? What kind of security are you talking about? User space filesystem and network implementations still need access to the hardware. Multiplexing that access is a kernels job. The more you want to separate and hide that between clients the higher the cost. As far as i understand your argument you are saying "If an application has a dedicated hard drive there will be little overhead"
- deleted 5y ago
  [deleted]
- jeffbee 5y ago
  People think a monolithic kernel can be faster because of high-level abstractions that make many calls within the kernel, for example if I write to a TCP socket and everything else is handled for me, in the kernel, by function calls only. People believe this is faster than having an isolated network stack that has to pass messages to an isolated network driver and all that. But increasingly people realize that the performance of writing to a TCP socket in a unikernel also pretty much sucks and you get a much better result as you move more and more of it into user space. For example you decide, correctly, that TCP is obsolete and you switch to QUIC. Now the existence of the Linux TCP stack is of no value to you. You furthermore discover that Linux firewall, traffic control, and routing also kinda sucks, so you start using raw networking. Then you discover that trying to get your frames processed on the right core at the right moment isn't great in Linux, so you just take over the whole net device with DPDK. Now, nothing in the whole Linux network stack is of any use to you at all. The same thing can happen with storage. Maybe you started with files on XFS but then eventually you were using disaggregated storage where a service takes over the whole device with SPDK, and all the storage users are talking to the service instead of the kernel.

nwmcsween 5y ago

This depends on the definition of the word microkernel, in the classical definition yes it will be much slower due to IPC, for something like an exokernel though it will be much faster than even a monolithic kernel.

snvzz 5y ago

There's more myth than truth[0]. In the early days, they were slow. Mach, used in OSX, is a representative of those early days. Liedtke's L4 proved that a performant microkernel is possible. Later, SMP changed the scenario considerably, as all of a sudden the microkernel multiserver fits SMP like a globe, while monolithic kernels need the complexity of locks to handle it. [0] https://news.ycombinator.com/item?id=10824382 https://news.ycombinator.com/item?id=10824382

erichocean 5y ago
> Mach, used in OSX, is a representative of those early days. Mach, the microkernel, is not using in OS X and never has been. Mach, the API, is used (mostly due to legacy reasons). OS X has a Linux-style monolithic kernel (with legacy Mach APIs in userspace).
- dralley 5y ago
  I thought Windows and OSX were closer to hybrid kernels, at least with respect to running a lot of their drivers in userspace.
  - dippersauce 5y ago
    Almost every developer I have worked with considers NT and XNU to be hybrid at this point. Unfortunately, almost everything in the parent comment is incorrect.
    - erichocean 5y ago
      > Unfortunately, almost everything in the parent comment is incorrect. Avie Tevanian would disagree, but what does he know? P.s. Nice job regurgitating Wikipedia in your other comment, without understanding it. I guess with only 30 karma, you've got to get your HN points somehow… Next time, read the Wikipedia entry on Mach so you'll learn that Mach didn't even become an actual microkernel until version 3.0. OS X's XNU kernel branched at 2.5. Oops.
- 5y ago

roblabla 5y ago

The Nintendo Switch operating system, Horizon/NX, is a capability-based microkernel architecture. It's nowhere near unusably slow. In fact, its performance are pretty good. It does "cheat" a bit in that the whole OS runs on a single core, dedicating the remaining three cores to the game. But that may actually be the most efficient use of the hardware.

01100011 5y ago

Generally yes. The microkernel architecture is theoretically better in many ways and this has led to many attempts over the years to switch to that architecture. Some attempts have been relatively successful. Over time, as hardware becomes more heterogeneous and more capable, we are likely going to see a move to microkernels. Like many engineering decisions, the choice is not always clear cut. Other commenters have suggested microkernels are clearly superior and not choosing them is stupid, but there are many facets to that decision.

pjmlp 5y ago

QNX is a microkernel and is used everywhere on embedded space, and it isn't the only one. Microkernels are the preferred choice in high integrity computing.