3 ms·

The usual reason for all of this is that programmer time is expensive (even if you're a volunteer, you have limited hours available), and not many people want t

The usual reason for all of this is that programmer time is expensive (even if you're a volunteer, you have limited hours available), and not many people want to volunteer to wade through tons of legacy tech debt. That's especially true when the outcome will be an OS that behaves identically to before. A lot of stuff stays in the kernel because it's just hard to move it out. Bear in mind, moving stuff out of the kernel is only really worth it if you can come up with a reasonable specification for how to solve a bunch of new problems. If you don't solve them it's easy to screw up and end up with a slower system yet no benefit. Consider what happens if you are overenthusiastic and try to move your core filesystem into userspace. What does the OS do if your filesystem process segfaults? Probably it can't do anything at that point beyond block everything and try to restart it? But every process then lost its connection to the FS server and so all the file handles are suddenly invalidated, meaning every process crashes. You might as well just panic and reboot, so, it might as well stay in the kernel. And what about security? GNU Hurd jumped on the microkernel bandwagon but ended up opening up security vulnerabilities "by design" because they didn't think it through deeply enough (in fairness, these issues are subtle). Having stuff be in the kernel simplifies your architecture tremendously and can avoid bugs as well as create them. People like to claim microkernels are inherently more secure but it's not the case unless you are very careful. So it's good to start monolithic and spin stuff out only when you're ready for the complexity that comes with that. Linux also has the unusual issue that the kernel and userspace are developed independently, which is an obvious problem if you want to move functionality between the two. Windows and macOS can make assumptions about userspace that Linux doesn't. If you want to improve terminals then the wrong place to start is fiddling with moving code between kernel and user space. The right place to start is with a brand new protocol that encodes what you like about text-only interaction and then try to get apps to adopt it or bridge old apps with libc shims etc.

fluoridation 1y ago
>Consider what happens if you are overenthusiastic and try to move your core filesystem into userspace. What does the OS do if your filesystem process segfaults? Probably it can't do anything at that point beyond block everything and try to restart it? But every process then lost its connection to the FS server and so all the file handles are suddenly invalidated, meaning every process crashes. You might as well just panic and reboot, so, it might as well stay in the kernel. I mean, it's not necessarily true that if a filesystem process crashes, every other process crashes. Depending on the design, each FS process may serve requests for each mountpoint, or for each FS type. That already is a huge boon to stability, especially if you're using experimental FSs. On top of that, I think the broken connection could be salvageable by the server storing handle metadata in the kernel and retrieving it when the kernel revives the process. It's hardly an insurmountable problem.
- mike_hearn 1y ago
  Sure it can all be solved, FUSE is an example of doing that for less important ancillary filesystems. I'd actually just make the protocol stateless and store fd state in the clients. My point is more general - the people who design operating systems know all about these tradeoffs and have to decide what to spend time on within a limited budget. Consider: crash bugs are finite. Do you spend your time on complex rearchitecting of your OS to try and fail slightly less hard when some critical code crashes, or do you spend that time fixing the bugs? If the code is big, fast changing and third party then it might make sense to put in the effort, hence FUSE and why graphics drivers often run a big chunk of code out of kernel. If the code is small, stable and performance sensitive, like a core filesystem where all your executables reside, then it doesn't make sense and stays in. Browsers also use a micro-kernelish concept these days. But they're very deliberate and measured about what gets split out into extra processes and what doesn't. The microkernel concept advocates for ignoring engineering tradeoffs in order to put everything into userspace all the time, and says precious little about how to ensure that translates into actual rewards. That's why it's an academic concept that's hardly used today.
  - fluoridation 1y ago
    >crash bugs are finite. Do you spend your time on complex rearchitecting of your OS to try and fail slightly less hard when some critical code crashes, or do you spend that time fixing the bugs? Finite can still be a very large number. Clearly the former is preferable, otherwise your argument applies just as well to usermode code. Why bother having memory protection when the code should be correct anyway? Remember the CloudStrike bug? That wouldn't have happened had the developer been able to put the driver in user mode. The module was not critical, so the system could have kept on running and a normal service could have reported that the driver had failed to start due to an error. That's much, much, much preferable to a boot loop.
    - mike_hearn 1y ago
      Everyone is responsible for their own software, but the OS is more critical than other pieces and also a lot more profitable, so they can afford to invest. Some userspace apps with large budgets do use microkernel architectures, most obviously browsers. But by and large, kernel code is much more tightly scoped and stable than userspace apps. The requirements for a core filesystem change very slowly and a migration from one version to another can take years. Userspace apps might update every week and still be too slow. We tolerate much more instability in the latter than the former.

fluoridation 1y ago

>Consider what happens if you are overenthusiastic and try to move your core filesystem into userspace. What does the OS do if your filesystem process segfaults? Probably it can't do anything at that point beyond block everything and try to restart it? But every process then lost its connection to the FS server and so all the file handles are suddenly invalidated, meaning every process crashes. You might as well just panic and reboot, so, it might as well stay in the kernel. I mean, it's not necessarily true that if a filesystem process crashes, every other process crashes. Depending on the design, each FS process may serve requests for each mountpoint, or for each FS type. That already is a huge boon to stability, especially if you're using experimental FSs. On top of that, I think the broken connection could be salvageable by the server storing handle metadata in the kernel and retrieving it when the kernel revives the process. It's hardly an insurmountable problem.

mike_hearn 1y ago
Sure it can all be solved, FUSE is an example of doing that for less important ancillary filesystems. I'd actually just make the protocol stateless and store fd state in the clients. My point is more general - the people who design operating systems know all about these tradeoffs and have to decide what to spend time on within a limited budget. Consider: crash bugs are finite. Do you spend your time on complex rearchitecting of your OS to try and fail slightly less hard when some critical code crashes, or do you spend that time fixing the bugs? If the code is big, fast changing and third party then it might make sense to put in the effort, hence FUSE and why graphics drivers often run a big chunk of code out of kernel. If the code is small, stable and performance sensitive, like a core filesystem where all your executables reside, then it doesn't make sense and stays in. Browsers also use a micro-kernelish concept these days. But they're very deliberate and measured about what gets split out into extra processes and what doesn't. The microkernel concept advocates for ignoring engineering tradeoffs in order to put everything into userspace all the time, and says precious little about how to ensure that translates into actual rewards. That's why it's an academic concept that's hardly used today.
- fluoridation 1y ago
  >crash bugs are finite. Do you spend your time on complex rearchitecting of your OS to try and fail slightly less hard when some critical code crashes, or do you spend that time fixing the bugs? Finite can still be a very large number. Clearly the former is preferable, otherwise your argument applies just as well to usermode code. Why bother having memory protection when the code should be correct anyway? Remember the CloudStrike bug? That wouldn't have happened had the developer been able to put the driver in user mode. The module was not critical, so the system could have kept on running and a normal service could have reported that the driver had failed to start due to an error. That's much, much, much preferable to a boot loop.
  - mike_hearn 1y ago
    Everyone is responsible for their own software, but the OS is more critical than other pieces and also a lot more profitable, so they can afford to invest. Some userspace apps with large budgets do use microkernel architectures, most obviously browsers. But by and large, kernel code is much more tightly scoped and stable than userspace apps. The requirements for a core filesystem change very slowly and a migration from one version to another can take years. Userspace apps might update every week and still be too slow. We tolerate much more instability in the latter than the former.