5 ms·

Can someone explain to me if the problem with Hurd was that the basic idea was impossible, or it was the way they went about it? i.e. was it the relentless res

Can someone explain to me if the problem with Hurd was that the basic idea was impossible, or it was the way they went about it? i.e. was it the relentless restarting of the project that was to blame, or did they keep restarting because every way they approached it turned out to be impossible?

loeg 14y ago
Microkernels were an immensely popular idea in academia (Mach, Minix, …) when Hurd was begun; I don't think the fundamental problem with them — performance is horrible — had been made obvious yet. Microkernel design is probably a good idea, but the message passing overhead kills actual microkernel implementations. I've heard the Windows NT kernel described as being designed like a microkernel architecture (separate modules with clear APIs), but with direct function calls instead of message passing. I don't know if the Linux/BSD kernels are much different — I've poked at all three, and at a high level they look very similar.
- mcartyem 14y ago
  The message passing overhead issues have been successfully addressed. http://ertos.nicta.com.au/research/l4/ http://ertos.nicta.com.au/research/l4/ http://ssrg.nicta.com.au/publications/papers/Heiser_08lca.slides.pdf http://ssrg.nicta.com.au/publications/papers/Heiser_08lca.sl...
  - AndrewDucker 14y ago
    Thanks for that. The PDF and http://en.wikipedia.org/wiki/L4_microkernel_family http://en.wikipedia.org/wiki/L4_microkernel_family were both fascinating. I then read a bit about an attempt to port Hurd to L4, which floundered for reasons which are beyond my understanding.
    - deleted 14y ago
      [deleted]
  - zanny 14y ago
    Those slides made my head hurt. The problem it tries to bring up is that there is a subset of processes running in the Linux Kernel as either baked in functionality or KOs that can avoid context switches and IPC calls by being baked into the same executable. In practice, you can share memory. In practice, a code base as large as Linux actually ends up having a lot of TLB flushing whenever it alternates between the very different sections of binary that dictate behaviors of the core kernel and devices. It doesn't help that the self fulfilling prophecy is that by fine tuning a baked in service into Linux means that a lot less effort is made trying to make it an efficient interface. I am obviously biased, I think microkernels are a good idea. I don't know if its unix or not, but I don't like having one binary that will inject itself with extra linkables when needed that manages every device, the network stack, the sound API, manages the filesystem, manages memory, and manages processor scheduling. Most of this stuff doesn't need the kernel mode special processor registers, in practice a well designed system shouldn't require anything more than a fixed memory page reference to the device that the kernel will handle mapping (I can't think of a way to build a kernel without virtual memory, virtual files, and virtual processing baked in at the ground level, and 2/3 of those really do require special processor permissions).
    - mcartyem 14y ago
      You might not be biased. It's hard to ignore Wombat demonstrating context-switching costs that are up to 30 times lower than native Linux. The status quo of kernels is reminiscent of the status quo of programming languages in the 90s. Where people programmed in faster, static languages and looked down on dynamic ones. I wonder what we will think of monolithic kernels in 50 years from now. EDIT: A binary injecting itself with multiple code (linkables) might not be bad. It's much like what a macro does to a program. You turn one flag on and the source gets populated with the appropriate code. It is lack of a dynamic way of injecting it that can bite you. The inability to easily change what was injected.
- zimbatm 14y ago
  I've read the same issues about performances and have always wondered how the QNX micro-kernel could power embedded devices (and the next BlackBerry). http://en.wikipedia.org/wiki/QNX http://en.wikipedia.org/wiki/QNX
  - jacquesm 14y ago
    QnX is extremely low overhead, and gets a much more responsive system out of the same hardware than a linux based system. Linux simply doesn't do soft-realtime well, forget about hard realtime, even with the patches applied the granularity is terrible, and prioritization is worse. Those are the things where a microkernel shines. Because realtime systems value responsiveness (and minimal latency) over raw throughput they look worse on paper, but in use they feel a lot more responsive. In a nutshell, having 300 MB/s throughput for some system function with latency ranging from 2 ms to 100 ms vs having 200 MB/s with latency ranging from 2 ms to 5 ms will give a very strong edge to the slower system if there is some cut-off beyond which the latency becomes unacceptable (for instance, machine or process control). Lower performance on paper does not always mean worse performance, it depends very much on which criteria are considered to be more important. And latency can be much more important than throughput.
- haberman 14y ago
  > I don't think the fundamental problem with them — performance is horrible — had been made obvious yet. On the contrary, the idea that performance of microkernels is "horrible" is the current received wisdom, believed by the majority of programmers without critical examination, and based on very performance-poor early microkernel designs like Mach. The truth is that modern microkernel designs like L4 can perform IPC over 20 times faster than Mach. Another important advance for microkernels are tagged TLBs which can alleviate the TLB misses that are usually incurred on every context switch. Someday, hopefully not too far in the future, someone is going to write a modern, practical, and free microkernel-based OS that implements all of POSIX with performance the rivals Linux, but that offers a level of isolation and modularity that Linux could never offer. When that happens, a lot of people are going to scratch their heads and wonder why they believed the people who told them that microkernels are "fundamentally" slow. I had hoped that HelenOS would become this (http://www.helenos.org/ http://www.helenos.org/), but its lowest-layer ipc abstraction is async-based, which I think is a mistake. One of L4's innovations was to use sync IPC, which offers the highest possible performance and gets the kernel out of the business of queueing. You can always build async ipc on top of sync without loss of performance; this puts the queues in user-space which is where they belong (they're easier to account for this way). I asked the HelenOS people why they decided to go this way, and this is their response (one of their points was "the L4's focus on performance is not that important these days", which I disagree with). http://www.mail-archive.com/helenos-devel@lists.modry.cz/msg00338.html http://www.mail-archive.com/helenos-devel@lists.modry.cz/msg...
  - derleth 14y ago
    > Someday, hopefully not too far in the future, someone is going to write a modern, practical, and free microkernel-based OS that implements all of POSIX with performance the rivals Linux, but that offers a level of isolation and modularity that Linux could never offer. Why, when we already have VMs like Xen which are modern, practical, free, host guest OSes that run at full speed to provide whatever API the applications want, and provide a level of isolation and modularity microkernels never could?
    - haberman 14y ago
- _delirium 14y ago
  One thing that's interesting is that, while microkernels haven't really won, Xen-style paravirtualization, which in a way is an even more radical version of the de-monolithization idea [1], has taken over in a lot of niches, despite performance being an even bigger problem, especially without hardware assistance. [1] This paper, for example, argues that VMMs are "microkernels done right": http://static.usenix.org/event/hotos05/final_papers/full_papers/hand/hand_html/ http://static.usenix.org/event/hotos05/final_papers/full_pap...
- jacquesm 14y ago
  > I don't think the fundamental problem with them — performance is horrible — had been made obvious yet. Microkernels can be just as fast as macrokernels and other than raw throughput can give latency guarantees that a macrokernel can't. Message passing overhead can be minimized (by using the paging mechanism for passing messages between tasks on the same node).
- tzs 14y ago
  At the time Linus was ranting about micro kernels, QNX, a micro kernel, soundly kicked the Linux kernel's butt in pretty much every way.
  - philh 14y ago
    That's (weak) evidence that macrokernels are superior. QNX had a head start, but did not maintain the lead.
- 0x0 14y ago
  Mac OS X' kernel is Xnu, http://en.wikipedia.org/wiki/Xnu http://en.wikipedia.org/wiki/Xnu , which is based on the Mach microkernel but with pieces and bits from FreeBSD bolted on, for a hybrid approach.
rogerbinns 14y ago
They emphasised architectural elegance over short term practicality - ie do it right rather than do it quickly. The problem with operating systems is that you need to have a minimum set of working functionality in order for it to be somewhat useful to people not developing the operating system itself. You need a way of storing data, you need a reasonable number of commands to run, you need editors etc. On the hardware side there was immense turnover in the late eighties and nineties. Processor architectures soared and waned, it wasn't uncommon for cpu speeds to double, memory become increasingly cheaper, peripherals changed (mice, cdroms, floppies changed size, Zip drives, tape drives), popular hardware buses changed (ISA 8/16 bit, ADB, VLB, PCI, PCMCIA, USB, serial, parallel). Just keeping up without changing your operating system core functionality was a lot of work! The GNU project tended to work very cathedral like, while Linux was very bazaar like. The latter meant people could support and update the kernel for their own devices ("crowdsourcing" in today's terminology). http://www.catb.org/esr/writings/homesteading/cathedral-bazaar/ http://www.catb.org/esr/writings/homesteading/cathedral-baza... To give you an idea of how many shortcuts Linux took, the way the original kernel worked was that it used a single large address space. Each process was slotted into 64MB of that address space. That left you with a maximum of 63 processes. A context switch just involved changing protections to enable the appropriate slot. This is far more lightweight than how Linux now and other operating systems do it with no such process limits and a far more heavyweight full address space switch. Back then it made Linux really fast at process switching. This is an example of "worse is better": http://en.wikipedia.org/wiki/Worse_is_better http://en.wikipedia.org/wiki/Worse_is_better What the detractors didn't realise is that the initial simplistic Linux implementation could evolve away the constraints, and were instead betting on up front "intelligent design".
- zby 14y ago
  Isn't that ironical when you consider that user space hardware controllers mean much less coordination needed - i.e. be much more bazaar style?
jacquesm 14y ago
Not only was it perfectly possible, microkernels have been sold commercially for decades and are extremely popular in hard real time environments where you want to be able to pre-empt a driver as though it is just another user process (which they are!).
qxcv 14y ago
This XKCD[0] explains the issue with Hurd perfectly, more-or-less echoing the ideas rogerbinns expressed. Note the mouseover text, which makes the observation that the Hurd loop is the loop in which you "code right" and the Android loop is the loop in which you "code fast". [0] https://xkcd.com/844/ https://xkcd.com/844/
frozenport 14y ago
Linux had a working version when the FSF had a longterm vision. There is nothing impossible in their design and more complex systems such as Windows or Inferno/Plan9 exist. In the case of Windows imagine how many hours were spent in its design. When Linux came out it worked and everybody was shocked, Hurd was never able to do this.

loeg 14y ago

Microkernels were an immensely popular idea in academia (Mach, Minix, …) when Hurd was begun; I don't think the fundamental problem with them — performance is horrible — had been made obvious yet. Microkernel design is probably a good idea, but the message passing overhead kills actual microkernel implementations. I've heard the Windows NT kernel described as being designed like a microkernel architecture (separate modules with clear APIs), but with direct function calls instead of message passing. I don't know if the Linux/BSD kernels are much different — I've poked at all three, and at a high level they look very similar.

mcartyem 14y ago
The message passing overhead issues have been successfully addressed. http://ertos.nicta.com.au/research/l4/ http://ertos.nicta.com.au/research/l4/ http://ssrg.nicta.com.au/publications/papers/Heiser_08lca.slides.pdf http://ssrg.nicta.com.au/publications/papers/Heiser_08lca.sl...
- AndrewDucker 14y ago
  Thanks for that. The PDF and http://en.wikipedia.org/wiki/L4_microkernel_family http://en.wikipedia.org/wiki/L4_microkernel_family were both fascinating. I then read a bit about an attempt to port Hurd to L4, which floundered for reasons which are beyond my understanding.
  - deleted 14y ago
    [deleted]
- zanny 14y ago
  Those slides made my head hurt. The problem it tries to bring up is that there is a subset of processes running in the Linux Kernel as either baked in functionality or KOs that can avoid context switches and IPC calls by being baked into the same executable. In practice, you can share memory. In practice, a code base as large as Linux actually ends up having a lot of TLB flushing whenever it alternates between the very different sections of binary that dictate behaviors of the core kernel and devices. It doesn't help that the self fulfilling prophecy is that by fine tuning a baked in service into Linux means that a lot less effort is made trying to make it an efficient interface. I am obviously biased, I think microkernels are a good idea. I don't know if its unix or not, but I don't like having one binary that will inject itself with extra linkables when needed that manages every device, the network stack, the sound API, manages the filesystem, manages memory, and manages processor scheduling. Most of this stuff doesn't need the kernel mode special processor registers, in practice a well designed system shouldn't require anything more than a fixed memory page reference to the device that the kernel will handle mapping (I can't think of a way to build a kernel without virtual memory, virtual files, and virtual processing baked in at the ground level, and 2/3 of those really do require special processor permissions).
  - mcartyem 14y ago
    You might not be biased. It's hard to ignore Wombat demonstrating context-switching costs that are up to 30 times lower than native Linux. The status quo of kernels is reminiscent of the status quo of programming languages in the 90s. Where people programmed in faster, static languages and looked down on dynamic ones. I wonder what we will think of monolithic kernels in 50 years from now. EDIT: A binary injecting itself with multiple code (linkables) might not be bad. It's much like what a macro does to a program. You turn one flag on and the source gets populated with the appropriate code. It is lack of a dynamic way of injecting it that can bite you. The inability to easily change what was injected.
zimbatm 14y ago
I've read the same issues about performances and have always wondered how the QNX micro-kernel could power embedded devices (and the next BlackBerry). http://en.wikipedia.org/wiki/QNX http://en.wikipedia.org/wiki/QNX
- jacquesm 14y ago
  QnX is extremely low overhead, and gets a much more responsive system out of the same hardware than a linux based system. Linux simply doesn't do soft-realtime well, forget about hard realtime, even with the patches applied the granularity is terrible, and prioritization is worse. Those are the things where a microkernel shines. Because realtime systems value responsiveness (and minimal latency) over raw throughput they look worse on paper, but in use they feel a lot more responsive. In a nutshell, having 300 MB/s throughput for some system function with latency ranging from 2 ms to 100 ms vs having 200 MB/s with latency ranging from 2 ms to 5 ms will give a very strong edge to the slower system if there is some cut-off beyond which the latency becomes unacceptable (for instance, machine or process control). Lower performance on paper does not always mean worse performance, it depends very much on which criteria are considered to be more important. And latency can be much more important than throughput.
haberman 14y ago
> I don't think the fundamental problem with them — performance is horrible — had been made obvious yet. On the contrary, the idea that performance of microkernels is "horrible" is the current received wisdom, believed by the majority of programmers without critical examination, and based on very performance-poor early microkernel designs like Mach. The truth is that modern microkernel designs like L4 can perform IPC over 20 times faster than Mach. Another important advance for microkernels are tagged TLBs which can alleviate the TLB misses that are usually incurred on every context switch. Someday, hopefully not too far in the future, someone is going to write a modern, practical, and free microkernel-based OS that implements all of POSIX with performance the rivals Linux, but that offers a level of isolation and modularity that Linux could never offer. When that happens, a lot of people are going to scratch their heads and wonder why they believed the people who told them that microkernels are "fundamentally" slow. I had hoped that HelenOS would become this (http://www.helenos.org/ http://www.helenos.org/), but its lowest-layer ipc abstraction is async-based, which I think is a mistake. One of L4's innovations was to use sync IPC, which offers the highest possible performance and gets the kernel out of the business of queueing. You can always build async ipc on top of sync without loss of performance; this puts the queues in user-space which is where they belong (they're easier to account for this way). I asked the HelenOS people why they decided to go this way, and this is their response (one of their points was "the L4's focus on performance is not that important these days", which I disagree with). http://www.mail-archive.com/helenos-devel@lists.modry.cz/msg00338.html http://www.mail-archive.com/helenos-devel@lists.modry.cz/msg...
- derleth 14y ago
  > Someday, hopefully not too far in the future, someone is going to write a modern, practical, and free microkernel-based OS that implements all of POSIX with performance the rivals Linux, but that offers a level of isolation and modularity that Linux could never offer. Why, when we already have VMs like Xen which are modern, practical, free, host guest OSes that run at full speed to provide whatever API the applications want, and provide a level of isolation and modularity microkernels never could?
  - haberman 14y ago
_delirium 14y ago
One thing that's interesting is that, while microkernels haven't really won, Xen-style paravirtualization, which in a way is an even more radical version of the de-monolithization idea [1], has taken over in a lot of niches, despite performance being an even bigger problem, especially without hardware assistance. [1] This paper, for example, argues that VMMs are "microkernels done right": http://static.usenix.org/event/hotos05/final_papers/full_papers/hand/hand_html/ http://static.usenix.org/event/hotos05/final_papers/full_pap...
jacquesm 14y ago
> I don't think the fundamental problem with them — performance is horrible — had been made obvious yet. Microkernels can be just as fast as macrokernels and other than raw throughput can give latency guarantees that a macrokernel can't. Message passing overhead can be minimized (by using the paging mechanism for passing messages between tasks on the same node).
tzs 14y ago
At the time Linus was ranting about micro kernels, QNX, a micro kernel, soundly kicked the Linux kernel's butt in pretty much every way.
- philh 14y ago
  That's (weak) evidence that macrokernels are superior. QNX had a head start, but did not maintain the lead.
0x0 14y ago
Mac OS X' kernel is Xnu, http://en.wikipedia.org/wiki/Xnu http://en.wikipedia.org/wiki/Xnu , which is based on the Mach microkernel but with pieces and bits from FreeBSD bolted on, for a hybrid approach.

rogerbinns 14y ago

They emphasised architectural elegance over short term practicality - ie do it right rather than do it quickly. The problem with operating systems is that you need to have a minimum set of working functionality in order for it to be somewhat useful to people not developing the operating system itself. You need a way of storing data, you need a reasonable number of commands to run, you need editors etc. On the hardware side there was immense turnover in the late eighties and nineties. Processor architectures soared and waned, it wasn't uncommon for cpu speeds to double, memory become increasingly cheaper, peripherals changed (mice, cdroms, floppies changed size, Zip drives, tape drives), popular hardware buses changed (ISA 8/16 bit, ADB, VLB, PCI, PCMCIA, USB, serial, parallel). Just keeping up without changing your operating system core functionality was a lot of work! The GNU project tended to work very cathedral like, while Linux was very bazaar like. The latter meant people could support and update the kernel for their own devices ("crowdsourcing" in today's terminology). http://www.catb.org/esr/writings/homesteading/cathedral-bazaar/ http://www.catb.org/esr/writings/homesteading/cathedral-baza... To give you an idea of how many shortcuts Linux took, the way the original kernel worked was that it used a single large address space. Each process was slotted into 64MB of that address space. That left you with a maximum of 63 processes. A context switch just involved changing protections to enable the appropriate slot. This is far more lightweight than how Linux now and other operating systems do it with no such process limits and a far more heavyweight full address space switch. Back then it made Linux really fast at process switching. This is an example of "worse is better": http://en.wikipedia.org/wiki/Worse_is_better http://en.wikipedia.org/wiki/Worse_is_better What the detractors didn't realise is that the initial simplistic Linux implementation could evolve away the constraints, and were instead betting on up front "intelligent design".

zby 14y ago
Isn't that ironical when you consider that user space hardware controllers mean much less coordination needed - i.e. be much more bazaar style?

jacquesm 14y ago

Not only was it perfectly possible, microkernels have been sold commercially for decades and are extremely popular in hard real time environments where you want to be able to pre-empt a driver as though it is just another user process (which they are!).

qxcv 14y ago

This XKCD[0] explains the issue with Hurd perfectly, more-or-less echoing the ideas rogerbinns expressed. Note the mouseover text, which makes the observation that the Hurd loop is the loop in which you "code right" and the Android loop is the loop in which you "code fast". [0] https://xkcd.com/844/ https://xkcd.com/844/

frozenport 14y ago

Linux had a working version when the FSF had a longterm vision. There is nothing impossible in their design and more complex systems such as Windows or Inferno/Plan9 exist. In the case of Windows imagine how many hours were spent in its design. When Linux came out it worked and everybody was shocked, Hurd was never able to do this.