5 ms·

Good to see multiserver microkernels back on HN. Have significant advances been made to make message passing better in modern CPUs?

snvzz 9y ago
If you're worried about performance, see http://blog.darknedgy.net/technology/2016/01/01/0/ http://blog.darknedgy.net/technology/2016/01/01/0/
- zerebubuth 9y ago
  An example from the linked blog post: > It strikes me that anti-microkernel sentiment most vocally originates as a sort of tribal affiliation mechanism by Linux users to ward off insecurity. I think the post would be a better reference and stand a greater chance of correcting common misconceptions if it didn't include so many ad-hominem attacks against those who don't agree with the post's author. As-is, it just appears to be a continuation of a long-running flame-war.
  - snvzz 9y ago
    I agree. Unfortunately, I am not aware of a compilation of misconceptions on microkernels article that's more valuable than this one.
    - naasking 9y ago
      It seems pretty trivial: every hypervisor and/or virtualized system is running a type of microkernel. If it's fast enough for the cloud, it's fast enough period.
      - sitkack 9y ago
        I find it ironic that Linux itself is an application running underneath a microkernel.
        kazinator 9y ago
        Running Linux under something lower level is old hat, see RTLinux: http://www.yodaiken.com/papers/BarabanovThesis.pdf http://www.yodaiken.com/papers/BarabanovThesis.pdf [1997] Well, that's not exactly a microkernel, but took lots of inspiration from microkernels. I take it that you may be alluding to the debate between Torvalds and Tannenbaum regarding microkernels. https://en.wikipedia.org/wiki/Tanenbaum%E2%80%93Torvalds_debate https://en.wikipedia.org/wiki/Tanenbaum%E2%80%93Torvalds_deb...
        naasking 9y ago
        The first PowerPC linux for the Power Mac ran under the Mach microkernel [1]. Then there was also L4Linux, which I believe was the first low-overhead virtualization of Linux using the L4 microkernel. Papers demonstrated < 10% overhead I believe, where previous virtualization efforts were somewhat higher. I'd say it kicked off the whole paravirtualization/virtualization craze. [1] http://www.mklinux.org/ http://www.mklinux.org/
aidenn0 9y ago
There are, and always will be, performance disadvantages to a microkernel. However, for many many tasks the difference is negligible. X11 has IPC, this has performance overhead, but nobody seriously suggests that all X11 applications run in the same process. I work regularly with a microkernel that allows threads (including device driver threads) either in their own separate address space, or in the supervisor address space. The difference in debugability between "there was memory corruption in the kernel" and "there was memory corruptoin in this specific device driver" is huge. The throughput overhead of message passing versus not on a data-heavy driver tends to be ~20%, though just like IPC in userspace multiprocessing systems you can reduce that somewhat with shared memory, batching and other tricks if needed. But seriously when people worry about message-passing overhead, I feel like it's 1990 again and we are debating whether the context switch overhead of processes on UNIX is worth it. I'm not sure why the userspace battle was decided so long ago while the kernel debate still goes on. [edit] Also sometimes the natural design for a microkernel yields better performance than the natural design for a monlithic kernel. When you are pushing the envelope for performance, sometimes some very counterintuitive things can give you benefits. I have seen cases where a single copy was faster than zero-copy, and cases where the safety of no-shared memory allowed easier parallelization of the work.
- snvzz 9y ago
  >I feel like it's 1990 again and we are debating whether the context switch overhead of processes on UNIX is worth it. Worth noting are modern (gen2/3, L4 onwards) microkernel context switch costs. This article has a neat table in it. Third page. http://sigops.org/sosp/sosp13/papers/p133-elphinstone.pdf http://sigops.org/sosp/sosp13/papers/p133-elphinstone.pdf In contrast, Linux takes whole microseconds. This should be carefully considered when thinking about multiserver architecture overhead.
  - aidenn0 9y ago
    I'm guessing those are uncontended timings. I haven't been able to think of a good way to do it, but I'd like to see contended timings compared as well. Having a fast fast-path is great for many reasons (a fast fast-path will reduce contention all on its own, and systems can be designed to minimize contention), but the contended case can be much slower on some microkernels, and I have no idea what the overhead is on linux.
Rusky 9y ago
Message passing performance is a bit of a red herring. Performance has been within a few percent of monolithic kernels for a long time. More importantly, message passing is already in widespread use for things like virtualization or userspace IPC anyway. Aside from that not-entirely-reassuring "eh, it's fast enough" argument, I've seen arguments that microkernels could actually leapfrog monolithic kernels on multi-core systems, if they moved on from their obsession with single-core message passing latency. They'd no longer use the sequential dance of "enter kernel; switch processes; exit kernel; do work; enter kernel; switch back; exit kernel." Instead, they'd use "enter kernel; write message for the cache hierarchy to pass; schedule and exit kernel." No idea how that'd turn out in practice, and it would probably involve some tradeoffs, but the idea of just entirely bypassing the whole "syscall vs message round trip" thing is appealing. Exokernels are similarly appealing, by rearranging their protection mechanisms so that things can be moved to libraries rather than server processes.

snvzz 9y ago

If you're worried about performance, see http://blog.darknedgy.net/technology/2016/01/01/0/ http://blog.darknedgy.net/technology/2016/01/01/0/

zerebubuth 9y ago
An example from the linked blog post: > It strikes me that anti-microkernel sentiment most vocally originates as a sort of tribal affiliation mechanism by Linux users to ward off insecurity. I think the post would be a better reference and stand a greater chance of correcting common misconceptions if it didn't include so many ad-hominem attacks against those who don't agree with the post's author. As-is, it just appears to be a continuation of a long-running flame-war.
- snvzz 9y ago
  I agree. Unfortunately, I am not aware of a compilation of misconceptions on microkernels article that's more valuable than this one.
  - naasking 9y ago
    It seems pretty trivial: every hypervisor and/or virtualized system is running a type of microkernel. If it's fast enough for the cloud, it's fast enough period.
    - sitkack 9y ago
      I find it ironic that Linux itself is an application running underneath a microkernel.
      - kazinator 9y ago
        Running Linux under something lower level is old hat, see RTLinux: http://www.yodaiken.com/papers/BarabanovThesis.pdf http://www.yodaiken.com/papers/BarabanovThesis.pdf [1997] Well, that's not exactly a microkernel, but took lots of inspiration from microkernels. I take it that you may be alluding to the debate between Torvalds and Tannenbaum regarding microkernels. https://en.wikipedia.org/wiki/Tanenbaum%E2%80%93Torvalds_debate https://en.wikipedia.org/wiki/Tanenbaum%E2%80%93Torvalds_deb...
        naasking 9y ago
        The first PowerPC linux for the Power Mac ran under the Mach microkernel [1]. Then there was also L4Linux, which I believe was the first low-overhead virtualization of Linux using the L4 microkernel. Papers demonstrated < 10% overhead I believe, where previous virtualization efforts were somewhat higher. I'd say it kicked off the whole paravirtualization/virtualization craze. [1] http://www.mklinux.org/ http://www.mklinux.org/

aidenn0 9y ago

There are, and always will be, performance disadvantages to a microkernel. However, for many many tasks the difference is negligible. X11 has IPC, this has performance overhead, but nobody seriously suggests that all X11 applications run in the same process. I work regularly with a microkernel that allows threads (including device driver threads) either in their own separate address space, or in the supervisor address space. The difference in debugability between "there was memory corruption in the kernel" and "there was memory corruptoin in this specific device driver" is huge. The throughput overhead of message passing versus not on a data-heavy driver tends to be ~20%, though just like IPC in userspace multiprocessing systems you can reduce that somewhat with shared memory, batching and other tricks if needed. But seriously when people worry about message-passing overhead, I feel like it's 1990 again and we are debating whether the context switch overhead of processes on UNIX is worth it. I'm not sure why the userspace battle was decided so long ago while the kernel debate still goes on. [edit] Also sometimes the natural design for a microkernel yields better performance than the natural design for a monlithic kernel. When you are pushing the envelope for performance, sometimes some very counterintuitive things can give you benefits. I have seen cases where a single copy was faster than zero-copy, and cases where the safety of no-shared memory allowed easier parallelization of the work.

snvzz 9y ago
>I feel like it's 1990 again and we are debating whether the context switch overhead of processes on UNIX is worth it. Worth noting are modern (gen2/3, L4 onwards) microkernel context switch costs. This article has a neat table in it. Third page. http://sigops.org/sosp/sosp13/papers/p133-elphinstone.pdf http://sigops.org/sosp/sosp13/papers/p133-elphinstone.pdf In contrast, Linux takes whole microseconds. This should be carefully considered when thinking about multiserver architecture overhead.
- aidenn0 9y ago
  I'm guessing those are uncontended timings. I haven't been able to think of a good way to do it, but I'd like to see contended timings compared as well. Having a fast fast-path is great for many reasons (a fast fast-path will reduce contention all on its own, and systems can be designed to minimize contention), but the contended case can be much slower on some microkernels, and I have no idea what the overhead is on linux.

Rusky 9y ago

Message passing performance is a bit of a red herring. Performance has been within a few percent of monolithic kernels for a long time. More importantly, message passing is already in widespread use for things like virtualization or userspace IPC anyway. Aside from that not-entirely-reassuring "eh, it's fast enough" argument, I've seen arguments that microkernels could actually leapfrog monolithic kernels on multi-core systems, if they moved on from their obsession with single-core message passing latency. They'd no longer use the sequential dance of "enter kernel; switch processes; exit kernel; do work; enter kernel; switch back; exit kernel." Instead, they'd use "enter kernel; write message for the cache hierarchy to pass; schedule and exit kernel." No idea how that'd turn out in practice, and it would probably involve some tradeoffs, but the idea of just entirely bypassing the whole "syscall vs message round trip" thing is appealing. Exokernels are similarly appealing, by rearranging their protection mechanisms so that things can be moved to libraries rather than server processes.