3 ms·

Shouldn't the microkernel architecture be capable of having fewer barriers in the fast path? e.g. you could have hardware like a NIC or GPU be capable of partit

Shouldn't the microkernel architecture be capable of having fewer barriers in the fast path? e.g. you could have hardware like a NIC or GPU be capable of partitioning address spaces to have per-program command/response queues. Then the kernel asks on an administrative queue for a program's IO queue to be created, configures the memory controller (assuming an IOMMU) for the program to have those queues in its address space, and context switches to the program. While the program runs, there are zero barriers and it can simply talk to the hardware directly by writing into pcie space. Hardware knows what queues commands are coming from and can enforce whatever resource partitioning within firmware. My understanding is that (perhaps except for virtualization/address space partitioning on the hardware device to be able to use it with many programs at once) this is approximately how/why user-mode networking is used for high performance today?

immibis 2y ago
Monolithic kernels also have these fast paths. It's how GPU drivers work under Linux - your process submits command buffers directly to the GPU.
foota 2y ago
That all requires a lot of hardware support that probably didn't exist at the time, and is probably still esoteric today.
nostrademons 2y ago
This is the basic idea behind the exokernel concept: have the kernel's only job be to enforce protection boundaries between processes, and then expose an interface to userspace that is identical to what the hardware exposes (eg. directly write to buffers and queues): https://en.wikipedia.org/wiki/Exokernel https://en.wikipedia.org/wiki/Exokernel No production exokernels were built, but the concept lives on as virtualization. Most hypervisors (eg. Xen) work on a substantially similar principal: the hypervisor is responsible for securely multiplexing the hardware between guest OSes, and then the guest OS talks to an interface that basically looks like the hardware. A step beyond this is to get rid of the guest OS and have application code talk directly to the hardware interface. This is a unikernel (or called a "libOS" in the original exokernel papers), and as you predict, unikernels running on a hypervisor are often faster than traditional monolithic kernel OSes. The main thing blocking their adoption is that they require significant rewrites to application-level code: most applications (and the libraries they depend on) generally assume POSIX syscalls are available and would need to be largely rewritten to work in terms of direct hardware access. You can rewrite POSIX as a library in userspace (this was the approach taken by the original exokernel research), but most of the performance gains come from bypassing inappropriate abstractions, so if you use the same abstractions you get basically the same performance. https://en.wikipedia.org/wiki/Unikernel https://en.wikipedia.org/wiki/Unikernel

immibis 2y ago

Monolithic kernels also have these fast paths. It's how GPU drivers work under Linux - your process submits command buffers directly to the GPU.

foota 2y ago

That all requires a lot of hardware support that probably didn't exist at the time, and is probably still esoteric today.

nostrademons 2y ago

This is the basic idea behind the exokernel concept: have the kernel's only job be to enforce protection boundaries between processes, and then expose an interface to userspace that is identical to what the hardware exposes (eg. directly write to buffers and queues): https://en.wikipedia.org/wiki/Exokernel https://en.wikipedia.org/wiki/Exokernel No production exokernels were built, but the concept lives on as virtualization. Most hypervisors (eg. Xen) work on a substantially similar principal: the hypervisor is responsible for securely multiplexing the hardware between guest OSes, and then the guest OS talks to an interface that basically looks like the hardware. A step beyond this is to get rid of the guest OS and have application code talk directly to the hardware interface. This is a unikernel (or called a "libOS" in the original exokernel papers), and as you predict, unikernels running on a hypervisor are often faster than traditional monolithic kernel OSes. The main thing blocking their adoption is that they require significant rewrites to application-level code: most applications (and the libraries they depend on) generally assume POSIX syscalls are available and would need to be largely rewritten to work in terms of direct hardware access. You can rewrite POSIX as a library in userspace (this was the approach taken by the original exokernel research), but most of the performance gains come from bypassing inappropriate abstractions, so if you use the same abstractions you get basically the same performance. https://en.wikipedia.org/wiki/Unikernel https://en.wikipedia.org/wiki/Unikernel