6 ms·

This really has nothing to do with the mono vs. micro kernel argument. Also micro kernels were very popular in the research community at the time that Linux was

This really has nothing to do with the mono vs. micro kernel argument. Also micro kernels were very popular in the research community at the time that Linux was in its infancy, mach and minix are both examples of this. Linux did win on merit and the future work of micro kernels was forked and past micro kernels are only vestigially micro. Edit: Not sure why this is being downvoted, micro kernels are the antithesis of what eBPF is trying to accomplish. I say this as a university researcher that has built and worked on many kernels with different tradeoffs. If you disagree please explain in a comment. Micro kernels increase security and decrease complexity by running a very small resource manager in the processor's highest privilege level. eBPF runs code at the highest privilege level by actually extending the kernel, it actually supplants doing kernel work in userspace. It greatly increases kernel complexity with an new subsystem and decreases security in order to allow extensibility in the face of a hard to change interface (syscalls in a mature operating systems are difficult to change).

rektide 4y ago
Too easy a dismissal, doesn't reach to explore. This very much enables a much more interesting future of smaller interoperating & pieces. For example the DPDK work created a small alternative micro-monolith for high speed networking. If someone needs to make a new fast protocol like quic or something further afield conventionally they might have split between building a similar application-owning hardware view, building a kernel module, or facing extreme performance loss. Io-uring opens up much greater fields of extensibility by removing huge barriers, allowing work to spread out far wider than it could have before. I agree on the surface this seems dissimilar. But architecturally what is enabled is a far greater range of considerations for how we might interlace responsibilities on systems. Io-uring on the surface is for apps to commuicate with the kernel, but if systems like fuse or uhid or other user land devices also prosper, it will disrupt the monolith's lock.
- tptacek 4y ago
  Isn't DPDK sort of an exception that proves the rule? Almost nothing gets implemented in DPDK precisely because pulling the TCP/IP stack out of the kernel is a huge logistical pain in the butt for development, and far more energy is put into making the kernel's TCP/IP interface more useful for developers. (I'm not pushing back on you, just exploring the space, so to speak.) I'm not sure I see how eBPF is moving us away from monolithic OS's; the whole point of eBPF is running more stuff inside the monolithic kernel. :)
  - rektide 4y ago
    Good point about eBPF. :) It is running in kernel space, but it's not code from the kernel, it's user code, which expands the model, that was kind of my thought. DPDK did seem to be limited to pretty niche/exceptional applications. There's like a really fast Open Vswitch backend (and some other very network-centric projects). Practically a partial-unikernel system... trying to almost never ever use the kernel, just stay in user land with your own network stack forever. Just... In Linux.
    - softirq 4y ago
      It's still running code inside of the kernel. The point of a microkernel is to run as little in ring 0 as possible and have a very thin resource manger that can talk to system servers. The point of eBPF is to move even more stuff dynamically into the kernel to avoid expanding the kernel's static interfaces (syscalls).
      - rektide 4y ago
        Which brings us back to my previous point, > The debate being anchored to either/or certainly didn't help, confined the dimensionality of what we might consider, eliminated vaster exploration of the possible. eBPF doesn't seem to fit squarely in either mold. It has some characteristics of monolithic kernels, in that code runs in ring 0. But it's also now it's not the kernel that's running there. It's user code. Which doesn't fit either pattern. I do think io_uring could potentially spawn more microkernel-y outcomes. I don't know how exactly eBPF comes into the picture but it's another tool. Just to whip up a nonsense example, maybe we use eBPF to build a secure network switch in ring0 that io_uring based processes communicate to each other with. Io_uring is still kernel based, isn't processes directly communicating with each other (not sure how many microkernels could do that), but again my point is less super specifically about microservices & more that this is a rebalancing possibility, where the kernel code might become less the focus of the system, where other non-kernel-code platform substrates have a higher chance of emerging/performing well. There's more possible than monolithic chunks of code allowed. What new dimensions we might find out there excites me.
kqr 4y ago
I appreciate what you're saying, but I feel like your argument (including downthread) is something like saying "fuel injectors have nothing to do with carburetors" or "vacuum tubes have nothing to do with transistors". It's true, but the user may end up solving similar problems with them.
- rektide 4y ago
  Heeyyyy much more cleanly said than my attempts. Shit yeah nicely made.
hinkley 4y ago
One of the bigger shortcomings of microkernels is the IPC delay. io_uring amortizes those delays, making it less painful to implement a chatty protocol, like the conversation between ukernel processes. EBNF is sending code instead of data, which is another common way to solve IPC problems, but with different trade offs. Being harder to prove is an important one.
- KMag 4y ago
  Presumably you mean eBPF, not EBNF.
never_inline 4y ago
> eBPF runs code at the highest privilege level by actually extending the kernel Yeah this was my thought when reading the above argument too. Maybe they're thinking that in future, more and more kernel functionality will be moved to eBPF programs, and eBPF has a restricted execution environment? I can follow both lines of logic.
- deleted 3y ago
  [deleted]

rektide 4y ago

Too easy a dismissal, doesn't reach to explore. This very much enables a much more interesting future of smaller interoperating & pieces. For example the DPDK work created a small alternative micro-monolith for high speed networking. If someone needs to make a new fast protocol like quic or something further afield conventionally they might have split between building a similar application-owning hardware view, building a kernel module, or facing extreme performance loss. Io-uring opens up much greater fields of extensibility by removing huge barriers, allowing work to spread out far wider than it could have before. I agree on the surface this seems dissimilar. But architecturally what is enabled is a far greater range of considerations for how we might interlace responsibilities on systems. Io-uring on the surface is for apps to commuicate with the kernel, but if systems like fuse or uhid or other user land devices also prosper, it will disrupt the monolith's lock.

tptacek 4y ago
Isn't DPDK sort of an exception that proves the rule? Almost nothing gets implemented in DPDK precisely because pulling the TCP/IP stack out of the kernel is a huge logistical pain in the butt for development, and far more energy is put into making the kernel's TCP/IP interface more useful for developers. (I'm not pushing back on you, just exploring the space, so to speak.) I'm not sure I see how eBPF is moving us away from monolithic OS's; the whole point of eBPF is running more stuff inside the monolithic kernel. :)
- rektide 4y ago
  Good point about eBPF. :) It is running in kernel space, but it's not code from the kernel, it's user code, which expands the model, that was kind of my thought. DPDK did seem to be limited to pretty niche/exceptional applications. There's like a really fast Open Vswitch backend (and some other very network-centric projects). Practically a partial-unikernel system... trying to almost never ever use the kernel, just stay in user land with your own network stack forever. Just... In Linux.
  - softirq 4y ago
    It's still running code inside of the kernel. The point of a microkernel is to run as little in ring 0 as possible and have a very thin resource manger that can talk to system servers. The point of eBPF is to move even more stuff dynamically into the kernel to avoid expanding the kernel's static interfaces (syscalls).
    - rektide 4y ago
      Which brings us back to my previous point, > The debate being anchored to either/or certainly didn't help, confined the dimensionality of what we might consider, eliminated vaster exploration of the possible. eBPF doesn't seem to fit squarely in either mold. It has some characteristics of monolithic kernels, in that code runs in ring 0. But it's also now it's not the kernel that's running there. It's user code. Which doesn't fit either pattern. I do think io_uring could potentially spawn more microkernel-y outcomes. I don't know how exactly eBPF comes into the picture but it's another tool. Just to whip up a nonsense example, maybe we use eBPF to build a secure network switch in ring0 that io_uring based processes communicate to each other with. Io_uring is still kernel based, isn't processes directly communicating with each other (not sure how many microkernels could do that), but again my point is less super specifically about microservices & more that this is a rebalancing possibility, where the kernel code might become less the focus of the system, where other non-kernel-code platform substrates have a higher chance of emerging/performing well. There's more possible than monolithic chunks of code allowed. What new dimensions we might find out there excites me.

kqr 4y ago

I appreciate what you're saying, but I feel like your argument (including downthread) is something like saying "fuel injectors have nothing to do with carburetors" or "vacuum tubes have nothing to do with transistors". It's true, but the user may end up solving similar problems with them.

rektide 4y ago
Heeyyyy much more cleanly said than my attempts. Shit yeah nicely made.

hinkley 4y ago

One of the bigger shortcomings of microkernels is the IPC delay. io_uring amortizes those delays, making it less painful to implement a chatty protocol, like the conversation between ukernel processes. EBNF is sending code instead of data, which is another common way to solve IPC problems, but with different trade offs. Being harder to prove is an important one.

KMag 4y ago
Presumably you mean eBPF, not EBNF.

never_inline 4y ago

> eBPF runs code at the highest privilege level by actually extending the kernel Yeah this was my thought when reading the above argument too. Maybe they're thinking that in future, more and more kernel functionality will be moved to eBPF programs, and eBPF has a restricted execution environment? I can follow both lines of logic.

deleted 3y ago
[deleted]