4 ms·

The intention here is to use wasm to allow you safely run user code _within_ the kernel. Their primary targets are nginx and FUSE. Conceivably, avoiding the con

by rusbus 8y ago

The intention here is to use wasm to allow you safely run user code _within_ the kernel. Their primary targets are nginx and FUSE. Conceivably, avoiding the context switch into and out of the kernel will have significant performance implications, but there aren't any numbers out yet for nginx specifically.

titzer 8y ago
Don't do this. https://news.ycombinator.com/item?id=18587353 https://news.ycombinator.com/item?id=18587353
- detaro 8y ago
  While I'm far from convinced this is a useful thing: nginx is "untrusted code"?
  - titzer 8y ago
    User code is generally not trusted with kernel privileges, no.
  - nickpsecurity 8y ago
    Anything receiving potentially malicious input should be untrusted and sandboxed if possible. That includes the network stack itself in high-assurance, security products. We also prefer simple, rigorously-analyzed software with high predictability. Other stuff often has vulnerabilities. Nginx is nearly 200,000 lines of code per an interview with CEO I just skimmed. Lwan, made for security and maintainability, is about 10,000 lines of code in comparison: https://lwan.ws/ https://lwan.ws/ Lwan's actually small enough that mathematical verification for correctness against a spec is feasible, even though costly. Unlike Lwan, I could never have any hope of proving the correctness of Nginx. Even its safety would be difficult just because of all the potential code interactions on malicious input. Leak-free for secrets it contains? Forget about it. Best bet is to shove that thing either in a partition on a separation kernel/VMM or on a dedicated machine. The automated tooling for large programs does get better every year, though. One can use any compatible with Nginx. And still shove that humongous server into a deprivileged partition just in case. ;) https://os.inf.tu-dresden.de/papers_ps/nizza.pdf https://os.inf.tu-dresden.de/papers_ps/nizza.pdf https://genode.org/ https://genode.org/ https://www.ghs.com/products/safety_critical/integrity-do-178b.html https://www.ghs.com/products/safety_critical/integrity-do-17...
simmons 8y ago
That's certainly a fascinating idea. My initial thought was "Wait, doesn't the kernel already provide a sandboxed execution environment -- called userspace?" This would still have scheduling overhead, but I assume the idea is to avoid a lot of the other context switching steps such as switching page tables. And instead rely on Wasm/JIT checks to ensure ahead of time that memory violations won't happen.
- wahern 8y ago
  Once upon a time syscalls were slow, but architectures now provide features like syscall/sysenter for switching privilege levels, with costs comparable to userspace function calls. Once upon a time switching page tables was slow, but now we have features like PCID that allow preserving buffers. Soon, if not already, the principle cost to context switching will be the necessity to flush prediction and data buffers. In-kernel solutions like Wasmjit must incur the same costs. Quite possibly they may turn out to be slower overall: 1) they won't be able to take advantage of the same hardware optimized privilege management facilities (existing and future ones--imagine tagged prediction buffers much like PCID), and 2) they still incur the extra runtime overhead of running in a VM which, JIT-optimized or not, eats into limited resources like those prediction and data buffers that have become so critical to maximizing performance. Granted, if it's going to work well at all than Nginx seems like a good bet, especially because of I/O. But there are many other solutions to that problem. Obsession with DPDK may be waning, but zero-copy AIO is still a thing and there are more ergonomic userspace alternatives (existing and in the pipeline) that let you leverage the in-kernel network stack without having to incur copying costs. And then there are solutions like QUIC that redefine the problem and which should work extremely well with existing zero-copy interfaces. CPUs are incredibly complex precisely because so much of the security heavy-lifting once performed in the OS is being accomplished in the CPU or dedicated controllers. And these newer optimizations were designed to be integrated within the context of the traditional userspace/kernel split. Wasmjit looks like an extremely cool project and I don't doubt its utility. There's plenty of room for alternative approaches, I just don't think the value-add is all that obvious.[1] Probably less to do with performance and more to do with providing a clear, stable, well-supported environment for solving (and subsequently maintaining!) difficult integration problems. [1] I just want to reiterate that by saying the value-add isn't obvious I'm not implying anything about the potential magnitude of that value-add. I've been around long enough to understand that most pain points are invisible and just because I can't see them or people can't articulate them doesn't mean they don't exist or that the potential for serious disruption isn't there.
amluto 8y ago
Last time I measured this, the time it took the Linux scheduler to decide what task to schedule was far more than the time it took the entry code and CPU to switch from user to kernel or vice verse. Meltdown changes this, but Meltdown-proof AMD CPUs are all over and Meltdown-proof Intel CPUs should show up eventually. So I don’t see the point.
Ajedi32 8y ago
One step closer to METAL[1] [1]: https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death... (at 18:46)
- ksec 8y ago
  Everything said in the talk went true. Which means we are very close to a Nuclear War. ( And it certainly looks like a possibility at the way things are going )
2trill2spill 8y ago
Use Sendfile and if that's not enough try netmap or ddpk, why would I want to run Nginx in kernel space?

by rusbus 8y ago

titzer 8y ago

Don't do this. https://news.ycombinator.com/item?id=18587353 https://news.ycombinator.com/item?id=18587353

detaro 8y ago
While I'm far from convinced this is a useful thing: nginx is "untrusted code"?
- titzer 8y ago
  User code is generally not trusted with kernel privileges, no.
- nickpsecurity 8y ago
  Anything receiving potentially malicious input should be untrusted and sandboxed if possible. That includes the network stack itself in high-assurance, security products. We also prefer simple, rigorously-analyzed software with high predictability. Other stuff often has vulnerabilities. Nginx is nearly 200,000 lines of code per an interview with CEO I just skimmed. Lwan, made for security and maintainability, is about 10,000 lines of code in comparison: https://lwan.ws/ https://lwan.ws/ Lwan's actually small enough that mathematical verification for correctness against a spec is feasible, even though costly. Unlike Lwan, I could never have any hope of proving the correctness of Nginx. Even its safety would be difficult just because of all the potential code interactions on malicious input. Leak-free for secrets it contains? Forget about it. Best bet is to shove that thing either in a partition on a separation kernel/VMM or on a dedicated machine. The automated tooling for large programs does get better every year, though. One can use any compatible with Nginx. And still shove that humongous server into a deprivileged partition just in case. ;) https://os.inf.tu-dresden.de/papers_ps/nizza.pdf https://os.inf.tu-dresden.de/papers_ps/nizza.pdf https://genode.org/ https://genode.org/ https://www.ghs.com/products/safety_critical/integrity-do-178b.html https://www.ghs.com/products/safety_critical/integrity-do-17...

simmons 8y ago

That's certainly a fascinating idea. My initial thought was "Wait, doesn't the kernel already provide a sandboxed execution environment -- called userspace?" This would still have scheduling overhead, but I assume the idea is to avoid a lot of the other context switching steps such as switching page tables. And instead rely on Wasm/JIT checks to ensure ahead of time that memory violations won't happen.

wahern 8y ago
Once upon a time syscalls were slow, but architectures now provide features like syscall/sysenter for switching privilege levels, with costs comparable to userspace function calls. Once upon a time switching page tables was slow, but now we have features like PCID that allow preserving buffers. Soon, if not already, the principle cost to context switching will be the necessity to flush prediction and data buffers. In-kernel solutions like Wasmjit must incur the same costs. Quite possibly they may turn out to be slower overall: 1) they won't be able to take advantage of the same hardware optimized privilege management facilities (existing and future ones--imagine tagged prediction buffers much like PCID), and 2) they still incur the extra runtime overhead of running in a VM which, JIT-optimized or not, eats into limited resources like those prediction and data buffers that have become so critical to maximizing performance. Granted, if it's going to work well at all than Nginx seems like a good bet, especially because of I/O. But there are many other solutions to that problem. Obsession with DPDK may be waning, but zero-copy AIO is still a thing and there are more ergonomic userspace alternatives (existing and in the pipeline) that let you leverage the in-kernel network stack without having to incur copying costs. And then there are solutions like QUIC that redefine the problem and which should work extremely well with existing zero-copy interfaces. CPUs are incredibly complex precisely because so much of the security heavy-lifting once performed in the OS is being accomplished in the CPU or dedicated controllers. And these newer optimizations were designed to be integrated within the context of the traditional userspace/kernel split. Wasmjit looks like an extremely cool project and I don't doubt its utility. There's plenty of room for alternative approaches, I just don't think the value-add is all that obvious.[1] Probably less to do with performance and more to do with providing a clear, stable, well-supported environment for solving (and subsequently maintaining!) difficult integration problems. [1] I just want to reiterate that by saying the value-add isn't obvious I'm not implying anything about the potential magnitude of that value-add. I've been around long enough to understand that most pain points are invisible and just because I can't see them or people can't articulate them doesn't mean they don't exist or that the potential for serious disruption isn't there.

amluto 8y ago

Last time I measured this, the time it took the Linux scheduler to decide what task to schedule was far more than the time it took the entry code and CPU to switch from user to kernel or vice verse. Meltdown changes this, but Meltdown-proof AMD CPUs are all over and Meltdown-proof Intel CPUs should show up eventually. So I don’t see the point.

Ajedi32 8y ago

One step closer to METAL[1] [1]: https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death... (at 18:46)

ksec 8y ago
Everything said in the talk went true. Which means we are very close to a Nuclear War. ( And it certainly looks like a possibility at the way things are going )

2trill2spill 8y ago

Use Sendfile and if that's not enough try netmap or ddpk, why would I want to run Nginx in kernel space?