14 ms·

Nginx on Wasmjit

272 points by wofo 8y ago

rusbus 8y ago
The intention here is to use wasm to allow you safely run user code _within_ the kernel. Their primary targets are nginx and FUSE. Conceivably, avoiding the context switch into and out of the kernel will have significant performance implications, but there aren't any numbers out yet for nginx specifically.
- titzer 8y ago
  Don't do this. https://news.ycombinator.com/item?id=18587353 https://news.ycombinator.com/item?id=18587353
  - detaro 8y ago
    While I'm far from convinced this is a useful thing: nginx is "untrusted code"?
    - titzer 8y ago
      User code is generally not trusted with kernel privileges, no.
    - nickpsecurity 8y ago
      Anything receiving potentially malicious input should be untrusted and sandboxed if possible. That includes the network stack itself in high-assurance, security products. We also prefer simple, rigorously-analyzed software with high predictability. Other stuff often has vulnerabilities. Nginx is nearly 200,000 lines of code per an interview with CEO I just skimmed. Lwan, made for security and maintainability, is about 10,000 lines of code in comparison: https://lwan.ws/ https://lwan.ws/ Lwan's actually small enough that mathematical verification for correctness against a spec is feasible, even though costly. Unlike Lwan, I could never have any hope of proving the correctness of Nginx. Even its safety would be difficult just because of all the potential code interactions on malicious input. Leak-free for secrets it contains? Forget about it. Best bet is to shove that thing either in a partition on a separation kernel/VMM or on a dedicated machine. The automated tooling for large programs does get better every year, though. One can use any compatible with Nginx. And still shove that humongous server into a deprivileged partition just in case. ;) https://os.inf.tu-dresden.de/papers_ps/nizza.pdf https://os.inf.tu-dresden.de/papers_ps/nizza.pdf https://genode.org/ https://genode.org/ https://www.ghs.com/products/safety_critical/integrity-do-178b.html https://www.ghs.com/products/safety_critical/integrity-do-17...
- simmons 8y ago
  That's certainly a fascinating idea. My initial thought was "Wait, doesn't the kernel already provide a sandboxed execution environment -- called userspace?" This would still have scheduling overhead, but I assume the idea is to avoid a lot of the other context switching steps such as switching page tables. And instead rely on Wasm/JIT checks to ensure ahead of time that memory violations won't happen.
  - wahern 8y ago
    Once upon a time syscalls were slow, but architectures now provide features like syscall/sysenter for switching privilege levels, with costs comparable to userspace function calls. Once upon a time switching page tables was slow, but now we have features like PCID that allow preserving buffers. Soon, if not already, the principle cost to context switching will be the necessity to flush prediction and data buffers. In-kernel solutions like Wasmjit must incur the same costs. Quite possibly they may turn out to be slower overall: 1) they won't be able to take advantage of the same hardware optimized privilege management facilities (existing and future ones--imagine tagged prediction buffers much like PCID), and 2) they still incur the extra runtime overhead of running in a VM which, JIT-optimized or not, eats into limited resources like those prediction and data buffers that have become so critical to maximizing performance. Granted, if it's going to work well at all than Nginx seems like a good bet, especially because of I/O. But there are many other solutions to that problem. Obsession with DPDK may be waning, but zero-copy AIO is still a thing and there are more ergonomic userspace alternatives (existing and in the pipeline) that let you leverage the in-kernel network stack without having to incur copying costs. And then there are solutions like QUIC that redefine the problem and which should work extremely well with existing zero-copy interfaces. CPUs are incredibly complex precisely because so much of the security heavy-lifting once performed in the OS is being accomplished in the CPU or dedicated controllers. And these newer optimizations were designed to be integrated within the context of the traditional userspace/kernel split. Wasmjit looks like an extremely cool project and I don't doubt its utility. There's plenty of room for alternative approaches, I just don't think the value-add is all that obvious.[1] Probably less to do with performance and more to do with providing a clear, stable, well-supported environment for solving (and subsequently maintaining!) difficult integration problems. [1] I just want to reiterate that by saying the value-add isn't obvious I'm not implying anything about the potential magnitude of that value-add. I've been around long enough to understand that most pain points are invisible and just because I can't see them or people can't articulate them doesn't mean they don't exist or that the potential for serious disruption isn't there.
- amluto 8y ago
  Last time I measured this, the time it took the Linux scheduler to decide what task to schedule was far more than the time it took the entry code and CPU to switch from user to kernel or vice verse. Meltdown changes this, but Meltdown-proof AMD CPUs are all over and Meltdown-proof Intel CPUs should show up eventually. So I don’t see the point.
- Ajedi32 8y ago
  One step closer to METAL[1] [1]: https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death... (at 18:46)
  - ksec 8y ago
    Everything said in the talk went true. Which means we are very close to a Nuclear War. ( And it certainly looks like a possibility at the way things are going )
- 2trill2spill 8y ago
  Use Sendfile and if that's not enough try netmap or ddpk, why would I want to run Nginx in kernel space?
drzaiusx11 8y ago
It seems like wasm will finally enable the "write once, run everywhere" promise that java made but never truly executed by starting with the premise that you don't need "one true language" (java), but rather just the VM. Yeah, I know the JVM supports several languages these days but most require non-superficial similarities to Java (garbage collected, etc.)
- kodablah 8y ago
  > but most require non-superficial similarities to Java (garbage collected, etc.) It should be noted that doing this requires non-superficial similarities to *nix/POSIX (signals, files, threads, etc). It's not like you could run Nginx on this without its POSIX impl or in the browser w/out Emscripten's POSIX impl or on any other WASM runtime w/out a POSIX impl.
- chrisseaton 8y ago
  > most require non-superficial similarities to Java (garbage collected, etc.) You can (obviously) implement any language without garbage collection semantics using garbage collection, so this requirement is false. See for example languages like C and C++ running on the JVM.
  - drzaiusx11 8y ago
    Has this been done without jumping through hoops like compiling to a specific CPU target with GCC then doing a binary conversion to JVM bytecode (NestedVM does this via MIPS I believe)?
    - pjmlp 8y ago
      GraalVM understands LLVM bitcode. https://www.graalvm.org/docs/reference-manual/languages/llvm/ https://www.graalvm.org/docs/reference-manual/languages/llvm...
      - drzaiusx11 8y ago
        GraavlVM Seems like an incredibly useful tool for sandboxing non-java languages with java/jvm interoperability. LLVM really is enabling a renaissance of language interop. This is fantastic stuff.
lachlan-sneff 8y ago
Wasmer just got nginx working on their wasm runtime as well!
gok 8y ago
Sounds like the big idea from Singularity (run everything in the same hardware security ring, use static analysis for memory safety) is going mainstream, incrementally. Unfortunate that this is happening right as it becomes clear that such a design is fundamentally unsound on modern hardware.
- yolo42 8y ago
  There is another layer of abstraction that is heavily being exploited of running layers of kernels. I'm really interested in seeing more advances in this and unikernal approach where if you are running in the cloud, the hypervisor already provides you with a sandbox, so run things at a lower level than necessary. There are certain secuirty challenges that we see because we keep thinking in terms of user and kernal space. If we try to narrow (and slowly remove) the line separating the spaces, we can address these problems in more efficient way than done today for sure.
  - bpye 8y ago
    gVisor is pretty interesting, it kinda takes this idea when you run it on KVM. https://github.com/google/gvisor https://github.com/google/gvisor
- _wmd 8y ago
  The big news this year was that the majority of hardware isolation designs in existence were fundamentally unsound, this is a software sandbox. Are you referring to the recent discovery of a new hardware bug class here?
  - stcredzero 8y ago
    So is the idea to take a page from the RISC and VLSI playbooks and outsource the isolation and security to the compiler/JIT VM?
  - gok 8y ago
    The big news this year was that you can't run code of different trust levels in the same address space.
traverseda 8y ago
Obligatory birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death... I suppose calling it METAL would have been too on-the-nose.
- molf 8y ago
  Not sure why you're downvoted, this is a very interesting talk.
  - traverseda 8y ago
    It's pretty relevant to the discussion of "why would you want to run wasm in the kernel", but I'm not too worried about the votes.
    - reitzensteinm 8y ago
      It's not, though... WebAssembly doesn't really have all that much to do with js, any more than Flash or Java plugins would if they ended up being standardized instead. Every time there's a wasm thread it gets posted, but it misses the point of the talk to suggest that wasm is the prediction bearing fruit. The talk is great, but I'd suggest that's the reason for the downvotes.
      - earenndil 8y ago
        > it misses the point of the talk to suggest that wasm is the prediction bearing fruit Does it? It seems that the talk has two main points: 1. Javascript succeeded because it was (at least initially) just good enough not to be completely unbearable, but bad enough that people ended up using it primarily as a target for other languages. 2. Ring 0 JIT can be 4% faster that normal binaries. WASM is primarily a target for other languages, and qualifies as a language that can theoretically be JITted 4% faster than native code can be run.
        8y ago
syrusakbary 8y ago
Hi HN! I'm Syrus, from the Wasmer team. We have been working in something similar, but with a special focus on maintainability and with bigger goals in mind: https://wasmer.io/ https://wasmer.io/ Here is the article about our journey on Running Nginx (which funnily enough we actually accomplished just before wasmjit): https://medium.com/@syrusakbary/running-nginx-with-webassembly-6353c02c08ac https://medium.com/@syrusakbary/running-nginx-with-webassemb...
- kodablah 8y ago
  > but much more maintainable. > we actually accomplished just before wasmjit > Wasmer is the first native WebAssembly runtime [...] Whoa there. I like both projects and respect competition as much as the next guy, but maintainability is subjective and being first is of little importance.
  - syrusakbary 8y ago
    True! This statements without analysis are wet paper. In the article I've linked there is a better analysis on why: 1. in wasmjit, the machine instructions are hardcoded into the runtime (this is like creating your own LLVM, by hand... and only available for x86) 2. it doesn't have a single test I was talking about my own experience here, because I tried to contribute to wasmjit before creating Wasmer... and was quite challenging! It might be useful to check how many people interacted with the code in each of this projects! ;) Also note the timeline that took for this projects to accomplish the same: wasmer (<2 months) wasmjit (6 months)
    - hangonhn 8y ago
      You may be correct on all those points but still come off as rude. And I think you're just trying to be helpful and steer people in the direction that you think is right. However, given the context of where you're doing it, it feels like someone is trying to ruin someone else's parade.
      - glibgil 8y ago
StellaQueen 8y ago
After leaving my previous job 12 months ago, i've had some good luck to learn about this website which was a life-saver for me... They offer jobs for which people can work online from their house. My latest paycheck after working for them for 4 months was for $4500... Amazing thing about is that the only thing required is simple typing skills and access to internet...Read all about it here... http://Careers7.com/ http://Careers7.com/
jeltz 8y ago
How do you avoid having to copy buffers from the kernel into WebAssembly? As far as I know WebAssembly does not provide a way to access memory outside the linear memory block, but maybe in this specific case (nginx) it is possible to have all syscalls write to a buffer allocated by WebAssembly.
zymhan 8y ago
Does anyone else look at this title and see a jumble of letters? I miss the days of pronounceable acronyms
- mxuribe 8y ago
  At a glance it actually looked like Dutch to me (with apologies to my Dutch buddies!)
- idle_zealot 8y ago
  "engine-ecks on whaz'm-jit" It's certainly pronounceable, though perhaps that pronunciation is non-obvious.
- deleted 8y ago
  [deleted]
coverband 8y ago
I might be getting too old, because I truly don't get the benefit of doing this... Nobody I know really cares about portability, and I don't see how running nginx with WASM is in any way better than running it directly on the system. Does anyone care to ELI5 for me?
- lwb 8y ago
  This is satire but still very interesting: https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death...
- gok 8y ago
  Running it in the kernel removes the overhead of context switching into the kernel for I/O (and other things), so it could be faster. Although as this post says, right now it's only running in user space.
sargun 8y ago
I don’t understand. How is this possible? The POSIX API is not implemented for WASM. Non web embedding have not yet been standardized, nor threads. How have they implemented this? Are they implementing a non standard embedding and pthreads? https://webassembly.org/docs/non-web/ https://webassembly.org/docs/non-web/

272 points by wofo 8y ago

The intention here is to use wasm to allow you safely run user code _within_ the kernel. Their primary targets are nginx and FUSE. Conceivably, avoiding the context switch into and out of the kernel will have significant performance implications, but there aren't any numbers out yet for nginx specifically.

titzer 8y ago
Don't do this. https://news.ycombinator.com/item?id=18587353 https://news.ycombinator.com/item?id=18587353
- detaro 8y ago
  While I'm far from convinced this is a useful thing: nginx is "untrusted code"?
  - titzer 8y ago
    User code is generally not trusted with kernel privileges, no.
  - nickpsecurity 8y ago
    Anything receiving potentially malicious input should be untrusted and sandboxed if possible. That includes the network stack itself in high-assurance, security products. We also prefer simple, rigorously-analyzed software with high predictability. Other stuff often has vulnerabilities. Nginx is nearly 200,000 lines of code per an interview with CEO I just skimmed. Lwan, made for security and maintainability, is about 10,000 lines of code in comparison: https://lwan.ws/ https://lwan.ws/ Lwan's actually small enough that mathematical verification for correctness against a spec is feasible, even though costly. Unlike Lwan, I could never have any hope of proving the correctness of Nginx. Even its safety would be difficult just because of all the potential code interactions on malicious input. Leak-free for secrets it contains? Forget about it. Best bet is to shove that thing either in a partition on a separation kernel/VMM or on a dedicated machine. The automated tooling for large programs does get better every year, though. One can use any compatible with Nginx. And still shove that humongous server into a deprivileged partition just in case. ;) https://os.inf.tu-dresden.de/papers_ps/nizza.pdf https://os.inf.tu-dresden.de/papers_ps/nizza.pdf https://genode.org/ https://genode.org/ https://www.ghs.com/products/safety_critical/integrity-do-178b.html https://www.ghs.com/products/safety_critical/integrity-do-17...
simmons 8y ago
That's certainly a fascinating idea. My initial thought was "Wait, doesn't the kernel already provide a sandboxed execution environment -- called userspace?" This would still have scheduling overhead, but I assume the idea is to avoid a lot of the other context switching steps such as switching page tables. And instead rely on Wasm/JIT checks to ensure ahead of time that memory violations won't happen.
- wahern 8y ago
  Once upon a time syscalls were slow, but architectures now provide features like syscall/sysenter for switching privilege levels, with costs comparable to userspace function calls. Once upon a time switching page tables was slow, but now we have features like PCID that allow preserving buffers. Soon, if not already, the principle cost to context switching will be the necessity to flush prediction and data buffers. In-kernel solutions like Wasmjit must incur the same costs. Quite possibly they may turn out to be slower overall: 1) they won't be able to take advantage of the same hardware optimized privilege management facilities (existing and future ones--imagine tagged prediction buffers much like PCID), and 2) they still incur the extra runtime overhead of running in a VM which, JIT-optimized or not, eats into limited resources like those prediction and data buffers that have become so critical to maximizing performance. Granted, if it's going to work well at all than Nginx seems like a good bet, especially because of I/O. But there are many other solutions to that problem. Obsession with DPDK may be waning, but zero-copy AIO is still a thing and there are more ergonomic userspace alternatives (existing and in the pipeline) that let you leverage the in-kernel network stack without having to incur copying costs. And then there are solutions like QUIC that redefine the problem and which should work extremely well with existing zero-copy interfaces. CPUs are incredibly complex precisely because so much of the security heavy-lifting once performed in the OS is being accomplished in the CPU or dedicated controllers. And these newer optimizations were designed to be integrated within the context of the traditional userspace/kernel split. Wasmjit looks like an extremely cool project and I don't doubt its utility. There's plenty of room for alternative approaches, I just don't think the value-add is all that obvious.[1] Probably less to do with performance and more to do with providing a clear, stable, well-supported environment for solving (and subsequently maintaining!) difficult integration problems. [1] I just want to reiterate that by saying the value-add isn't obvious I'm not implying anything about the potential magnitude of that value-add. I've been around long enough to understand that most pain points are invisible and just because I can't see them or people can't articulate them doesn't mean they don't exist or that the potential for serious disruption isn't there.
amluto 8y ago
Last time I measured this, the time it took the Linux scheduler to decide what task to schedule was far more than the time it took the entry code and CPU to switch from user to kernel or vice verse. Meltdown changes this, but Meltdown-proof AMD CPUs are all over and Meltdown-proof Intel CPUs should show up eventually. So I don’t see the point.
Ajedi32 8y ago
One step closer to METAL[1] [1]: https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death... (at 18:46)
- ksec 8y ago
  Everything said in the talk went true. Which means we are very close to a Nuclear War. ( And it certainly looks like a possibility at the way things are going )
2trill2spill 8y ago
Use Sendfile and if that's not enough try netmap or ddpk, why would I want to run Nginx in kernel space?

drzaiusx11 8y ago

It seems like wasm will finally enable the "write once, run everywhere" promise that java made but never truly executed by starting with the premise that you don't need "one true language" (java), but rather just the VM. Yeah, I know the JVM supports several languages these days but most require non-superficial similarities to Java (garbage collected, etc.)

kodablah 8y ago
> but most require non-superficial similarities to Java (garbage collected, etc.) It should be noted that doing this requires non-superficial similarities to *nix/POSIX (signals, files, threads, etc). It's not like you could run Nginx on this without its POSIX impl or in the browser w/out Emscripten's POSIX impl or on any other WASM runtime w/out a POSIX impl.
chrisseaton 8y ago
> most require non-superficial similarities to Java (garbage collected, etc.) You can (obviously) implement any language without garbage collection semantics using garbage collection, so this requirement is false. See for example languages like C and C++ running on the JVM.
- drzaiusx11 8y ago
  Has this been done without jumping through hoops like compiling to a specific CPU target with GCC then doing a binary conversion to JVM bytecode (NestedVM does this via MIPS I believe)?
  - pjmlp 8y ago
    GraalVM understands LLVM bitcode. https://www.graalvm.org/docs/reference-manual/languages/llvm/ https://www.graalvm.org/docs/reference-manual/languages/llvm...
    - drzaiusx11 8y ago
      GraavlVM Seems like an incredibly useful tool for sandboxing non-java languages with java/jvm interoperability. LLVM really is enabling a renaissance of language interop. This is fantastic stuff.

lachlan-sneff 8y ago

Wasmer just got nginx working on their wasm runtime as well!

gok 8y ago

Sounds like the big idea from Singularity (run everything in the same hardware security ring, use static analysis for memory safety) is going mainstream, incrementally. Unfortunate that this is happening right as it becomes clear that such a design is fundamentally unsound on modern hardware.

yolo42 8y ago
There is another layer of abstraction that is heavily being exploited of running layers of kernels. I'm really interested in seeing more advances in this and unikernal approach where if you are running in the cloud, the hypervisor already provides you with a sandbox, so run things at a lower level than necessary. There are certain secuirty challenges that we see because we keep thinking in terms of user and kernal space. If we try to narrow (and slowly remove) the line separating the spaces, we can address these problems in more efficient way than done today for sure.
- bpye 8y ago
  gVisor is pretty interesting, it kinda takes this idea when you run it on KVM. https://github.com/google/gvisor https://github.com/google/gvisor
_wmd 8y ago
The big news this year was that the majority of hardware isolation designs in existence were fundamentally unsound, this is a software sandbox. Are you referring to the recent discovery of a new hardware bug class here?
- stcredzero 8y ago
  So is the idea to take a page from the RISC and VLSI playbooks and outsource the isolation and security to the compiler/JIT VM?
- gok 8y ago
  The big news this year was that you can't run code of different trust levels in the same address space.

traverseda 8y ago

Obligatory birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death... I suppose calling it METAL would have been too on-the-nose.

molf 8y ago
Not sure why you're downvoted, this is a very interesting talk.
- traverseda 8y ago
  It's pretty relevant to the discussion of "why would you want to run wasm in the kernel", but I'm not too worried about the votes.
  - reitzensteinm 8y ago
    It's not, though... WebAssembly doesn't really have all that much to do with js, any more than Flash or Java plugins would if they ended up being standardized instead. Every time there's a wasm thread it gets posted, but it misses the point of the talk to suggest that wasm is the prediction bearing fruit. The talk is great, but I'd suggest that's the reason for the downvotes.
    - earenndil 8y ago
      > it misses the point of the talk to suggest that wasm is the prediction bearing fruit Does it? It seems that the talk has two main points: 1. Javascript succeeded because it was (at least initially) just good enough not to be completely unbearable, but bad enough that people ended up using it primarily as a target for other languages. 2. Ring 0 JIT can be 4% faster that normal binaries. WASM is primarily a target for other languages, and qualifies as a language that can theoretically be JITted 4% faster than native code can be run.
      - 8y ago

syrusakbary 8y ago

Hi HN! I'm Syrus, from the Wasmer team. We have been working in something similar, but with a special focus on maintainability and with bigger goals in mind: https://wasmer.io/ https://wasmer.io/ Here is the article about our journey on Running Nginx (which funnily enough we actually accomplished just before wasmjit): https://medium.com/@syrusakbary/running-nginx-with-webassembly-6353c02c08ac https://medium.com/@syrusakbary/running-nginx-with-webassemb...

kodablah 8y ago
> but much more maintainable. > we actually accomplished just before wasmjit > Wasmer is the first native WebAssembly runtime [...] Whoa there. I like both projects and respect competition as much as the next guy, but maintainability is subjective and being first is of little importance.
- syrusakbary 8y ago
  True! This statements without analysis are wet paper. In the article I've linked there is a better analysis on why: 1. in wasmjit, the machine instructions are hardcoded into the runtime (this is like creating your own LLVM, by hand... and only available for x86) 2. it doesn't have a single test I was talking about my own experience here, because I tried to contribute to wasmjit before creating Wasmer... and was quite challenging! It might be useful to check how many people interacted with the code in each of this projects! ;) Also note the timeline that took for this projects to accomplish the same: wasmer (<2 months) wasmjit (6 months)
  - hangonhn 8y ago
    You may be correct on all those points but still come off as rude. And I think you're just trying to be helpful and steer people in the direction that you think is right. However, given the context of where you're doing it, it feels like someone is trying to ruin someone else's parade.
    - glibgil 8y ago

StellaQueen 8y ago

After leaving my previous job 12 months ago, i've had some good luck to learn about this website which was a life-saver for me... They offer jobs for which people can work online from their house. My latest paycheck after working for them for 4 months was for $4500... Amazing thing about is that the only thing required is simple typing skills and access to internet...Read all about it here... http://Careers7.com/ http://Careers7.com/

jeltz 8y ago

How do you avoid having to copy buffers from the kernel into WebAssembly? As far as I know WebAssembly does not provide a way to access memory outside the linear memory block, but maybe in this specific case (nginx) it is possible to have all syscalls write to a buffer allocated by WebAssembly.

zymhan 8y ago

Does anyone else look at this title and see a jumble of letters? I miss the days of pronounceable acronyms

mxuribe 8y ago
At a glance it actually looked like Dutch to me (with apologies to my Dutch buddies!)
idle_zealot 8y ago
"engine-ecks on whaz'm-jit" It's certainly pronounceable, though perhaps that pronunciation is non-obvious.
deleted 8y ago
[deleted]

coverband 8y ago

I might be getting too old, because I truly don't get the benefit of doing this... Nobody I know really cares about portability, and I don't see how running nginx with WASM is in any way better than running it directly on the system. Does anyone care to ELI5 for me?

lwb 8y ago
This is satire but still very interesting: https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death...
gok 8y ago
Running it in the kernel removes the overhead of context switching into the kernel for I/O (and other things), so it could be faster. Although as this post says, right now it's only running in user space.

sargun 8y ago

I don’t understand. How is this possible? The POSIX API is not implemented for WASM. Non web embedding have not yet been standardized, nor threads. How have they implemented this? Are they implementing a non standard embedding and pthreads? https://webassembly.org/docs/non-web/ https://webassembly.org/docs/non-web/