8 ms·

Kernel-WASM: Sandboxed kernel mode WebAssembly runtime for Linux

103 points by nateb2022 4y ago

hulitu 4y ago
Haha. Now the browser runs in the kernel. What can go wrong ? I think it was Microsoft that saw that running graphics in kernel is a bad idea. But, yeah, this was a very long time ago and why not make the same mistakes over and over again.
- brundolf 4y ago
  It's not a browser (or even a piece of a browser), it's a method of statically sandboxing a native binary.
  - yencabulator 4y ago
    WASM isn't really "native". A C/Rust/etc compiler can target it, but it's very much a non-native ISA, with its own "syscall equivalents", its own limitations, etc.
    - brundolf 4y ago
      But it doesn't have to be run in a runtime; it can compile directly to a platform-native binary. Internally it has limitations, as any sandboxed code would, but it lets you take native code and compile it to a native binary with sandboxing built-in
      - yencabulator 4y ago
        By that argument, Python is "native" too then, with e.g. https://github.com/exaloop/codon https://github.com/exaloop/codon The reason why it doesn't quite work out is that typically such a thing, AOT-compiled into native code, still carries with it its limitations and assumptions. It's technically native, but experiencing the native APIs only through a narrow peephole in the fence. If we let this meaning of "native" take over, we're gonna need another word for actually native things. Sometimes, that works well enough, though!
        brundolf 4y ago
        That's fair, although I guess "having a runtime" can also refer to things like running a garbage collector (which WASM doesn't but Python does, and even Go does), even if it's part of the same binary. But we're entering the territory where these terms start to get poorly-defined I think (new tech is pushing the boundaries!) Subjectively I think WASM's feature and non-feature list makes it plausibly kernel-friendly in a way that eg. Python isn't, but there may not be a concrete line to draw around that
- chx 4y ago
  When I put webassembly into google the very first result has this to say: > WebAssembly (abbreviated Wasm) is a binary instruction format for a stack-based virtual machine. Yes, browsers implement this nowadays but there are other implementations as well. You can compile them into llvm bytecode for example or there's another for the GraalVM etc
- jay-barronville 4y ago
  Sorry but respectfully, you might want to research things before jumping to conclusions. WebAssembly doesn’t have anything to do with a web browser. It’s literally just a virtual machine standard. The standard was written with web browsers in mind (i.e., secure sandboxes within the browser environment), but nothing about WebAssembly’s semantics and runtime environment requires a web browser.
phoe-krk 4y ago
It's amazing to see https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death... becoming more and more true with every year.
- jay-barronville 4y ago
  WebAssembly isn’t JavaScript nor does it require JavaScript.
  - phoe-krk 4y ago
    Your response is technically correct but misses the entire point of the video. To make a tl;dr, the video talks about a technology called asm.js which other applications end up compiled into so they can run with near-native speeds on any system which supports asm.js. This technology ends up being a compilation target meaning that JS gets eventually forgotten as a language because everyone just compiles everything into asm.js which is executed natively on the kernel. Just do a s/asm.js/WASM/g and it'll suddenly start making sense.
    - jay-barronville 4y ago
      Okay, fair enough. I read the description but I didn’t watch the video, so it’s possible I’m lacking context. Thank you for clarifying.
      - phoe-krk 4y ago
        Yes, the video is pretty amazing - Gary in 2014 managed to predict pretty much everything that's happening with WASM nowadays: the push for safe execution of software compiled in unsafe languages, a common VMesque instruction set, web browsers implementing it first and pushing the frontier in this regard, and now kernels also starting to be capable of running that in order to avoid the cost of switching rings. What we're yet to see is this last point becoming the default, so execution of user programs kind-of makes a circle by returning to kernel mode, except now a trusted WASM-to-native compiler/runtime is responsible for ensuring system safety. Might happen in the next few years.
IncRnd 4y ago
> Safely run WebAssembly in the Linux kernel, with faster-than-native performance. That's the first sentence on the GH page. It seems they decided to mix marketing in first.
- imtringued 4y ago
  And faster than "native" means faster than userspace... And by faster they mean IO performance which is fine in principle but generally people think about compute when they care about the performance overhead of WASM because there is nothing about it that should make IO slower than native but everything about it makes compute slower than natively compiled programs.
  - deleted 4y ago
    [deleted]
  - dahfizz 4y ago
    I would love to see an actually fair comparison, where the native implementation uses io_uring and huge pages.
  - deleted 4y ago
    [deleted]
  - deleted 4y ago
    [deleted]
syrusakbary 4y ago
It’s always refreshing to see interest on WebAssembly and Wasmer projects! This kernel project is a bit outdated since is not using the latest Wasmer API (3.1), but it would be great to see community can pick it up and move it further. By making Wasm run in the kernel we were able to get really great runtime systemcall speeds since the program has not to pay any costs for crossing kernel protection rings
k__ 4y ago
Faster than native? So, WebAssembly has achieved what Java tried to sell us since it's inception?
- egberts1 4y ago
  Yeah, basically by bypassing the OS VM memory switching/context-switch with their own brand of memory-switching/context-switching, but with one notable difference: stackless internal representation (IR) bytecodes. Can we open up a new class of vulnerabilities in Mitre?
  - k__ 4y ago
    Could operating systems use this technique to speed up things?
    - dahfizz 4y ago
      We already have things like io_uring to achieve this speedup. As usual, this benchmark is comparing against an extremely naive implementation. It even uses a non-optimizing compiler! A fair benchmark would take advantage of all the features that native code has available - huge pages, CPU acceleration, io_uring, and, most obviously, a compiler that supports -O2. If they did that, the native code would smoke wasm.
- AnIdiotOnTheNet 4y ago
  To be fair, almost any JIT VM is capable of being "faster than native" (by which is meant pure AoT) in some circumstances because it has more information on the environment and circumstances of execution.
  - k__ 4y ago
    That's what I'm reading since the first release of Java. Still, C/C++ and Rust are running circles around it.
    - 4y ago
mgaunard 4y ago
Or you could just use io_uring and stay in userland without context switching.
- rektide 4y ago
  90% accurate. If you are on a single core system, you might avoid a lot of syscalls, yes. But the kernel is still going to be context switching your work out to actually do the io.
  - mgaunard 4y ago
    Single core systems don't really exist anymore outside embedded systems.
    - rektide 4y ago
      Sure. I guess the point is that the kernel still does work. There's no official kernel mechanism where IO is entirely done in userland (Like Intel's DPDK and SPDK). The kernel will be context switching & doing IO. Single core was just an extreme example to make that extremely visible & obvious, but the core point is the kernel needs time too during io_uring, whatever the core count.
      - mgaunard 4y ago
        So long as you have multiple cores, you can make it so that userland and the kernel are on different cores with nothing else on them. You can also disable interrupts and just spin continuously both the user and kernel thread. No context switches ever needs to occur.
        rektide 4y ago
        Are you proposing giving up a core or more of your system to kernel- and moving your data between cores to ship it!- as a win? There's some scenarios where I can picture that being a win, where ultra-low latency compute is key but higher latency is fine- but this technical possibility doesnt general excite me. We may technically be at 100% no context switching, but we did kill 1/8th of our cores or whatever & stress the fabric a lot more, just to satisfy a technical constraint we set up that sounds good but is actually foolish to ask for. Going totally userland is probably smarter. Go all in on SPDK or DPDK (storage/networking). You'll need dedicated networking for the app. In general though, I think your top post was 90% right, and that's good enough, and right now hunting for 100% is a mis-goal.
egberts1 4y ago
Today is January 21, 2023 and JavaScript must die.
- dmytrish 4y ago
  I don't quite understand what Javascript has to do with the topic of the discussion. WebAssembly is to Web what JavaScript is to Java.
  - pffft8888 4y ago
    I was thinking that these comments about death of JS in this thread which is about running in OS kernel mode not inside the browser are coming from ChatGPT type bots or the users are product managers with no clue.
remram 4y ago
I'm surprised no one is mentioning eBPF. I know the goals are slightly different but there seems to be a lot of overlap, e.g. would we really want both runtimes in the kernel?
- Laaas 4y ago
  eBPF is mentioned in the README, but I think you could trivially implement eBPF on top of this (and quite possibly have it be at least as performant).
Laaas 4y ago
This is 2 years old, does anyone know of any newer progress in this area?
- brabel 4y ago
  It might be just my impression, but it seems WASM itself has not moved forward in the last 2 years.
  - Laaas 4y ago
    No it definitely has moved forward. See: https://news.ycombinator.com/item?id=32069418 https://news.ycombinator.com/item?id=32069418 https://github.com/WebAssembly/tail-call/issues/15#issuecomment-1385891079 https://github.com/WebAssembly/tail-call/issues/15#issuecomm...
yjftsjthsd-h 4y ago
So the performance win appears to be from relying on sandboxing to safely run everything in ring zero so they can bypass the overhead of system calls. Which is actually pretty cool, and depending on the workload could actually reasonably lead to performance being better than user space.
- naasking 4y ago
  It also introduces DoS opportunities because resource ownership is murkier in the kernel.

103 points by nateb2022 4y ago

Haha. Now the browser runs in the kernel. What can go wrong ? I think it was Microsoft that saw that running graphics in kernel is a bad idea. But, yeah, this was a very long time ago and why not make the same mistakes over and over again.

brundolf 4y ago
It's not a browser (or even a piece of a browser), it's a method of statically sandboxing a native binary.
- yencabulator 4y ago
  WASM isn't really "native". A C/Rust/etc compiler can target it, but it's very much a non-native ISA, with its own "syscall equivalents", its own limitations, etc.
  - brundolf 4y ago
    But it doesn't have to be run in a runtime; it can compile directly to a platform-native binary. Internally it has limitations, as any sandboxed code would, but it lets you take native code and compile it to a native binary with sandboxing built-in
    - yencabulator 4y ago
      By that argument, Python is "native" too then, with e.g. https://github.com/exaloop/codon https://github.com/exaloop/codon The reason why it doesn't quite work out is that typically such a thing, AOT-compiled into native code, still carries with it its limitations and assumptions. It's technically native, but experiencing the native APIs only through a narrow peephole in the fence. If we let this meaning of "native" take over, we're gonna need another word for actually native things. Sometimes, that works well enough, though!
      - brundolf 4y ago
        That's fair, although I guess "having a runtime" can also refer to things like running a garbage collector (which WASM doesn't but Python does, and even Go does), even if it's part of the same binary. But we're entering the territory where these terms start to get poorly-defined I think (new tech is pushing the boundaries!) Subjectively I think WASM's feature and non-feature list makes it plausibly kernel-friendly in a way that eg. Python isn't, but there may not be a concrete line to draw around that
chx 4y ago
When I put webassembly into google the very first result has this to say: > WebAssembly (abbreviated Wasm) is a binary instruction format for a stack-based virtual machine. Yes, browsers implement this nowadays but there are other implementations as well. You can compile them into llvm bytecode for example or there's another for the GraalVM etc
jay-barronville 4y ago
Sorry but respectfully, you might want to research things before jumping to conclusions. WebAssembly doesn’t have anything to do with a web browser. It’s literally just a virtual machine standard. The standard was written with web browsers in mind (i.e., secure sandboxes within the browser environment), but nothing about WebAssembly’s semantics and runtime environment requires a web browser.

phoe-krk 4y ago

It's amazing to see https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death... becoming more and more true with every year.

jay-barronville 4y ago
WebAssembly isn’t JavaScript nor does it require JavaScript.
- phoe-krk 4y ago
  Your response is technically correct but misses the entire point of the video. To make a tl;dr, the video talks about a technology called asm.js which other applications end up compiled into so they can run with near-native speeds on any system which supports asm.js. This technology ends up being a compilation target meaning that JS gets eventually forgotten as a language because everyone just compiles everything into asm.js which is executed natively on the kernel. Just do a s/asm.js/WASM/g and it'll suddenly start making sense.
  - jay-barronville 4y ago
    Okay, fair enough. I read the description but I didn’t watch the video, so it’s possible I’m lacking context. Thank you for clarifying.
    - phoe-krk 4y ago
      Yes, the video is pretty amazing - Gary in 2014 managed to predict pretty much everything that's happening with WASM nowadays: the push for safe execution of software compiled in unsafe languages, a common VMesque instruction set, web browsers implementing it first and pushing the frontier in this regard, and now kernels also starting to be capable of running that in order to avoid the cost of switching rings. What we're yet to see is this last point becoming the default, so execution of user programs kind-of makes a circle by returning to kernel mode, except now a trusted WASM-to-native compiler/runtime is responsible for ensuring system safety. Might happen in the next few years.

IncRnd 4y ago

> Safely run WebAssembly in the Linux kernel, with faster-than-native performance. That's the first sentence on the GH page. It seems they decided to mix marketing in first.

imtringued 4y ago
And faster than "native" means faster than userspace... And by faster they mean IO performance which is fine in principle but generally people think about compute when they care about the performance overhead of WASM because there is nothing about it that should make IO slower than native but everything about it makes compute slower than natively compiled programs.
- deleted 4y ago
  [deleted]
- dahfizz 4y ago
  I would love to see an actually fair comparison, where the native implementation uses io_uring and huge pages.
- deleted 4y ago
  [deleted]
- deleted 4y ago
  [deleted]

syrusakbary 4y ago

It’s always refreshing to see interest on WebAssembly and Wasmer projects! This kernel project is a bit outdated since is not using the latest Wasmer API (3.1), but it would be great to see community can pick it up and move it further. By making Wasm run in the kernel we were able to get really great runtime systemcall speeds since the program has not to pay any costs for crossing kernel protection rings

k__ 4y ago

Faster than native? So, WebAssembly has achieved what Java tried to sell us since it's inception?

egberts1 4y ago
Yeah, basically by bypassing the OS VM memory switching/context-switch with their own brand of memory-switching/context-switching, but with one notable difference: stackless internal representation (IR) bytecodes. Can we open up a new class of vulnerabilities in Mitre?
- k__ 4y ago
  Could operating systems use this technique to speed up things?
  - dahfizz 4y ago
    We already have things like io_uring to achieve this speedup. As usual, this benchmark is comparing against an extremely naive implementation. It even uses a non-optimizing compiler! A fair benchmark would take advantage of all the features that native code has available - huge pages, CPU acceleration, io_uring, and, most obviously, a compiler that supports -O2. If they did that, the native code would smoke wasm.
AnIdiotOnTheNet 4y ago
To be fair, almost any JIT VM is capable of being "faster than native" (by which is meant pure AoT) in some circumstances because it has more information on the environment and circumstances of execution.
- k__ 4y ago
  That's what I'm reading since the first release of Java. Still, C/C++ and Rust are running circles around it.
  - 4y ago

mgaunard 4y ago

Or you could just use io_uring and stay in userland without context switching.

rektide 4y ago
90% accurate. If you are on a single core system, you might avoid a lot of syscalls, yes. But the kernel is still going to be context switching your work out to actually do the io.
- mgaunard 4y ago
  Single core systems don't really exist anymore outside embedded systems.
  - rektide 4y ago
    Sure. I guess the point is that the kernel still does work. There's no official kernel mechanism where IO is entirely done in userland (Like Intel's DPDK and SPDK). The kernel will be context switching & doing IO. Single core was just an extreme example to make that extremely visible & obvious, but the core point is the kernel needs time too during io_uring, whatever the core count.
    - mgaunard 4y ago
      So long as you have multiple cores, you can make it so that userland and the kernel are on different cores with nothing else on them. You can also disable interrupts and just spin continuously both the user and kernel thread. No context switches ever needs to occur.
      - rektide 4y ago
        Are you proposing giving up a core or more of your system to kernel- and moving your data between cores to ship it!- as a win? There's some scenarios where I can picture that being a win, where ultra-low latency compute is key but higher latency is fine- but this technical possibility doesnt general excite me. We may technically be at 100% no context switching, but we did kill 1/8th of our cores or whatever & stress the fabric a lot more, just to satisfy a technical constraint we set up that sounds good but is actually foolish to ask for. Going totally userland is probably smarter. Go all in on SPDK or DPDK (storage/networking). You'll need dedicated networking for the app. In general though, I think your top post was 90% right, and that's good enough, and right now hunting for 100% is a mis-goal.

egberts1 4y ago

Today is January 21, 2023 and JavaScript must die.

dmytrish 4y ago
I don't quite understand what Javascript has to do with the topic of the discussion. WebAssembly is to Web what JavaScript is to Java.
- pffft8888 4y ago
  I was thinking that these comments about death of JS in this thread which is about running in OS kernel mode not inside the browser are coming from ChatGPT type bots or the users are product managers with no clue.

remram 4y ago

I'm surprised no one is mentioning eBPF. I know the goals are slightly different but there seems to be a lot of overlap, e.g. would we really want both runtimes in the kernel?

Laaas 4y ago
eBPF is mentioned in the README, but I think you could trivially implement eBPF on top of this (and quite possibly have it be at least as performant).

Laaas 4y ago

This is 2 years old, does anyone know of any newer progress in this area?

brabel 4y ago
It might be just my impression, but it seems WASM itself has not moved forward in the last 2 years.
- Laaas 4y ago
  No it definitely has moved forward. See: https://news.ycombinator.com/item?id=32069418 https://news.ycombinator.com/item?id=32069418 https://github.com/WebAssembly/tail-call/issues/15#issuecomment-1385891079 https://github.com/WebAssembly/tail-call/issues/15#issuecomm...

yjftsjthsd-h 4y ago

So the performance win appears to be from relying on sandboxing to safely run everything in ring zero so they can bypass the overhead of system calls. Which is actually pretty cool, and depending on the workload could actually reasonably lead to performance being better than user space.

naasking 4y ago
It also introduces DoS opportunities because resource ownership is murkier in the kernel.