6 ms·

> the downside is that it means you have to re-invent the world to work with this flavor of runtime. This is at least one of the reasons we've been building th

by titzer 2y ago

> the downside is that it means you have to re-invent the world to work with this flavor of runtime. This is at least one of the reasons we've been building thin kernel interfaces for Wasm. We've built two now, one for the Linux syscall interface (https://github.com/arjunr2/WALI https://github.com/arjunr2/WALI) and one for Zephyr. A preliminary paper we wrote a year or so back is here (https://arxiv.org/abs/2312.03858 https://arxiv.org/abs/2312.03858), and we have a new one coming up in Eurosys 25. One of the advantages of a thin kernel interface to something like Linux is really low overhead and low implementation burden for Wasm engines. This makes it easier to then build things like WASI just one level up, compiled against the kernel interface and delivered as a Wasm module. Thus a single WASI implementation can be reused across engines.

yencabulator 2y ago
> One of the advantages of a thin kernel interface to something like Linux is really low overhead and low implementation burden for Wasm engines. Such a low burden that both Google (gVisor) and Microsoft (WLS1) failed at it!
- titzer 2y ago
  A thin kernel interface isn't a reimplementation of a kernel. The WALI implementation in WAMR is ~2000 lines of C, most of which is just pass-through system calls.
  - yencabulator 2y ago
    Okay, so you mean forwarding the syscalls, not implementing them, and thus throwing away the wasm sandbox.
    - titzer 2y ago
      It does not throw away the Wasm sandbox. Sandboxing means two things: memory sandboxing and system sandboxing. It retains the former. For the latter you can apply the same kinds of sandboxing policies as native processes and achieve the same effect, or even do it more efficiently in-process by the engine, and do interposition and whitelist/blacklisting more robustly than, e.g. seccomp.
      - yencabulator 2y ago
        Alright, selectively forwarding the syscalls, now you're approaching the problem again where you need to reimplement parts of Linux to understand the state machine of what fd 432 means at any given point in time etc; basically you're implementing the ideas of gVisor in a slightly different shape, without being able to run preexisting binaries. Doesn't seem like a useful combination of features, to me.
        titzer 2y ago
        > you need to reimplement parts of Linux Again, no. The security policies we have in mind can be implemented above the WALI call layer and supplied as an interposition library as a Wasm module. So you can have custom policies that run on any engine, such as implementing the WASI security model as a library. As it is now, all of WASI has to be implemented within the Wasm engine because the engine is the only entity with authority to do so. That's problematic in that engines have N different incompatible, incomplete and buggy implementations of WASI, and those bugs can be memory safety violations that own the entire process. Thin kernel interfaces separate the engine evolution problem from the system interface evolution problem and make the entire software stack more robust by providing isolation for higher-level interfaces.
        yencabulator 2y ago
        To filter out syscalls for complex policies, you need to understand the semantics of prior syscalls. For example, you need to keep track of what the dirfs in an unlinkat call refers to. And to keep track of FDs you need to reimplement fcntl. And so on. This is why gVisor contains a reimplementation of parts of Linux.
        titzer 2y ago
        Yes, but the engine doesn't need to do this, you can do this on your own time as a library. As there are literally dozens of Wasm engines now, thin kernel interfaces are a stable interface that they can all implement in exactly the same way[1] (simple safety checks + pass through) and then higher-level, more safe, and in some way better policies and APIs can be implemented as Wasm modules on top. [1] This makes the interface per-kernel, not per-kernel x per-engine. It's also not per-kernel x per-kernel; engines would not be required to emulate one kernel on another kernel.

by titzer 2y ago

yencabulator 2y ago

> One of the advantages of a thin kernel interface to something like Linux is really low overhead and low implementation burden for Wasm engines. Such a low burden that both Google (gVisor) and Microsoft (WLS1) failed at it!

titzer 2y ago
A thin kernel interface isn't a reimplementation of a kernel. The WALI implementation in WAMR is ~2000 lines of C, most of which is just pass-through system calls.
- yencabulator 2y ago
  Okay, so you mean forwarding the syscalls, not implementing them, and thus throwing away the wasm sandbox.
  - titzer 2y ago
    It does not throw away the Wasm sandbox. Sandboxing means two things: memory sandboxing and system sandboxing. It retains the former. For the latter you can apply the same kinds of sandboxing policies as native processes and achieve the same effect, or even do it more efficiently in-process by the engine, and do interposition and whitelist/blacklisting more robustly than, e.g. seccomp.
    - yencabulator 2y ago
      Alright, selectively forwarding the syscalls, now you're approaching the problem again where you need to reimplement parts of Linux to understand the state machine of what fd 432 means at any given point in time etc; basically you're implementing the ideas of gVisor in a slightly different shape, without being able to run preexisting binaries. Doesn't seem like a useful combination of features, to me.
      - titzer 2y ago
        > you need to reimplement parts of Linux Again, no. The security policies we have in mind can be implemented above the WALI call layer and supplied as an interposition library as a Wasm module. So you can have custom policies that run on any engine, such as implementing the WASI security model as a library. As it is now, all of WASI has to be implemented within the Wasm engine because the engine is the only entity with authority to do so. That's problematic in that engines have N different incompatible, incomplete and buggy implementations of WASI, and those bugs can be memory safety violations that own the entire process. Thin kernel interfaces separate the engine evolution problem from the system interface evolution problem and make the entire software stack more robust by providing isolation for higher-level interfaces.
        yencabulator 2y ago
        To filter out syscalls for complex policies, you need to understand the semantics of prior syscalls. For example, you need to keep track of what the dirfs in an unlinkat call refers to. And to keep track of FDs you need to reimplement fcntl. And so on. This is why gVisor contains a reimplementation of parts of Linux.
        titzer 2y ago
        Yes, but the engine doesn't need to do this, you can do this on your own time as a library. As there are literally dozens of Wasm engines now, thin kernel interfaces are a stable interface that they can all implement in exactly the same way[1] (simple safety checks + pass through) and then higher-level, more safe, and in some way better policies and APIs can be implemented as Wasm modules on top. [1] This makes the interface per-kernel, not per-kernel x per-engine. It's also not per-kernel x per-kernel; engines would not be required to emulate one kernel on another kernel.