5 ms·

True, but I suspect it'll be a lot easier to virtualise all those APIs through WASM than it is for a regular native binary. I mean, half the point of docker is

True, but I suspect it'll be a lot easier to virtualise all those APIs through WASM than it is for a regular native binary. I mean, half the point of docker is that all syscalls are routed into an LXD container with its own filesystem and network. It should be pretty easy to do the same thing in userland with a wasm runtime. And the nice thing about that is you can pick which environment a wasm bundle runs in. Want to run it on the browser? Sure! Want to give it r/w access to some particular path? Fine! Or you want to run it "natively", with full access to the host operating system? That can work too! We just ("just") need wasi, and a good set of implementations which support all the different kinds of sandboxing that people want, and allow wasm containers to talk to each other in all the desirable ways. Make no mistake - this is a serious amount of work. But it looks like a very solvable problem. I think the bigger adoption problem will be all the performance you leave on the table by using wasm instead of native code. For all its flaws, docker on linux runs native binaries at native speed. I suspect big companies running big cloud deployments will stick with docker because it runs code faster.

danpalmer 2y ago
This assumes that everyone implements the same set of APIs that work in the same way. More likely, the browser will implement some that make sense there, some browsers will implement more than others, Cloudflare workers will implement a different set, AWS Lambda will implement a different set or have some that don't work the same way... and now you need to write your WASM code to deal with these differing implementations. Unless the API layer is, essentially, a Linux OS or maybe POSIX(?) for Docker, which I doubt it would be as that's a completely different level of abstraction to WASM, I don't have a lot of faith in this being a utopian ideal common API, given that as an industry we've so far failed almost every opportunity to make those common APIs.
- josephg 2y ago
  True. Its probably worth creating a validation suite for wasi which can check that any given implementation implements all the functions correctly & consistently. Like, I'm imagining a wasm bundle which calls all the APIs it expects in every different configuration and outputs a scorecard showing what works properly and what doesn't. I suspect you're right - unless people are careful, it'll be a jungle out there. Just like javascript is at the moment.
  - danpalmer 2y ago
    I understand your point but sadly I think it's too idealistic (not that we shouldn't strive for these goals). We already have those sorts of tests for browsers, and browser compatibility is still a problem. We have acceptance tests for other areas, like Android's CTS tests, but there are still incompatibilities. That also assumes that everyone involved wants compatibility, and that's unlikely. Imagine a world where every WASM implementation is identical. If one implementation decides to change something to implement an improvement to differentiate themselves in the market, they'll likely win marketshare from the others. Most companies implementing WASM will tend to want to a) control the spec in their own favour, and b) gain advantages over other implementations. And this is why we can't have nice things.
    - hardwaresofton 2y ago
      > I understand your point but sadly I think it's too idealistic (not that we shouldn't strive for these goals). We already have those sorts of tests for browsers, and browser compatibility is still a problem. We have acceptance tests for other areas, like Android's CTS tests, but there are still incompatibilities. > I think the browser problem is a marketshare/market power problem, and Wasm doesn't have that problem. Also, I'd argue that compat tests for JS engines and browsers are an overall positive thing -- at least compared to the world where there is no attempt to standardize at all. > That also assumes that everyone involved wants compatibility, and that's unlikely. Imagine a world where every WASM implementation is identical. If one implementation decides to change something to implement an improvement to differentiate themselves in the market, they'll likely win marketshare from the others. This is a good thing though -- as long as it happens without breaking compatibility. Users are very sensitive to changes that introduce lock-in/break standards, and the value would have to be outsized for someone to forgo having other options. > Most companies implementing WASM will tend to want to a) control the spec in their own favour, and b) gain advantages over other implementations. And this is why we can't have nice things. I think you can see this playing out right now in the Wasm ecosystem, and it isn't working out like you might expect. There are great benefits in building standards because of friction reduction for users -- as long as there is a "standards first" approach, people overwhelmingly pick it if functionality is close enough. Places that make sense to differentiate are differentiated, but those that do not start to get eaten by standards. I think organizations that are aware of this problem and attempt to address is directly like the Bytecode Alliance are also one of the only forms of bulwark against this.
- hardwaresofton 2y ago
  Good point! This is the hard work that people are undertaking right now. Things are going to change a little bit with the introduction of Preview3 (the flagship feature there is async without function coloring), but you can look at the core interfaces: https://github.com/WebAssembly/WASI/tree/main/wasip2 https://github.com/WebAssembly/WASI/tree/main/wasip2 This is what people are building on, in the upstream, and in the bytecode alliance You're absolutely right about embeddings being varied, but the standard existing enforces the expectations around a core set of these, and then the carving out of embeddings to support different use cases is a welcome and intended consequence. WASI started as closer to POSIX now, but there is a spectacular opportunity to not repeat some mistakes of the past, so some of those opportunities are taken where they make sense/won't cause too much disruption to people building in support.
  - DaiPlusPlus 2y ago
    > the flagship feature there is async without function coloring Correct me if I’m wrong, but that’s only possible if you separate runtime threads from OS threads, which sounds straightforward but introduces problems relating to stack-lifetimes in continuations so it introduces demands on the compiler and/or significant runtime memory overhead - which kinda defeats the point of trying to avoid blocking OS threads in the first place. I’m not belittling the achievement there - I’m just saying (again, correct me if I’m wrong) there’s a use-case for function-colouring in high-thread, high-memory applications. …but if WASI is simply adding more options without taking anything away then my point above is moot :)
    - danpalmer 2y ago
      Further to this, my (very basic) understanding is that the actual threading implementation will be left up to the integrator, so some implementations may not actually implement any concurrency (a little like the Python GIL in a way), while others may implement real concurrency, therefore meaning that subtle threading bugs could be introduced that wouldn't be seen until you run in other environments.
    - hardwaresofton 2y ago
- cudder 2y ago
  I have a funny feeling that some day we'll see Docker ported to wasm to "abstract it away cleanly". History is a circle and all that.
quesomaster9000 2y ago
As somebody who's in the process of building a sandbox for RISC-V 64 Linux ELF executables, even I'm still on the fence. The problem is that in WASM-land we're heading towards WASI and WAT components, which is similar to the .NET, COM & IDL ecosystems. While this is actually really cool in terms of component and interface discovery, the downside is that it means you have to re-invent the world to work with this flavor of runtime. Meaning... no, I can't really just output WASM from Go or Rust and it'll work, there's more to it, much more to it. With a RISC-V userland emulator I could compile that to WASM to run normal binaries in the browser, and provide a sandboxed syscall interface (or even just pass-through the syscalls to the host, like qemu-user does when running natively). Meaning I have high compatibility with most of the Linux userland within a few weeks of development effort. But yes, threads, forking, sockets, lots of edge cases - it's difficult to provide a minimal spoof of a Linux userland that's convincing enough that you can do interesting enough things, but surprisingly it's not too difficult - and with that you get Go, Rust, Zig, C++, C, D etc. and all the native tooling that you'd expect (e.g. it's quite easy to write a gdbserver compatible interface, but ... you usually don't need it, as you can just run & debug locally then cross-compile).
- hardwaresofton 2y ago
  > The problem is that in WASM-land we're heading towards WASI and WAT components, which is similar to the .NET, COM & IDL ecosystems. While this is actually really cool in terms of component and interface discovery, the downside is that it means you have to re-invent the world to work with this flavor of runtime. At the application level, you're generally going to write to the standards + your embedding. Companies that write embeddings are encouraced/incentivized to write good abstractions that work with standards to reduce user friction. For example, for making HTTP requests and responding to HTTP requests, there is WASI HTTP: https://github.com/WebAssembly/wasi-http https://github.com/WebAssembly/wasi-http It's written in a way that is robust enough to handle most use cases without much loss of efficiency. There are a few inefficiencies in the WIT contracts (that will go away soon, as async lands in p3), but it represents a near-ideal representation of a HTTP request and is easy for many vendors to build on/against. As far as rewriting the world, this happens to luckily not be quite true, thanks to projects like wasi-libc: https://github.com/webassembly/wasi-libc https://github.com/webassembly/wasi-libc Networking is actually much more solved in WASI now than it was roughly a year ago -- threads is taking a little longer to cook (for good reasons), but async (without function coloring) is coming this year (likely in the next 3-4 months). The sandboxing abilities of WASM are near unmatched, along with it's startup time and execution speed compared to native.
SEJeff 2y ago
docker and lxd are competing projects. Docker does not use lxd to launch containers. lxd was written by the lead dev (at canonical) of lxc which was not as polished as docker but sort of kind of did the same thing (ran better chroots) They both use Linux kernel features such as control groups and namespaces. When put together this is referred to as a container but the kernel has zero concept of “a container”.
- tecleandor 2y ago
  Docker started using an LXC driver to run workloads, but it was deprecated 10 years ago. No LXC remaining there :P
  - SEJeff 2y ago
    Cool but lxd is still a competitor to docker. I’ve read the move code for some work reasons. No lxc, as you said :)
    - tecleandor 2y ago
      Late reply but sorry, I was trying to reply to parent coment, that was mixing docker and LXD. Anyway, I don't see them as competitors. Although they do (in the core) a similar thing, I use them both for different tasks: LXD more similar to classic virtualization, and Docker for more tightly packaged things that are separated from its configuration and/or storage.
funcDropShadow 2y ago
> True, but I suspect it'll be a lot easier to virtualise all those APIs through WASM than it is for a regular native binary. I mean, half the point of docker is that all syscalls are routed into an LXD container with its own filesystem and network. It should be pretty easy to do the same thing in userland with a wasm runtime. All of this sounds too good to be true. The JVM tried to use one abstraction to abstract different processor ISAs, different operating systems, and a security boundary. The security boundary failed completely. As far as I understand WASM is choosing a different approach here, good. The abstraction over operating systems was a partial failure. It succeeded good enough for many types of server applications, but it was never good enough for desktop applications and system software. The abstraction over CPU was and is a big success, I'd say. What exactly makes you think it is easier with WASM as a CPU abstraction to do all the rest again? Even when thinking about so diverse use-cases like in-browser apps and long running servers. A big downside of all these super powerful abstraction layer is reaction to upstream changes. What happens when Linux introduces a next generation network API that has no counterpart in Windows or in the browser. What happens if the next language runtime wants to implement low-latency GC? Azul first designed a custom CPU and later changed the Linux API for memory management to make that possible for their JVM. All in all the track record of attempts to build the one true solution for all our problems is quite bad. Some of these attempt discovered niches in which they are a very good fit, like the JVM and others are a curiosity of history.
v3ss0n 2y ago
When did docker started using LXD ? i never knew.
- SSLy 2y ago
  never, they GP is wrong
guappa 2y ago
> It should be pretty easy to do the same thing in userland with a wasm runtime. Not easy and certainly not fast.
yencabulator 2y ago
> half the point of docker is that all syscalls are routed into an LXD container with its own filesystem and network. It should be pretty easy to do the same thing in userland with a wasm runtime. This is a serious misunderstanding of how containers work. Containers make syscalls. The Linux kernel serves them. Linux kernel has features that let one put userspace processes in namespaces where they don't see everything. There is no "routing". There is no "its own filesystem and network", just a namespace where only some of the host filesystems and networks are visible. There is no second implementation of the syscalls in that scenario. For WASM, someone has to implement the server-side "file I/O over WASI", "network I/O over WASI", and so on. And those APIs are likely going to be somewhat different looking than Linux syscalls, because the whole point is WASM was sandboxing. Quite far from "pretty easy".

danpalmer 2y ago

This assumes that everyone implements the same set of APIs that work in the same way. More likely, the browser will implement some that make sense there, some browsers will implement more than others, Cloudflare workers will implement a different set, AWS Lambda will implement a different set or have some that don't work the same way... and now you need to write your WASM code to deal with these differing implementations. Unless the API layer is, essentially, a Linux OS or maybe POSIX(?) for Docker, which I doubt it would be as that's a completely different level of abstraction to WASM, I don't have a lot of faith in this being a utopian ideal common API, given that as an industry we've so far failed almost every opportunity to make those common APIs.

josephg 2y ago
True. Its probably worth creating a validation suite for wasi which can check that any given implementation implements all the functions correctly & consistently. Like, I'm imagining a wasm bundle which calls all the APIs it expects in every different configuration and outputs a scorecard showing what works properly and what doesn't. I suspect you're right - unless people are careful, it'll be a jungle out there. Just like javascript is at the moment.
- danpalmer 2y ago
  I understand your point but sadly I think it's too idealistic (not that we shouldn't strive for these goals). We already have those sorts of tests for browsers, and browser compatibility is still a problem. We have acceptance tests for other areas, like Android's CTS tests, but there are still incompatibilities. That also assumes that everyone involved wants compatibility, and that's unlikely. Imagine a world where every WASM implementation is identical. If one implementation decides to change something to implement an improvement to differentiate themselves in the market, they'll likely win marketshare from the others. Most companies implementing WASM will tend to want to a) control the spec in their own favour, and b) gain advantages over other implementations. And this is why we can't have nice things.
  - hardwaresofton 2y ago
    > I understand your point but sadly I think it's too idealistic (not that we shouldn't strive for these goals). We already have those sorts of tests for browsers, and browser compatibility is still a problem. We have acceptance tests for other areas, like Android's CTS tests, but there are still incompatibilities. > I think the browser problem is a marketshare/market power problem, and Wasm doesn't have that problem. Also, I'd argue that compat tests for JS engines and browsers are an overall positive thing -- at least compared to the world where there is no attempt to standardize at all. > That also assumes that everyone involved wants compatibility, and that's unlikely. Imagine a world where every WASM implementation is identical. If one implementation decides to change something to implement an improvement to differentiate themselves in the market, they'll likely win marketshare from the others. This is a good thing though -- as long as it happens without breaking compatibility. Users are very sensitive to changes that introduce lock-in/break standards, and the value would have to be outsized for someone to forgo having other options. > Most companies implementing WASM will tend to want to a) control the spec in their own favour, and b) gain advantages over other implementations. And this is why we can't have nice things. I think you can see this playing out right now in the Wasm ecosystem, and it isn't working out like you might expect. There are great benefits in building standards because of friction reduction for users -- as long as there is a "standards first" approach, people overwhelmingly pick it if functionality is close enough. Places that make sense to differentiate are differentiated, but those that do not start to get eaten by standards. I think organizations that are aware of this problem and attempt to address is directly like the Bytecode Alliance are also one of the only forms of bulwark against this.
hardwaresofton 2y ago
Good point! This is the hard work that people are undertaking right now. Things are going to change a little bit with the introduction of Preview3 (the flagship feature there is async without function coloring), but you can look at the core interfaces: https://github.com/WebAssembly/WASI/tree/main/wasip2 https://github.com/WebAssembly/WASI/tree/main/wasip2 This is what people are building on, in the upstream, and in the bytecode alliance You're absolutely right about embeddings being varied, but the standard existing enforces the expectations around a core set of these, and then the carving out of embeddings to support different use cases is a welcome and intended consequence. WASI started as closer to POSIX now, but there is a spectacular opportunity to not repeat some mistakes of the past, so some of those opportunities are taken where they make sense/won't cause too much disruption to people building in support.
- DaiPlusPlus 2y ago
  > the flagship feature there is async without function coloring Correct me if I’m wrong, but that’s only possible if you separate runtime threads from OS threads, which sounds straightforward but introduces problems relating to stack-lifetimes in continuations so it introduces demands on the compiler and/or significant runtime memory overhead - which kinda defeats the point of trying to avoid blocking OS threads in the first place. I’m not belittling the achievement there - I’m just saying (again, correct me if I’m wrong) there’s a use-case for function-colouring in high-thread, high-memory applications. …but if WASI is simply adding more options without taking anything away then my point above is moot :)
  - danpalmer 2y ago
    Further to this, my (very basic) understanding is that the actual threading implementation will be left up to the integrator, so some implementations may not actually implement any concurrency (a little like the Python GIL in a way), while others may implement real concurrency, therefore meaning that subtle threading bugs could be introduced that wouldn't be seen until you run in other environments.
  - hardwaresofton 2y ago
cudder 2y ago
I have a funny feeling that some day we'll see Docker ported to wasm to "abstract it away cleanly". History is a circle and all that.

quesomaster9000 2y ago

As somebody who's in the process of building a sandbox for RISC-V 64 Linux ELF executables, even I'm still on the fence. The problem is that in WASM-land we're heading towards WASI and WAT components, which is similar to the .NET, COM & IDL ecosystems. While this is actually really cool in terms of component and interface discovery, the downside is that it means you have to re-invent the world to work with this flavor of runtime. Meaning... no, I can't really just output WASM from Go or Rust and it'll work, there's more to it, much more to it. With a RISC-V userland emulator I could compile that to WASM to run normal binaries in the browser, and provide a sandboxed syscall interface (or even just pass-through the syscalls to the host, like qemu-user does when running natively). Meaning I have high compatibility with most of the Linux userland within a few weeks of development effort. But yes, threads, forking, sockets, lots of edge cases - it's difficult to provide a minimal spoof of a Linux userland that's convincing enough that you can do interesting enough things, but surprisingly it's not too difficult - and with that you get Go, Rust, Zig, C++, C, D etc. and all the native tooling that you'd expect (e.g. it's quite easy to write a gdbserver compatible interface, but ... you usually don't need it, as you can just run & debug locally then cross-compile).

hardwaresofton 2y ago
> The problem is that in WASM-land we're heading towards WASI and WAT components, which is similar to the .NET, COM & IDL ecosystems. While this is actually really cool in terms of component and interface discovery, the downside is that it means you have to re-invent the world to work with this flavor of runtime. At the application level, you're generally going to write to the standards + your embedding. Companies that write embeddings are encouraced/incentivized to write good abstractions that work with standards to reduce user friction. For example, for making HTTP requests and responding to HTTP requests, there is WASI HTTP: https://github.com/WebAssembly/wasi-http https://github.com/WebAssembly/wasi-http It's written in a way that is robust enough to handle most use cases without much loss of efficiency. There are a few inefficiencies in the WIT contracts (that will go away soon, as async lands in p3), but it represents a near-ideal representation of a HTTP request and is easy for many vendors to build on/against. As far as rewriting the world, this happens to luckily not be quite true, thanks to projects like wasi-libc: https://github.com/webassembly/wasi-libc https://github.com/webassembly/wasi-libc Networking is actually much more solved in WASI now than it was roughly a year ago -- threads is taking a little longer to cook (for good reasons), but async (without function coloring) is coming this year (likely in the next 3-4 months). The sandboxing abilities of WASM are near unmatched, along with it's startup time and execution speed compared to native.

SEJeff 2y ago

docker and lxd are competing projects. Docker does not use lxd to launch containers. lxd was written by the lead dev (at canonical) of lxc which was not as polished as docker but sort of kind of did the same thing (ran better chroots) They both use Linux kernel features such as control groups and namespaces. When put together this is referred to as a container but the kernel has zero concept of “a container”.

tecleandor 2y ago
Docker started using an LXC driver to run workloads, but it was deprecated 10 years ago. No LXC remaining there :P
- SEJeff 2y ago
  Cool but lxd is still a competitor to docker. I’ve read the move code for some work reasons. No lxc, as you said :)
  - tecleandor 2y ago
    Late reply but sorry, I was trying to reply to parent coment, that was mixing docker and LXD. Anyway, I don't see them as competitors. Although they do (in the core) a similar thing, I use them both for different tasks: LXD more similar to classic virtualization, and Docker for more tightly packaged things that are separated from its configuration and/or storage.

funcDropShadow 2y ago

> True, but I suspect it'll be a lot easier to virtualise all those APIs through WASM than it is for a regular native binary. I mean, half the point of docker is that all syscalls are routed into an LXD container with its own filesystem and network. It should be pretty easy to do the same thing in userland with a wasm runtime. All of this sounds too good to be true. The JVM tried to use one abstraction to abstract different processor ISAs, different operating systems, and a security boundary. The security boundary failed completely. As far as I understand WASM is choosing a different approach here, good. The abstraction over operating systems was a partial failure. It succeeded good enough for many types of server applications, but it was never good enough for desktop applications and system software. The abstraction over CPU was and is a big success, I'd say. What exactly makes you think it is easier with WASM as a CPU abstraction to do all the rest again? Even when thinking about so diverse use-cases like in-browser apps and long running servers. A big downside of all these super powerful abstraction layer is reaction to upstream changes. What happens when Linux introduces a next generation network API that has no counterpart in Windows or in the browser. What happens if the next language runtime wants to implement low-latency GC? Azul first designed a custom CPU and later changed the Linux API for memory management to make that possible for their JVM. All in all the track record of attempts to build the one true solution for all our problems is quite bad. Some of these attempt discovered niches in which they are a very good fit, like the JVM and others are a curiosity of history.

v3ss0n 2y ago

When did docker started using LXD ? i never knew.

SSLy 2y ago
never, they GP is wrong

guappa 2y ago

> It should be pretty easy to do the same thing in userland with a wasm runtime. Not easy and certainly not fast.

yencabulator 2y ago

> half the point of docker is that all syscalls are routed into an LXD container with its own filesystem and network. It should be pretty easy to do the same thing in userland with a wasm runtime. This is a serious misunderstanding of how containers work. Containers make syscalls. The Linux kernel serves them. Linux kernel has features that let one put userspace processes in namespaces where they don't see everything. There is no "routing". There is no "its own filesystem and network", just a namespace where only some of the host filesystems and networks are visible. There is no second implementation of the syscalls in that scenario. For WASM, someone has to implement the server-side "file I/O over WASI", "network I/O over WASI", and so on. And those APIs are likely going to be somewhat different looking than Linux syscalls, because the whole point is WASM was sandboxing. Quite far from "pretty easy".