4 ms·

There's some quite different systems programming topics getting conflated here. This assumption that WASM is a competitor to Docker doesn't seem very sensible a

There's some quite different systems programming topics getting conflated here. This assumption that WASM is a competitor to Docker doesn't seem very sensible and it's hard to understand where it comes from. Will Python or Postgres suddenly stop having versioning and installation issues if they run on a WASM interpreter? No, the issues are at the ecosystem level, they're not related to how code is translated for the host CPU. You can optimize the size of your containers to be pretty small, like tens of megabytes… But WebAssembly is, at its core, designed to be more portable than that. You’re talking about tens of kilobytes, instead of tens of megabytes. And the boot-up times can be measured in microseconds, instead of milliseconds, or tens of milliseconds, or even seconds (!) for containers. This paragraph is emblematic: it's mixing up code size, startup time, portability, containers and virtual machines of two different kinds without really linking them together or defining requirements. The thing is, WASM isn't a new idea. The industry has a lot of experience with VMs that JIT compile portable bytecode. Startup time of any WASM program is going to be dominated by the fact that you're in the interpreter the whole time. As programs get larger they do things like load config files, connect to databases, initialize in-memory caches, possibly reflect themselves because that's often convenient, and so on. Because it's all init code none of this is hot, so it gets interpreted and that's slow. WASM offers nothing in the way of a standard library that could be accelerated, so languages have to bring their own which makes things worse. And the WASM environment is extremely minimalist, so you need to ship a lot of code with your app to make things comfortable especially as most servers aren't written in C++ or Rust and aren't going to be any time soon because that's not their sweet spot.

ilaksh 3y ago
WASM can be pre-compiled I assume. A basic use case for moving from containers to WASM is to provide the option for custom backend code for users in a more lightweight way. For example, I have a site that allows procedural image generation via an API and I originally built it using containers. It could be much less resource-intensive and faster startup if I could run that code in web assembly. Or, I was building a generative AI site that creates custom web applications. In order to allow backend code, I set it up so everyone gets their own fly.io VM/container(s). This is very flexible but not very fast startup time and the smallest option is like 128MB or 256MB RAM which needs to stay loaded otherwise they will have to wait for another restart. I never really needed to provide arbitrary Linux capabilities, just sandboxed backend code. So if I do something like that again I will try to use web assembly instead. Probably with lightweight TinyGo or Rust-based request handlers or something like that.
paulgb 3y ago
I mostly agree with you, but I think I can shed some light here: > As programs get larger they do things like load config files, connect to databases, initialize in-memory caches, possibly reflect themselves because that's often convenient, and so on. Because it's all init code none of this is hot, so it gets interpreted and that's slow. One of the most interesting aspects of Wasm is that because it is built with isolation in mind, it's really easy to get a snapshot and run it later. So instead of doing all the cache initialization etc. at init time, you can initialize the module at build time and then take a snapshot of it for a very quick cold start next time. When running in the browser, there is still an initialization step to process the wasm bytecode, but in the server-side world this can also be done in advance since you can trust the compiler and know in advance which processor architecture you will want to run it on.
- mike_hearn 3y ago
  Right, but that trick also isn't new. emacs was doing it decades ago and there are JVMs that also do it. That's how GraalVM native images can start and run faster than C programs do: the app is initialized at build time and then the state is just mmapped into a pre-compiled native process. HotSpot has a less aggressive version of this where you can pre-initialize a lot of VM state and dump it to a file for faster startup time. It's about a 30% win and that's going up over time, but it doesn't AOT compile everything. Of course, if you're serious about instant startup from zero then you end up going the native-image route, but then what value is WASM adding? What's being shipped to the edge in that case is a compiled binary anyway. There is some scope here for threading the needle - for example, an edge worker implementation could be given compiled bytecode of some kind and then do the compilation and dumping process server side. This is a bit like how Apple compiled LLVM bitcode itself for delivery. However then the build process needs to be sandboxed and at that point why not go full FaaS and let users upload whatever binaries they like? AWS is doing it successfully.
rapnie 3y ago
> This assumption that WASM is a competitor to Docker doesn't seem very sensible and it's hard to understand where it comes from. Exactly. WebAssembly is complementary. Docker has Wasm beta support [0]. See: Announcing Docker+Wasm Technical Preview 2 [1] (Mar 22, 2023) [0] https://docs.docker.com/desktop/wasm/ https://docs.docker.com/desktop/wasm/ [1] https://www.docker.com/blog/announcing-dockerwasm-technical-preview-2/ https://www.docker.com/blog/announcing-dockerwasm-technical-...
no_wizard 3y ago
WASM isn't a new idea. I will say, the one thing that WASM has "right" over say, the JVM, Roslyn, or other portable runtimes is the fact you can share WASM modules across language boundaries. I can't drop in use a Rust module in my Go code, at minimum, I'd need to dip into some kind of FFI. With WASM, you can just import the WASM module for usage into your language, as long as it supports WASM, regardless of original source. Its a portable format and runtime. That is the key difference than VMs of the past to me at least. It also makes sharing code that was written in say, C or C++ easier and safer too. I agree that WASM replacing Docker is something I can't "see" right now, because its just very different, but I do think the highlights above are pretty great. EDIT: Actually, I think I understand the point of WASM replacing Docker potentially which is, if WASM runtimes support the underlying operating systems sufficiently (like virtualization has to) then you could just compile what would be a container into a WASM module. IE, I could compile Postgres, Redis etc. to WASM (or more importantly, the projects could) and simply use their WASM binaries to support my application. Thats what I think they're going for here. Its hard to "see" it because there are so many unknowns (WASI doesn't have all the features currently needed for this to happen I don't think, for instance)
- torginus 3y ago
  I believe that's a bit strained as well - I'm pretty sure if you want to use a Rust module in Go, you also need to follow a standard calling convention analogous to cdecl in C - you'll have a hard time supporting any higher level feature across languages. The incompatibility on higher level constructs seems like much more of a bugbear to resolve. - for example, almost all popular high level languages have some idea of garbage collection/reference counting. There's a WASM GC proposal, I'm not sure how can they make it play nice with every languages idea/semantics of garbage collection as many languages place their own particular set of restrictions on GC like interior pointers/pinning/copying etc. some of which might be mutually exclusive. I could go on, but my point is that I believe there always needs to be an FFI between languages, that is usually barely more expressive than what a protobuf contract/C API would be (and we already have those)
- o1y32 3y ago
  JVM isn't bound to language either -- it can run bytecode generated by another language, for like, decades. But I'm afraid it has never received mainstream attention.
titzer 3y ago
> Startup time of any WASM program is going to be dominated by the fact that you're in the interpreter the whole time. The rest of your comment I agree with, but this isn't really true. One of the advantages of engines/platforms like wasmtime is that they do indeed do AOT compilation and transparently cache JITed modules. Few Wasm runtimes have interpreters. So the fast startup that you get on the Fastly platform is primarily because Wasm modules are precompiled and instantiation has been heavily optimized. I think overall that Wasm and containers solve mostly orthogonal problems, but there is some overlap in the serverless space. If programs are small and have few dependencies that can be inlined into a Wasm module, Wasm will do well against containers. If you have a very large software stack that has a lot of stuff in the container or base image, Wasm won't be as competitive; at least yet.
- mike_hearn 3y ago
  I suppose the question is why do the compilation server side? With a Linux sandbox policy as restrictive as WASI is, like by blocking all but a handful of syscalls, is there much difference between native machine code and WASM from the cloud operator perspective? Is maybe the issue here that low level Linux sandboxing with bpf and stuff isn't well documented, or maybe not trusted?
  - titzer 3y ago
    For one thing, Wasm gives CFI out of the box, so binaries you run are not vulnerable to RCE. Also, it's portable, so you can run on, e.g. arm servers.

ilaksh 3y ago

WASM can be pre-compiled I assume. A basic use case for moving from containers to WASM is to provide the option for custom backend code for users in a more lightweight way. For example, I have a site that allows procedural image generation via an API and I originally built it using containers. It could be much less resource-intensive and faster startup if I could run that code in web assembly. Or, I was building a generative AI site that creates custom web applications. In order to allow backend code, I set it up so everyone gets their own fly.io VM/container(s). This is very flexible but not very fast startup time and the smallest option is like 128MB or 256MB RAM which needs to stay loaded otherwise they will have to wait for another restart. I never really needed to provide arbitrary Linux capabilities, just sandboxed backend code. So if I do something like that again I will try to use web assembly instead. Probably with lightweight TinyGo or Rust-based request handlers or something like that.

paulgb 3y ago

I mostly agree with you, but I think I can shed some light here: > As programs get larger they do things like load config files, connect to databases, initialize in-memory caches, possibly reflect themselves because that's often convenient, and so on. Because it's all init code none of this is hot, so it gets interpreted and that's slow. One of the most interesting aspects of Wasm is that because it is built with isolation in mind, it's really easy to get a snapshot and run it later. So instead of doing all the cache initialization etc. at init time, you can initialize the module at build time and then take a snapshot of it for a very quick cold start next time. When running in the browser, there is still an initialization step to process the wasm bytecode, but in the server-side world this can also be done in advance since you can trust the compiler and know in advance which processor architecture you will want to run it on.

mike_hearn 3y ago
Right, but that trick also isn't new. emacs was doing it decades ago and there are JVMs that also do it. That's how GraalVM native images can start and run faster than C programs do: the app is initialized at build time and then the state is just mmapped into a pre-compiled native process. HotSpot has a less aggressive version of this where you can pre-initialize a lot of VM state and dump it to a file for faster startup time. It's about a 30% win and that's going up over time, but it doesn't AOT compile everything. Of course, if you're serious about instant startup from zero then you end up going the native-image route, but then what value is WASM adding? What's being shipped to the edge in that case is a compiled binary anyway. There is some scope here for threading the needle - for example, an edge worker implementation could be given compiled bytecode of some kind and then do the compilation and dumping process server side. This is a bit like how Apple compiled LLVM bitcode itself for delivery. However then the build process needs to be sandboxed and at that point why not go full FaaS and let users upload whatever binaries they like? AWS is doing it successfully.

rapnie 3y ago

> This assumption that WASM is a competitor to Docker doesn't seem very sensible and it's hard to understand where it comes from. Exactly. WebAssembly is complementary. Docker has Wasm beta support [0]. See: Announcing Docker+Wasm Technical Preview 2 [1] (Mar 22, 2023) [0] https://docs.docker.com/desktop/wasm/ https://docs.docker.com/desktop/wasm/ [1] https://www.docker.com/blog/announcing-dockerwasm-technical-preview-2/ https://www.docker.com/blog/announcing-dockerwasm-technical-...

no_wizard 3y ago

WASM isn't a new idea. I will say, the one thing that WASM has "right" over say, the JVM, Roslyn, or other portable runtimes is the fact you can share WASM modules across language boundaries. I can't drop in use a Rust module in my Go code, at minimum, I'd need to dip into some kind of FFI. With WASM, you can just import the WASM module for usage into your language, as long as it supports WASM, regardless of original source. Its a portable format and runtime. That is the key difference than VMs of the past to me at least. It also makes sharing code that was written in say, C or C++ easier and safer too. I agree that WASM replacing Docker is something I can't "see" right now, because its just very different, but I do think the highlights above are pretty great. EDIT: Actually, I think I understand the point of WASM replacing Docker potentially which is, if WASM runtimes support the underlying operating systems sufficiently (like virtualization has to) then you could just compile what would be a container into a WASM module. IE, I could compile Postgres, Redis etc. to WASM (or more importantly, the projects could) and simply use their WASM binaries to support my application. Thats what I think they're going for here. Its hard to "see" it because there are so many unknowns (WASI doesn't have all the features currently needed for this to happen I don't think, for instance)

torginus 3y ago
I believe that's a bit strained as well - I'm pretty sure if you want to use a Rust module in Go, you also need to follow a standard calling convention analogous to cdecl in C - you'll have a hard time supporting any higher level feature across languages. The incompatibility on higher level constructs seems like much more of a bugbear to resolve. - for example, almost all popular high level languages have some idea of garbage collection/reference counting. There's a WASM GC proposal, I'm not sure how can they make it play nice with every languages idea/semantics of garbage collection as many languages place their own particular set of restrictions on GC like interior pointers/pinning/copying etc. some of which might be mutually exclusive. I could go on, but my point is that I believe there always needs to be an FFI between languages, that is usually barely more expressive than what a protobuf contract/C API would be (and we already have those)
o1y32 3y ago
JVM isn't bound to language either -- it can run bytecode generated by another language, for like, decades. But I'm afraid it has never received mainstream attention.

titzer 3y ago

> Startup time of any WASM program is going to be dominated by the fact that you're in the interpreter the whole time. The rest of your comment I agree with, but this isn't really true. One of the advantages of engines/platforms like wasmtime is that they do indeed do AOT compilation and transparently cache JITed modules. Few Wasm runtimes have interpreters. So the fast startup that you get on the Fastly platform is primarily because Wasm modules are precompiled and instantiation has been heavily optimized. I think overall that Wasm and containers solve mostly orthogonal problems, but there is some overlap in the serverless space. If programs are small and have few dependencies that can be inlined into a Wasm module, Wasm will do well against containers. If you have a very large software stack that has a lot of stuff in the container or base image, Wasm won't be as competitive; at least yet.

mike_hearn 3y ago
I suppose the question is why do the compilation server side? With a Linux sandbox policy as restrictive as WASI is, like by blocking all but a handful of syscalls, is there much difference between native machine code and WASM from the cloud operator perspective? Is maybe the issue here that low level Linux sandboxing with bpf and stuff isn't well documented, or maybe not trusted?
- titzer 3y ago
  For one thing, Wasm gives CFI out of the box, so binaries you run are not vulnerable to RCE. Also, it's portable, so you can run on, e.g. arm servers.