4 ms·

Your post got me wondering, what advantage might it provide over an FFI? Does WASM ABI define higher level common primitives than C?

lesuorac 3y ago
If it ends up becoming more similar to LUA then a big advantage is that the WASM code won't randomly read your harddrive and send your bitcoins to North Korea unless you explicitly gave the WASM code disk/network permissions.
- brabel 3y ago
  Lua allows full control over which APIs you want to expose to a script you embed in your application. With some effort you can even expose only a constrained version of the `os` module for example which only lets you access a few resources. Why do you believe WASM can do better here? In fact, as far as I know , there's nothing in WASM that lets you sandbox it yet once you've given it WASI access (unless you're talking about host specific features, which are NOT part of WASM spec itself).
  - josephg 3y ago
    Yeah lua is a weird example because it’s actually amazing at this. Lua gives you a massive amount of control over what scripts can access. It is much safer than pulling in an opaque C library that works via ffi. Eg a nodejs native module. Those are written in C and can indeed sell your data to North Korea. (Just like any other package in npm.) I’m excited by the idea of being able to depend on 3rd party code without it having access to my entire OS.
  - ncruces 3y ago
    That depends entirely on the runtime, and its WASI implementation. wazero [1], which I'm most familiar with, allows you to decide in a relatively fine-grained way what capabilities your WASI module will have: command line arguments, environment variables, stdin/out/err, monotonic/wall clock, sleeping, even yielding CPU… Maybe more importantly, filesystem access can be fully emulated, or sandboxed to a specific folder, or have some directories mounted read-only, etc; it's very much up to you. I've used it to wrap command line utilities, and package them as Go libraries. For one example, dcraw [2]. WASM makes this memory safe, and I can sandbox it to access only the single file I want it to process (which can be a memory buffer, or something in blob storage, if I want it to). Notice in [3] how you provide an io.ReadSeeker which could be anything from a file, a buffer in memory, or an HTTP resource. The spaghetti C that dcraw is made of won't be able to access any other file, bring your server down, etc. 1: https://wazero.io/ https://wazero.io/ 2: https://dechifro.org/dcraw/ https://dechifro.org/dcraw/ 3: https://pkg.go.dev/github.com/ncruces/rethinkraw/pkg/dcraw https://pkg.go.dev/github.com/ncruces/rethinkraw/pkg/dcraw
    - emmanueloga_ 3y ago
      Wazero looks super cool. I saw somewhere that programs can be run with a timeout, which sounds great for sandboxing. The program input is just a slice of bytes [1], so an interesting use case would be to use something like Nats [2] to distribute programs to different servers. Super simple distributed computing! -- 1: https://github.com/tetratelabs/wazero/blob/main/examples/basic/add.go#L45C38-L45C38 https://github.com/tetratelabs/wazero/blob/main/examples/bas... 2: https://natsbyexample.com/examples/messaging/pub-sub/go https://natsbyexample.com/examples/messaging/pub-sub/go
      - ncruces 3y ago
        Yes: if so configured, wazero respects context cancelation (including, but not limited to, timeouts). This has a slight toll on performance: a call back from WASM AOT-compiled-assembly into Go is introduced regularly (on every backwards jump?) to give the Go runtime the opportunity to yield the goroutine and update the context (and break infinite loops), even when GOMAXPROCS=1. Coordinating with the Go scheduler might be an area where there's some room for improvement, if fact.
nonethewiser 3y ago
I had to look up FFI (Foreign Function Interface). Im not sure if WASM is better. Im aware there are language bindings (maybe synonymous or overlaps with FFI?) but Im not that familiar with them either. I wondered if perhaps this WASM use case for a cross-language library was already just as possible and ergonomic using language bindings and maybe thats why this use case doesnt seem like a big deal to people. It does seem possible that the allure of running in the browser might prompt deeper support for WASM compilation than language bindings. The WASM case is also a many-to-one relationship (all languages to WASM) whereas language bindings are a many-to-many relationship (all languages to all languages) so it would take a lot mote effort for the same level of support.
- josephg 3y ago
  > I wondered if perhaps this WASM use case for a cross-language library was already just as possible and ergonomic using language bindings and maybe thats why this use case doesnt seem like a big deal to people. Yeah that’s the reason. You don’t notice it a lot of the time, but FFIs are everywhere already. The most common foreign function interface is basically the ability to call C code, or have functions made available to C code. C is used because everyone knows it and it’s simple. And most languages either compile to native code (eg rust) - which makes linking to C code easy. Or the runtime is implemented in C or C++ (eg V8, Ruby). In languages like that, the standard library is already basically implemented via a FFI to C/C++ code. I’ve got an iOS app I’m working on that’s half rust and half swift, with a touch of C in the middle. The bindings work great - the whole thing links together into one binary, even with link time optimizations. But the glue code is gross, and when I want to fiddle with the rust to Swift API I need to change my code in about 4 different places. Most FFIs are a one to many relationship in that if you write a clean C API, you can probably write bindings in every language. But you don’t actually want to call naked C code from Ruby or Javascript. Good bindings will make you forget everything is done via ffi. Eg numpy. I haven’t looked at the wasm component model proposal - I assume it’s trying to make this process cleaner, which sounds lovely. I maintain the nodejs bindings for foundationdb. Foundationdb bindings are all done via ffi linking to their C code. And the API is complex, using promises and things. I find it really interesting browsing their official bindings to go, Java, Python and Ruby. Same bindings. Same wrapped api. Same team of authors. Just different languages. And that’s enough to make the wrapper wildly different in every language. From memory the Java ffi wrapper is 4x as much code as it is in Ruby. https://github.com/apple/foundationdb/tree/main/bindings https://github.com/apple/foundationdb/tree/main/bindings
coderedart 3y ago
well, for starters, wasm is sandboxed. So, if a wasm library needs an import (eg: read/write filesystem), it has to be explicitly provided. It cannot do anything except math by default. This allows host a high amount of control. different wasm libraries can have separate memories. So, if a library X depends on a jpeg decoder library, the host has to provide that as import. The jpeg decoder library might export a fn called "decode" which takes an array of bytes of a jpeg, and returns an Image struct with rgba pixels. This allows the "memory" of the two libraries to be separate. the jpeg decoder cannot "write" to the X's memory, cleanly separating the two of them. Wasm component model recognizes higher level objects called resources, which can be shared between libraries (components). This allows X to simply pass a file descriptor to jpeg decode fn, and the sandbox model makes sure that jpeg library can read from that file only and the rest of the filesystem is still offlimits. wasm is already getting support for garbage collector. So, a high level language can just rely on wasm's GC and avoid shipping its entire runtime. Or the host can guarantee that all the imports a language needs will be provided, so that the language libraries can shed as much runtime weight as possible. Finally, Component model is designed from ground up to be modular, which allows imports/exports/namespaces and other such modern features. C.. well, only has headers and usually brings a lot of backwards compatibility baggage. The tooling (eg: wit-bindgen) will provide higher level support like generating code for different language bindings by taking a wit (header for wasm) declaration file. If you are familiar with rust, then https://github.com/bytecodealliance/cargo-component#getting-started https://github.com/bytecodealliance/cargo-component#getting-... shows how easy it is to safely create (or bind to) wasm bindings

lesuorac 3y ago

If it ends up becoming more similar to LUA then a big advantage is that the WASM code won't randomly read your harddrive and send your bitcoins to North Korea unless you explicitly gave the WASM code disk/network permissions.

brabel 3y ago
Lua allows full control over which APIs you want to expose to a script you embed in your application. With some effort you can even expose only a constrained version of the `os` module for example which only lets you access a few resources. Why do you believe WASM can do better here? In fact, as far as I know , there's nothing in WASM that lets you sandbox it yet once you've given it WASI access (unless you're talking about host specific features, which are NOT part of WASM spec itself).
- josephg 3y ago
  Yeah lua is a weird example because it’s actually amazing at this. Lua gives you a massive amount of control over what scripts can access. It is much safer than pulling in an opaque C library that works via ffi. Eg a nodejs native module. Those are written in C and can indeed sell your data to North Korea. (Just like any other package in npm.) I’m excited by the idea of being able to depend on 3rd party code without it having access to my entire OS.
- ncruces 3y ago
  That depends entirely on the runtime, and its WASI implementation. wazero [1], which I'm most familiar with, allows you to decide in a relatively fine-grained way what capabilities your WASI module will have: command line arguments, environment variables, stdin/out/err, monotonic/wall clock, sleeping, even yielding CPU… Maybe more importantly, filesystem access can be fully emulated, or sandboxed to a specific folder, or have some directories mounted read-only, etc; it's very much up to you. I've used it to wrap command line utilities, and package them as Go libraries. For one example, dcraw [2]. WASM makes this memory safe, and I can sandbox it to access only the single file I want it to process (which can be a memory buffer, or something in blob storage, if I want it to). Notice in [3] how you provide an io.ReadSeeker which could be anything from a file, a buffer in memory, or an HTTP resource. The spaghetti C that dcraw is made of won't be able to access any other file, bring your server down, etc. 1: https://wazero.io/ https://wazero.io/ 2: https://dechifro.org/dcraw/ https://dechifro.org/dcraw/ 3: https://pkg.go.dev/github.com/ncruces/rethinkraw/pkg/dcraw https://pkg.go.dev/github.com/ncruces/rethinkraw/pkg/dcraw
  - emmanueloga_ 3y ago
    Wazero looks super cool. I saw somewhere that programs can be run with a timeout, which sounds great for sandboxing. The program input is just a slice of bytes [1], so an interesting use case would be to use something like Nats [2] to distribute programs to different servers. Super simple distributed computing! -- 1: https://github.com/tetratelabs/wazero/blob/main/examples/basic/add.go#L45C38-L45C38 https://github.com/tetratelabs/wazero/blob/main/examples/bas... 2: https://natsbyexample.com/examples/messaging/pub-sub/go https://natsbyexample.com/examples/messaging/pub-sub/go
    - ncruces 3y ago
      Yes: if so configured, wazero respects context cancelation (including, but not limited to, timeouts). This has a slight toll on performance: a call back from WASM AOT-compiled-assembly into Go is introduced regularly (on every backwards jump?) to give the Go runtime the opportunity to yield the goroutine and update the context (and break infinite loops), even when GOMAXPROCS=1. Coordinating with the Go scheduler might be an area where there's some room for improvement, if fact.

nonethewiser 3y ago

I had to look up FFI (Foreign Function Interface). Im not sure if WASM is better. Im aware there are language bindings (maybe synonymous or overlaps with FFI?) but Im not that familiar with them either. I wondered if perhaps this WASM use case for a cross-language library was already just as possible and ergonomic using language bindings and maybe thats why this use case doesnt seem like a big deal to people. It does seem possible that the allure of running in the browser might prompt deeper support for WASM compilation than language bindings. The WASM case is also a many-to-one relationship (all languages to WASM) whereas language bindings are a many-to-many relationship (all languages to all languages) so it would take a lot mote effort for the same level of support.

josephg 3y ago
> I wondered if perhaps this WASM use case for a cross-language library was already just as possible and ergonomic using language bindings and maybe thats why this use case doesnt seem like a big deal to people. Yeah that’s the reason. You don’t notice it a lot of the time, but FFIs are everywhere already. The most common foreign function interface is basically the ability to call C code, or have functions made available to C code. C is used because everyone knows it and it’s simple. And most languages either compile to native code (eg rust) - which makes linking to C code easy. Or the runtime is implemented in C or C++ (eg V8, Ruby). In languages like that, the standard library is already basically implemented via a FFI to C/C++ code. I’ve got an iOS app I’m working on that’s half rust and half swift, with a touch of C in the middle. The bindings work great - the whole thing links together into one binary, even with link time optimizations. But the glue code is gross, and when I want to fiddle with the rust to Swift API I need to change my code in about 4 different places. Most FFIs are a one to many relationship in that if you write a clean C API, you can probably write bindings in every language. But you don’t actually want to call naked C code from Ruby or Javascript. Good bindings will make you forget everything is done via ffi. Eg numpy. I haven’t looked at the wasm component model proposal - I assume it’s trying to make this process cleaner, which sounds lovely. I maintain the nodejs bindings for foundationdb. Foundationdb bindings are all done via ffi linking to their C code. And the API is complex, using promises and things. I find it really interesting browsing their official bindings to go, Java, Python and Ruby. Same bindings. Same wrapped api. Same team of authors. Just different languages. And that’s enough to make the wrapper wildly different in every language. From memory the Java ffi wrapper is 4x as much code as it is in Ruby. https://github.com/apple/foundationdb/tree/main/bindings https://github.com/apple/foundationdb/tree/main/bindings

coderedart 3y ago

well, for starters, wasm is sandboxed. So, if a wasm library needs an import (eg: read/write filesystem), it has to be explicitly provided. It cannot do anything except math by default. This allows host a high amount of control. different wasm libraries can have separate memories. So, if a library X depends on a jpeg decoder library, the host has to provide that as import. The jpeg decoder library might export a fn called "decode" which takes an array of bytes of a jpeg, and returns an Image struct with rgba pixels. This allows the "memory" of the two libraries to be separate. the jpeg decoder cannot "write" to the X's memory, cleanly separating the two of them. Wasm component model recognizes higher level objects called resources, which can be shared between libraries (components). This allows X to simply pass a file descriptor to jpeg decode fn, and the sandbox model makes sure that jpeg library can read from that file only and the rest of the filesystem is still offlimits. wasm is already getting support for garbage collector. So, a high level language can just rely on wasm's GC and avoid shipping its entire runtime. Or the host can guarantee that all the imports a language needs will be provided, so that the language libraries can shed as much runtime weight as possible. Finally, Component model is designed from ground up to be modular, which allows imports/exports/namespaces and other such modern features. C.. well, only has headers and usually brings a lot of backwards compatibility baggage. The tooling (eg: wit-bindgen) will provide higher level support like generating code for different language bindings by taking a wit (header for wasm) declaration file. If you are familiar with rust, then https://github.com/bytecodealliance/cargo-component#getting-started https://github.com/bytecodealliance/cargo-component#getting-... shows how easy it is to safely create (or bind to) wasm bindings