5 ms·

One thing that's really great about wasm is how it enables lots of native libraries to be usable in the browser. Recently I ran into performance issues due to l

One thing that's really great about wasm is how it enables lots of native libraries to be usable in the browser. Recently I ran into performance issues due to limitations in the WebAudio API and was able to workaround it by compiling FFmpeg to wasm and using it instead. Being able to use FFmpeg in the browser is very powerful and brings a huge suite of encoding/decoding capabilities that browsers currently lack right now (at least until WebCodecs becomes standardized and implemented). On another side, compiling existing C/C++ libraries to wasm is still very painful. I struggled a lot using Emscripten to port FFmpeg to wasm - you have to compile EVERY dependency to wasm (so I ended up re-compiling libopus, libmp3lame, etc.) and the compiler warnings are often super cryptic which made the entire process a complete nightmare. The process really needs a lot of work still. For anyone that's interested, this is the result from my endeavor: https://github.com/aeroheim/audio-file-decoder https://github.com/aeroheim/audio-file-decoder And also check out a great general purpose FFmpeg wasm port here: https://github.com/Kagami/ffmpeg.js https://github.com/Kagami/ffmpeg.js

haberman 6y ago
> I struggled a lot using Emscripten to port FFmpeg to wasm - you have to compile EVERY dependency to wasm This is a scenario where the Bazel model is a good fit IMHO. Bazel rebuilds all of your dependencies from source. This makes it easy to compile all of your dependencies with an extra flag (eg -fsanitize=address for ASAN) or using a different compiler (eg Emscripten). While it can be annoying to wait for the world to rebuild every time, this is a use case where it shines.
- jez 6y ago
  Do you have an example of getting bazel to work with emscripten? I’ll share mine if you share yours: https://github.com/sorbet/sorbet/tree/master/tools/toolchain/webasm-linux https://github.com/sorbet/sorbet/tree/master/tools/toolchain... When we set this up to compile Sorbet (C++ codebase) for https://sorbet.run https://sorbet.run, it involved what I considered an inordinate amount of boilerplate and arcana. To be fair since we set it up it’s hardly ever needed to be touched, and I could probably cargo cult this into future projects where I wanted to use it, but I wouldn’t exactly say that bazel magically makes the pain of emscripten go away. Curious to hear otherwise.
  - haberman 6y ago
    Yikes. That definitely seems more complicated than it should be. I have not used Emscripten in this way before, I was going from my experience of passing custom flags like -fsanitize=address. I was under the impression that Emscripten was just an alternative compiler binary, such that you could just use CC=emcc. Is that not the case?
    - jez 6y ago
      The trick is that to provide Bazel with a custom toolchain involves way more than just setting an environment variable, because Bazel wants to control installing and making available the compiler reliably (e.g., what if `emcc` is not present on the system where Bazel was invoked? Bazel solves that problem by fetching it and building it for that system) There are projects that provide drop-in support for custom toolchains (e.g., we use this project[0] in Sorbet to fetch and build a custom LLVM/Clang toolchain for every host we build on (rather than relying on the system toolchain). But I'm not aware of a project that has done that for Emscripten. Maybe it would be as easy as plucking out what we've done in our project into a project that others could depend on, but to quote a colleague: > Setting up a cc toolchain in Bazel is a unique sort of pain. [0] https://github.com/grailbio/bazel-toolchain/ https://github.com/grailbio/bazel-toolchain/
pjmlp 6y ago
PNaCL and CrossBridge already allowed it in 2011, but alas politics, and yes Emscripten sucks. Why do I need to have a mix? of Python, Java and JS based tools in addition to the C and C++ compilers? With no option to skip downloading them.

haberman 6y ago

> I struggled a lot using Emscripten to port FFmpeg to wasm - you have to compile EVERY dependency to wasm This is a scenario where the Bazel model is a good fit IMHO. Bazel rebuilds all of your dependencies from source. This makes it easy to compile all of your dependencies with an extra flag (eg -fsanitize=address for ASAN) or using a different compiler (eg Emscripten). While it can be annoying to wait for the world to rebuild every time, this is a use case where it shines.

jez 6y ago
Do you have an example of getting bazel to work with emscripten? I’ll share mine if you share yours: https://github.com/sorbet/sorbet/tree/master/tools/toolchain/webasm-linux https://github.com/sorbet/sorbet/tree/master/tools/toolchain... When we set this up to compile Sorbet (C++ codebase) for https://sorbet.run https://sorbet.run, it involved what I considered an inordinate amount of boilerplate and arcana. To be fair since we set it up it’s hardly ever needed to be touched, and I could probably cargo cult this into future projects where I wanted to use it, but I wouldn’t exactly say that bazel magically makes the pain of emscripten go away. Curious to hear otherwise.
- haberman 6y ago
  Yikes. That definitely seems more complicated than it should be. I have not used Emscripten in this way before, I was going from my experience of passing custom flags like -fsanitize=address. I was under the impression that Emscripten was just an alternative compiler binary, such that you could just use CC=emcc. Is that not the case?
  - jez 6y ago
    The trick is that to provide Bazel with a custom toolchain involves way more than just setting an environment variable, because Bazel wants to control installing and making available the compiler reliably (e.g., what if `emcc` is not present on the system where Bazel was invoked? Bazel solves that problem by fetching it and building it for that system) There are projects that provide drop-in support for custom toolchains (e.g., we use this project[0] in Sorbet to fetch and build a custom LLVM/Clang toolchain for every host we build on (rather than relying on the system toolchain). But I'm not aware of a project that has done that for Emscripten. Maybe it would be as easy as plucking out what we've done in our project into a project that others could depend on, but to quote a colleague: > Setting up a cc toolchain in Bazel is a unique sort of pain. [0] https://github.com/grailbio/bazel-toolchain/ https://github.com/grailbio/bazel-toolchain/

pjmlp 6y ago

PNaCL and CrossBridge already allowed it in 2011, but alas politics, and yes Emscripten sucks. Why do I need to have a mix? of Python, Java and JS based tools in addition to the C and C++ compilers? With no option to skip downloading them.