3 ms·

Is this really an alternative to Emscripten? It the README makes it seem 100% Python focused. I want to switch from Emscripten to a more maintainable, modulare

by jitl 4y ago

Is this really an alternative to Emscripten? It the README makes it seem 100% Python focused. I want to switch from Emscripten to a more maintainable, modulare WASM build tool - how would I go about porting a simple C->WASM w/ Typescript library project to CoWasm? Does CoWasm support asyncify?

mkl 4y ago
Not yet, and it's not clear that's the goal, exactly. Up near the top, the README says CoWasm "goes far beyond just Python". Then later: "The goal of CoWasm is overall similar to all of emscripten, WebAssembly.sh, wapm.io, and Pyodide in various ways." But the "How does CoWasm compare to Emscripten and Pyodide?" section doesn't mention Emscripten.
- williamstein 4y ago
  Thanks. The docs could use some work! (https://github.com/sagemathinc/cowasm/issues/41 https://github.com/sagemathinc/cowasm/issues/41)
williamstein 4y ago
I am using the Python ecosystem (with full support for dynamic loading of C extension modules) as an initial motivating project. Also, the Python test suite is extremely useful to root out problems. I certainly hope that this can provide a more complete alternative to Emscripten to the community eventually. That said, Emscripten is huge, and the problems involved in creating a more maintainable modular WASM build tool are subtle. For example, when implementing a custom module loader for Python-wasm last week, I discovered several bugs in the memfs and unionfs Javascript libraries (https://github.com/streamich/memfs/pulls?q=is%3Apr+author%3Awilliamstein https://github.com/streamich/memfs/pulls?q=is%3Apr+author%3A... and https://github.com/streamich/unionfs/pulls?q=is%3Apr+author%3Awilliamstein https://github.com/streamich/unionfs/pulls?q=is%3Apr+author%...). I had to learn the code sufficiently to fix all these bugs, submit PR's, etc. Emscripten has its own analogue of memfs, which is optimized specifically for WebAssembly in the browser, where memfs is a more general widely used library (with 10M+ downloads/week). CoWasm has no support for asyncify. Where I've run into setjmp/longjmp so far, I've been rewriting the code instead. E.g., the dash shell uses setjmp/longjmp, and I'm rewriting that to use return error codes instead (see, e.g., https://github.com/sagemathinc/dash/commit/7117e1f6496728af04bca19cf5434a55963b246f https://github.com/sagemathinc/dash/commit/7117e1f6496728af0...). > how would I go about porting a simple C->WASM w/ Typescript library project to CoWasm? That's a great question, which I'm not sure how to quickly answer, so I've created a discussion item here https://github.com/sagemathinc/cowasm/discussions/40 https://github.com/sagemathinc/cowasm/discussions/40 I did a massive refactor of the codebase during the last month, and it makes doing what you want much easier. However, it will help a lot once there are some good examples. I'm probably boing to individually package some of the things I've ported for CoWasm as standalone examples. In any case, CoWasm is extremely open source (BSD 3-clause), and I strongly encourage anybody who is interested to grab a copy, build it, poke around, make a PR, etc.
- ridruejo 4y ago
  Thanks for all the effort you have put into this. It is pretty cool and enormously valuable to have an alternative to Emscripten
flohofwoe 4y ago
This is a slim alternative to Emscripten which focuses only on the C/C++ <=> JS interoperability part: https://github.com/schellingb/wajic https://github.com/schellingb/wajic I don't think there are actual Emscripten replacements though which give you all the features, but in a 'cleaner' package. Emscripten does a lot on top of clang/llvm. A lot of the 'Emscripten magic' has been moved into the Binaryen tools though, so it can be used by other toolchains (like wajic): https://github.com/WebAssembly/binaryen https://github.com/WebAssembly/binaryen.
- jitl 4y ago
  It's great that eg asyncify is upstreamed to binaryen/LLVM. My issue with Emscripten is that it's quite challenging to fix bugs in the wrapper Javascript code it generates. I spent a few nights trying to understand how to change the compiler output so I could import the Emscripten result as an ES6 module in both a browser and Node 16+. The way this works is unfortunately 1000s of lines of `if` statements that decide whether or not to concatenate this or that helper into a file, and all the little helper snippets may or may not call into each other. Some of the concatenation is in Python, some of it is in Javascript. I eventually gave up without making any upstreamable progress. Relevant github issue: https://github.com/emscripten-core/emscripten/issues/11792 https://github.com/emscripten-core/emscripten/issues/11792 A quote from that thread: > Btw, this issue shouldn't be tagged good-first-bug, because fixing it will require a substantial overhaul of how Emscripten sets up its imports and organises its startup code. There are several ways it can be accomplished, but it really needs direction from one of the project leads. It doesn't seem like there's any progress here :(

by jitl 4y ago

mkl 4y ago

Not yet, and it's not clear that's the goal, exactly. Up near the top, the README says CoWasm "goes far beyond just Python". Then later: "The goal of CoWasm is overall similar to all of emscripten, WebAssembly.sh, wapm.io, and Pyodide in various ways." But the "How does CoWasm compare to Emscripten and Pyodide?" section doesn't mention Emscripten.

williamstein 4y ago
Thanks. The docs could use some work! (https://github.com/sagemathinc/cowasm/issues/41 https://github.com/sagemathinc/cowasm/issues/41)

williamstein 4y ago

I am using the Python ecosystem (with full support for dynamic loading of C extension modules) as an initial motivating project. Also, the Python test suite is extremely useful to root out problems. I certainly hope that this can provide a more complete alternative to Emscripten to the community eventually. That said, Emscripten is huge, and the problems involved in creating a more maintainable modular WASM build tool are subtle. For example, when implementing a custom module loader for Python-wasm last week, I discovered several bugs in the memfs and unionfs Javascript libraries (https://github.com/streamich/memfs/pulls?q=is%3Apr+author%3Awilliamstein https://github.com/streamich/memfs/pulls?q=is%3Apr+author%3A... and https://github.com/streamich/unionfs/pulls?q=is%3Apr+author%3Awilliamstein https://github.com/streamich/unionfs/pulls?q=is%3Apr+author%...). I had to learn the code sufficiently to fix all these bugs, submit PR's, etc. Emscripten has its own analogue of memfs, which is optimized specifically for WebAssembly in the browser, where memfs is a more general widely used library (with 10M+ downloads/week). CoWasm has no support for asyncify. Where I've run into setjmp/longjmp so far, I've been rewriting the code instead. E.g., the dash shell uses setjmp/longjmp, and I'm rewriting that to use return error codes instead (see, e.g., https://github.com/sagemathinc/dash/commit/7117e1f6496728af04bca19cf5434a55963b246f https://github.com/sagemathinc/dash/commit/7117e1f6496728af0...). > how would I go about porting a simple C->WASM w/ Typescript library project to CoWasm? That's a great question, which I'm not sure how to quickly answer, so I've created a discussion item here https://github.com/sagemathinc/cowasm/discussions/40 https://github.com/sagemathinc/cowasm/discussions/40 I did a massive refactor of the codebase during the last month, and it makes doing what you want much easier. However, it will help a lot once there are some good examples. I'm probably boing to individually package some of the things I've ported for CoWasm as standalone examples. In any case, CoWasm is extremely open source (BSD 3-clause), and I strongly encourage anybody who is interested to grab a copy, build it, poke around, make a PR, etc.

ridruejo 4y ago
Thanks for all the effort you have put into this. It is pretty cool and enormously valuable to have an alternative to Emscripten

flohofwoe 4y ago

This is a slim alternative to Emscripten which focuses only on the C/C++ <=> JS interoperability part: https://github.com/schellingb/wajic https://github.com/schellingb/wajic I don't think there are actual Emscripten replacements though which give you all the features, but in a 'cleaner' package. Emscripten does a lot on top of clang/llvm. A lot of the 'Emscripten magic' has been moved into the Binaryen tools though, so it can be used by other toolchains (like wajic): https://github.com/WebAssembly/binaryen https://github.com/WebAssembly/binaryen.

jitl 4y ago
It's great that eg asyncify is upstreamed to binaryen/LLVM. My issue with Emscripten is that it's quite challenging to fix bugs in the wrapper Javascript code it generates. I spent a few nights trying to understand how to change the compiler output so I could import the Emscripten result as an ES6 module in both a browser and Node 16+. The way this works is unfortunately 1000s of lines of `if` statements that decide whether or not to concatenate this or that helper into a file, and all the little helper snippets may or may not call into each other. Some of the concatenation is in Python, some of it is in Javascript. I eventually gave up without making any upstreamable progress. Relevant github issue: https://github.com/emscripten-core/emscripten/issues/11792 https://github.com/emscripten-core/emscripten/issues/11792 A quote from that thread: > Btw, this issue shouldn't be tagged good-first-bug, because fixing it will require a substantial overhaul of how Emscripten sets up its imports and organises its startup code. There are several ways it can be accomplished, but it really needs direction from one of the project leads. It doesn't seem like there's any progress here :(