5 ms·

CoWasm: An alternative to Emscripten, based on Zig (demo: Python in the browser)

130 points by BlackFingolfin 4y ago

10241024 4y ago
Duplicate of this? https://news.ycombinator.com/item?id=33370434 https://news.ycombinator.com/item?id=33370434
- kiwicopple 4y ago
  looks like a duplicate, but it has no discussion. I'm not OP of either of these but I'd love for this to get to the top to foster a discussion about their approach (which seems exciting)
kiwicopple 4y ago
Some of the interesting goals from the repo: https://github.com/sagemathinc/cowasm https://github.com/sagemathinc/cowasm: - There is no business unfriendly GPL code in CoWasm. CoWasm itself is extremely liberally licensed and business friendly. - Unlike emscripten, we use modern Typescript, our code is more modular, and we make use of existing components when possible (e.g., the nodejs memfs project), instead of using our own. - A core design constraint is to efficiently run on a wide range of platforms, not mainly in the browser like emscripten, and not mainly on servers like wasmer. CoWasm should run on servers, desktops (e.g., as an electron app), an iPad/iOS app, and in web browsers.
- pjmlp 4y ago
  > There is no business unfriendly GPL code in CoWasm. CoWasm itself is extremely liberally licensed and business friendly. This is one to save for posterity, when they eventually change the license because the likes of Amazon decided to make a product out of it.
williamstein 4y ago
I started this project and can answer any questions about CoWasm.
- bobuk 4y ago
  no questions, but please keep working on that. I'm very impressed with how smoothly the current cowasm implementation is working. For the future do your have a plans to support WASI and other modern WebAssembly stuff (like WebAssembly Component Model support f.e)?
  - williamstein 4y ago
    CoWasm supports WASI right now via this library https://www.npmjs.com/package/wasi-js https://www.npmjs.com/package/wasi-js, which I actually develop as part of CoWasm . One unusually thing I did, which goes beyond what emscripten does, is I implemented a quite a bit of posix functionality, often by writing extension code to nodejs and calling it from Javascript, because there's a lot of POSIX that Node.js doesn't expose. This only works on Mac and Linux and is also available standalone in this library https://www.npmjs.com/package/posix-node https://www.npmjs.com/package/posix-node, which is implemented in Zig. You can get a sense of the scope of POSIX functionality that goes beyond what WASI defines here: https://github.com/sagemathinc/cowasm/tree/main/packages/kernel/src/wasm/posix https://github.com/sagemathinc/cowasm/tree/main/packages/ker... One motivation for doing this is to try to get the full Python test suite to pass, including all the functionality that involves subprocesses, posix calls, etc. I've only got to about 85% at this point. It can be a ton of tedious work, but at least Zig helps impose some discipline (e.g, it doesn't let you ignore handling errors until later), and makes it easy to test compilation for all supported targets on every change (due to excellent cross compilation support).
- boothby 4y ago
  This is really cool, William! A lifetime ago, we were pondering what it would take to get Sage running in the browser from a static server. It feels like you're nearly there. Is that in the plans?
  - williamstein 4y ago
    Yes, porting https://SageMath.org https://SageMath.org to the browser is part of the plan, and this is a key foundation for that. I ported some of the components of Sage (e.g., https://libntl.org/ https://libntl.org/) already as part of https://github.com/sagemathinc/jsage https://github.com/sagemathinc/jsage, and I don't see any fundamental obstruction to porting all of Sage, except time. It would be great if once CoWasm is more stable, I can get some help porting some components to WASM.
jitl 4y ago
Is this really an alternative to Emscripten? It the README makes it seem 100% Python focused. I want to switch from Emscripten to a more maintainable, modulare WASM build tool - how would I go about porting a simple C->WASM w/ Typescript library project to CoWasm? Does CoWasm support asyncify?
- mkl 4y ago
  Not yet, and it's not clear that's the goal, exactly. Up near the top, the README says CoWasm "goes far beyond just Python". Then later: "The goal of CoWasm is overall similar to all of emscripten, WebAssembly.sh, wapm.io, and Pyodide in various ways." But the "How does CoWasm compare to Emscripten and Pyodide?" section doesn't mention Emscripten.
  - williamstein 4y ago
    Thanks. The docs could use some work! (https://github.com/sagemathinc/cowasm/issues/41 https://github.com/sagemathinc/cowasm/issues/41)
- williamstein 4y ago
  I am using the Python ecosystem (with full support for dynamic loading of C extension modules) as an initial motivating project. Also, the Python test suite is extremely useful to root out problems. I certainly hope that this can provide a more complete alternative to Emscripten to the community eventually. That said, Emscripten is huge, and the problems involved in creating a more maintainable modular WASM build tool are subtle. For example, when implementing a custom module loader for Python-wasm last week, I discovered several bugs in the memfs and unionfs Javascript libraries (https://github.com/streamich/memfs/pulls?q=is%3Apr+author%3Awilliamstein https://github.com/streamich/memfs/pulls?q=is%3Apr+author%3A... and https://github.com/streamich/unionfs/pulls?q=is%3Apr+author%3Awilliamstein https://github.com/streamich/unionfs/pulls?q=is%3Apr+author%...). I had to learn the code sufficiently to fix all these bugs, submit PR's, etc. Emscripten has its own analogue of memfs, which is optimized specifically for WebAssembly in the browser, where memfs is a more general widely used library (with 10M+ downloads/week). CoWasm has no support for asyncify. Where I've run into setjmp/longjmp so far, I've been rewriting the code instead. E.g., the dash shell uses setjmp/longjmp, and I'm rewriting that to use return error codes instead (see, e.g., https://github.com/sagemathinc/dash/commit/7117e1f6496728af04bca19cf5434a55963b246f https://github.com/sagemathinc/dash/commit/7117e1f6496728af0...). > how would I go about porting a simple C->WASM w/ Typescript library project to CoWasm? That's a great question, which I'm not sure how to quickly answer, so I've created a discussion item here https://github.com/sagemathinc/cowasm/discussions/40 https://github.com/sagemathinc/cowasm/discussions/40 I did a massive refactor of the codebase during the last month, and it makes doing what you want much easier. However, it will help a lot once there are some good examples. I'm probably boing to individually package some of the things I've ported for CoWasm as standalone examples. In any case, CoWasm is extremely open source (BSD 3-clause), and I strongly encourage anybody who is interested to grab a copy, build it, poke around, make a PR, etc.

Duplicate of this? https://news.ycombinator.com/item?id=33370434 https://news.ycombinator.com/item?id=33370434

kiwicopple 4y ago
looks like a duplicate, but it has no discussion. I'm not OP of either of these but I'd love for this to get to the top to foster a discussion about their approach (which seems exciting)

Some of the interesting goals from the repo: https://github.com/sagemathinc/cowasm https://github.com/sagemathinc/cowasm: - There is no business unfriendly GPL code in CoWasm. CoWasm itself is extremely liberally licensed and business friendly. - Unlike emscripten, we use modern Typescript, our code is more modular, and we make use of existing components when possible (e.g., the nodejs memfs project), instead of using our own. - A core design constraint is to efficiently run on a wide range of platforms, not mainly in the browser like emscripten, and not mainly on servers like wasmer. CoWasm should run on servers, desktops (e.g., as an electron app), an iPad/iOS app, and in web browsers.

pjmlp 4y ago
> There is no business unfriendly GPL code in CoWasm. CoWasm itself is extremely liberally licensed and business friendly. This is one to save for posterity, when they eventually change the license because the likes of Amazon decided to make a product out of it.

williamstein 4y ago

I started this project and can answer any questions about CoWasm.

bobuk 4y ago
no questions, but please keep working on that. I'm very impressed with how smoothly the current cowasm implementation is working. For the future do your have a plans to support WASI and other modern WebAssembly stuff (like WebAssembly Component Model support f.e)?
- williamstein 4y ago
  CoWasm supports WASI right now via this library https://www.npmjs.com/package/wasi-js https://www.npmjs.com/package/wasi-js, which I actually develop as part of CoWasm . One unusually thing I did, which goes beyond what emscripten does, is I implemented a quite a bit of posix functionality, often by writing extension code to nodejs and calling it from Javascript, because there's a lot of POSIX that Node.js doesn't expose. This only works on Mac and Linux and is also available standalone in this library https://www.npmjs.com/package/posix-node https://www.npmjs.com/package/posix-node, which is implemented in Zig. You can get a sense of the scope of POSIX functionality that goes beyond what WASI defines here: https://github.com/sagemathinc/cowasm/tree/main/packages/kernel/src/wasm/posix https://github.com/sagemathinc/cowasm/tree/main/packages/ker... One motivation for doing this is to try to get the full Python test suite to pass, including all the functionality that involves subprocesses, posix calls, etc. I've only got to about 85% at this point. It can be a ton of tedious work, but at least Zig helps impose some discipline (e.g, it doesn't let you ignore handling errors until later), and makes it easy to test compilation for all supported targets on every change (due to excellent cross compilation support).
boothby 4y ago
This is really cool, William! A lifetime ago, we were pondering what it would take to get Sage running in the browser from a static server. It feels like you're nearly there. Is that in the plans?
- williamstein 4y ago
  Yes, porting https://SageMath.org https://SageMath.org to the browser is part of the plan, and this is a key foundation for that. I ported some of the components of Sage (e.g., https://libntl.org/ https://libntl.org/) already as part of https://github.com/sagemathinc/jsage https://github.com/sagemathinc/jsage, and I don't see any fundamental obstruction to porting all of Sage, except time. It would be great if once CoWasm is more stable, I can get some help porting some components to WASM.

jitl 4y ago

Is this really an alternative to Emscripten? It the README makes it seem 100% Python focused. I want to switch from Emscripten to a more maintainable, modulare WASM build tool - how would I go about porting a simple C->WASM w/ Typescript library project to CoWasm? Does CoWasm support asyncify?

mkl 4y ago
Not yet, and it's not clear that's the goal, exactly. Up near the top, the README says CoWasm "goes far beyond just Python". Then later: "The goal of CoWasm is overall similar to all of emscripten, WebAssembly.sh, wapm.io, and Pyodide in various ways." But the "How does CoWasm compare to Emscripten and Pyodide?" section doesn't mention Emscripten.
- williamstein 4y ago
  Thanks. The docs could use some work! (https://github.com/sagemathinc/cowasm/issues/41 https://github.com/sagemathinc/cowasm/issues/41)
williamstein 4y ago
I am using the Python ecosystem (with full support for dynamic loading of C extension modules) as an initial motivating project. Also, the Python test suite is extremely useful to root out problems. I certainly hope that this can provide a more complete alternative to Emscripten to the community eventually. That said, Emscripten is huge, and the problems involved in creating a more maintainable modular WASM build tool are subtle. For example, when implementing a custom module loader for Python-wasm last week, I discovered several bugs in the memfs and unionfs Javascript libraries (https://github.com/streamich/memfs/pulls?q=is%3Apr+author%3Awilliamstein https://github.com/streamich/memfs/pulls?q=is%3Apr+author%3A... and https://github.com/streamich/unionfs/pulls?q=is%3Apr+author%3Awilliamstein https://github.com/streamich/unionfs/pulls?q=is%3Apr+author%...). I had to learn the code sufficiently to fix all these bugs, submit PR's, etc. Emscripten has its own analogue of memfs, which is optimized specifically for WebAssembly in the browser, where memfs is a more general widely used library (with 10M+ downloads/week). CoWasm has no support for asyncify. Where I've run into setjmp/longjmp so far, I've been rewriting the code instead. E.g., the dash shell uses setjmp/longjmp, and I'm rewriting that to use return error codes instead (see, e.g., https://github.com/sagemathinc/dash/commit/7117e1f6496728af04bca19cf5434a55963b246f https://github.com/sagemathinc/dash/commit/7117e1f6496728af0...). > how would I go about porting a simple C->WASM w/ Typescript library project to CoWasm? That's a great question, which I'm not sure how to quickly answer, so I've created a discussion item here https://github.com/sagemathinc/cowasm/discussions/40 https://github.com/sagemathinc/cowasm/discussions/40 I did a massive refactor of the codebase during the last month, and it makes doing what you want much easier. However, it will help a lot once there are some good examples. I'm probably boing to individually package some of the things I've ported for CoWasm as standalone examples. In any case, CoWasm is extremely open source (BSD 3-clause), and I strongly encourage anybody who is interested to grab a copy, build it, poke around, make a PR, etc.