4 ms·

Build Your Own WebAssembly Compiler

144 points by simonpure 7y ago

6nf 7y ago
My self hosted compiler: https://github.com/PierreRossouw/wats https://github.com/PierreRossouw/wats
Aardappel 7y ago
Documentation on how Lobster's Wasm-backend works, which contains plenty of useful info for anyone making their own backend: http://aardappel.github.io/lobster/implementation_wasm.html http://aardappel.github.io/lobster/implementation_wasm.html
- pjmlp 7y ago
  Cool, thanks for sharing.
thomasballinger 7y ago
Here's Colin's blog post with code and diagrams inline: https://blog.scottlogic.com/2019/05/17/webassembly-compiler.html https://blog.scottlogic.com/2019/05/17/webassembly-compiler.... I found Colin's posts real helpful when writing some WebAssembly by hand for the first few days of Advent of Code: https://observablehq.com/collection/@ballingt/advent-of-wasm https://observablehq.com/collection/@ballingt/advent-of-wasm
catpolice 7y ago
Last year I decided to write a WASM-targeting compiler in Javascript, completely by hand. The idea was to compile a vaguely C-like toy language to WASM without any external dependencies. Part of the motivation was that while I've written many simple parsers, I mostly used parser generators with BNF grammars and I didn't feel like I had a good sense of how the code I ended up generating actually worked for something with complex syntax and semantics. I felt like I was writing specs for a parser, rather than writing a parser. And I didn't have a huge amount of experience with code generation. My toy language has vaguely C-like syntax with block scope and infix operators with various precedences, so it was a bit more complicated than JSON, but I ended up using something like Pratt parsing/Precedence Climbing (see https://www.oilshell.org/blog/2017/03/31.html https://www.oilshell.org/blog/2017/03/31.html) and wrote the whole thing in a way that's - hopefully - pretty easy to read for folks interested in wrapping their head around parsing complex syntax (e.g. with scope and name resolution). The lexer, parser and language definition ended up being about 1000 lines of JS (counting some pretty extensive comments). Code generation is pretty straightforward once you have an AST. Any JS programmers that are interested in really getting into the nitty-gritty of writing your own parser/compiler should check it out. The source is here: https://github.com/j-s-n/WebBS https://github.com/j-s-n/WebBS. If you want to play around with the language and inspect the generated ASTs and WASM bytecode, there's an interactive IDE with example code here: https://j-s-n.github.io/WebBS/index.html#splash https://j-s-n.github.io/WebBS/index.html#splash
- vchak1 7y ago
  I found this one pretty interesting. https://maierfelix.github.io/mini-c/ https://maierfelix.github.io/mini-c/
ridiculous_fish 7y ago
Are there any fast ways to run wasm on systems that forbid creating new executable pages? A high performance interpreter?
- vbezhenar 7y ago
  What system do you mean? iOS supports wasm AFAIK.
  - jimmcslim 7y ago
    Mobile Safari may support WASM, but I think the GP is suggesting that a native iOS app would not be permitted to load, compile, and execute a WebAssembly module.
    - vbezhenar 7y ago
      I might be wrong, but I think that one should be able to utilize JavaScript engine within native iOS app which will use fast implementation.
      - nickmain 7y ago
        JavaScriptCore and JS running in a web page using WKWebView are both running in the same app process and do not support WASM since they cannot generate code (cannot write to executable memory pages). SFSafariViewController does support WASM by executing in a separate more privileged process but interaction with the app is very limited.
        saagarjha 7y ago
        WKWebView does as well because it too runs out-of-process. You can load WebAssembly into it.
        throwGuardian 7y ago
        But WKWebView is equivalent to running inside a browser, with no system access. If JSCore allowed WASM JIT-ing, one could easily port WASI to iOS for full system access, and update iOS apps by downloading new wasm versions. Of course Apple won't do that citing __Insert__App__Store__excuse__here, but everyone knows it's because they want that sweet 30% cut, and that can only be effectively enforced with a tight chokehold on updates
ryohey 7y ago
Here’s my wasm parser/runtime written in TypeScript, for instance. https://github.com/ryohey/ts-wasm-runtime https://github.com/ryohey/ts-wasm-runtime

144 points by simonpure 7y ago

6nf 7y ago

My self hosted compiler: https://github.com/PierreRossouw/wats https://github.com/PierreRossouw/wats

Documentation on how Lobster's Wasm-backend works, which contains plenty of useful info for anyone making their own backend: http://aardappel.github.io/lobster/implementation_wasm.html http://aardappel.github.io/lobster/implementation_wasm.html

pjmlp 7y ago
Cool, thanks for sharing.

thomasballinger 7y ago

Here's Colin's blog post with code and diagrams inline: https://blog.scottlogic.com/2019/05/17/webassembly-compiler.html https://blog.scottlogic.com/2019/05/17/webassembly-compiler.... I found Colin's posts real helpful when writing some WebAssembly by hand for the first few days of Advent of Code: https://observablehq.com/collection/@ballingt/advent-of-wasm https://observablehq.com/collection/@ballingt/advent-of-wasm

catpolice 7y ago

Last year I decided to write a WASM-targeting compiler in Javascript, completely by hand. The idea was to compile a vaguely C-like toy language to WASM without any external dependencies. Part of the motivation was that while I've written many simple parsers, I mostly used parser generators with BNF grammars and I didn't feel like I had a good sense of how the code I ended up generating actually worked for something with complex syntax and semantics. I felt like I was writing specs for a parser, rather than writing a parser. And I didn't have a huge amount of experience with code generation. My toy language has vaguely C-like syntax with block scope and infix operators with various precedences, so it was a bit more complicated than JSON, but I ended up using something like Pratt parsing/Precedence Climbing (see https://www.oilshell.org/blog/2017/03/31.html https://www.oilshell.org/blog/2017/03/31.html) and wrote the whole thing in a way that's - hopefully - pretty easy to read for folks interested in wrapping their head around parsing complex syntax (e.g. with scope and name resolution). The lexer, parser and language definition ended up being about 1000 lines of JS (counting some pretty extensive comments). Code generation is pretty straightforward once you have an AST. Any JS programmers that are interested in really getting into the nitty-gritty of writing your own parser/compiler should check it out. The source is here: https://github.com/j-s-n/WebBS https://github.com/j-s-n/WebBS. If you want to play around with the language and inspect the generated ASTs and WASM bytecode, there's an interactive IDE with example code here: https://j-s-n.github.io/WebBS/index.html#splash https://j-s-n.github.io/WebBS/index.html#splash

vchak1 7y ago
I found this one pretty interesting. https://maierfelix.github.io/mini-c/ https://maierfelix.github.io/mini-c/

ridiculous_fish 7y ago

Are there any fast ways to run wasm on systems that forbid creating new executable pages? A high performance interpreter?

vbezhenar 7y ago
What system do you mean? iOS supports wasm AFAIK.
- jimmcslim 7y ago
  Mobile Safari may support WASM, but I think the GP is suggesting that a native iOS app would not be permitted to load, compile, and execute a WebAssembly module.
  - vbezhenar 7y ago
    I might be wrong, but I think that one should be able to utilize JavaScript engine within native iOS app which will use fast implementation.
    - nickmain 7y ago
      JavaScriptCore and JS running in a web page using WKWebView are both running in the same app process and do not support WASM since they cannot generate code (cannot write to executable memory pages). SFSafariViewController does support WASM by executing in a separate more privileged process but interaction with the app is very limited.
      - saagarjha 7y ago
        WKWebView does as well because it too runs out-of-process. You can load WebAssembly into it.
        throwGuardian 7y ago
        But WKWebView is equivalent to running inside a browser, with no system access. If JSCore allowed WASM JIT-ing, one could easily port WASI to iOS for full system access, and update iOS apps by downloading new wasm versions. Of course Apple won't do that citing __Insert__App__Store__excuse__here, but everyone knows it's because they want that sweet 30% cut, and that can only be effectively enforced with a tight chokehold on updates

ryohey 7y ago

Here’s my wasm parser/runtime written in TypeScript, for instance. https://github.com/ryohey/ts-wasm-runtime https://github.com/ryohey/ts-wasm-runtime