13 ms·
Wasm3 compiles itself (using LLVM/Clang compiled to WASM)
- nindalf 6y agoFor those wondering, Wasm3 describes itself as "the fastest WebAssembly interpreter". Other webassembly runtimes are JIT based, rather than being interpreters. The project's readme (https://github.com/wasm3/wasm3 https://github.com/wasm3/wasm3) talks more about this decision. For more on the difference, and an explanation of what JIT is, check out this section of the book Crafting Interpreters (https://craftinginterpreters.com/a-map-of-the-territory.html#shortcuts-and-alternate-routes https://craftinginterpreters.com/a-map-of-the-territory.html...)
- Dylan16807 6y ago> Other webassembly runtimes are JIT based, rather than being interpreters. Do none of the notable ones do normal compilation?
- TazeTSchnitzel 6y agoWhat would the difference be between “JIT” and “normal” compilation for WebAssembly?
- smasher164 6y agoAn AOT compiler would do a one-time translation to a platform-specific binary. If the environment you run on doesn’t support JIT compilation (iOS for example), AOT compiling WASM is useful.
- Blikkentrekker 6y agoHow can a platform not support j.i.t. compilation? In what way does a platform need to coöperate with that?
- erik_seaberg 6y agoSome platforms are enforcing https://en.wikipedia.org/wiki/W%5EX https://en.wikipedia.org/wiki/W%5EX because malware also wants to create novel machine code within a running process.
- saagarjha 6y agoNote that it is possible to JIT even with W^X. What these platforms do is prevent memory that has been mapped writable and overwritten with new code from ever being mapped executable again.
- jlokier 6y agoA JIT generally means it compiles parts of the program to machine code on the fly before running them, as needed. To support that, the environment must allow the JIT to write machine code to memory, and then execute that same code. CPUs have memory protection flags to control which memory areas can be written, and which can be executed. The OS is in charge of setting those flags, on request from the application. Eg. mmap and mprotect system calls. iOS denies requests for memory that is both writable and executable at the same time. So applications cannot get the type of memory area a JIT needs. There are indirect methods where a file is written then mapped, like generating a small program or shared library on the fly. But iOS restricts these as well. There are workarounds for a developer's personal applications, used on their own registered iOS devices. But these workarounds cannot be run by everyone else, except people with a jailbroken iOS. They cannot be used in applications on the App Store.
- moonchild 6y agoW^X is not the issue here. JITs can deal with that, and in fact should do so even when it's not OS-enforced for security reasons. The problem on ios specifically is code signing, which is a problem for both JIT and AOT.
- jlokier 6y ago"There are indirect methods where a file is written then mapped, like generating a small program or shared library on the fly. But iOS restricts these as well."
- brundolf 6y agoAt that point, what's the purpose of using WASM at all?
- ReactiveJelly 6y agoYou can keep the sandbox intact after AOT. https://hacks.mozilla.org/2020/02/securing-firefox-with-webassembly/ https://hacks.mozilla.org/2020/02/securing-firefox-with-weba... > The core implementation idea behind wasm sandboxing is that you can compile C/C++ into wasm code, and then you can compile that wasm code into native code for the machine your program actually runs on. These steps are similar to what you’d do to run C/C++ applications in the browser, but we’re performing the wasm to native code translation ahead of time, when Firefox itself is built. It's a middle ground between sandboxing with subprocesses (which adds all the overhead of IPC) and switching to a fully memory-safe language (Rust, JS, etc.)
- smasher164 6y agoIf I were writing a sandboxing or instrumentation tool, operating on WASM would be much simpler than dealing with the quirks of assembly, syscalls, or some higher-level IR.
- seg_lol 6y agoControl Flow Integrity, https://en.wikipedia.org/wiki/Control-flow_integrity https://en.wikipedia.org/wiki/Control-flow_integrity
- Dylan16807 6y agoSo mostly I'm interested in compilers that compile the entire thing before any execution happens. Technically both AoT and JIT compilers can do that, but 95% of the things associated with "JIT" involve compiling the program one piece at a time. But even with the broad definition of JIT, one important difference is that AoT compilers don't have to mark pages executable.
- jedisct1 6y agoWAVM and SSVM do normal compilation.
- the_duke 6y agoSome runtimes like wasmtime support AOT compilation artifacts that can be loaded. But in general you can't "just compile" Webassembly, because you need a runtime. There is a C project that can compile an executable that includes the runtime and the compiled WASM, but the name is escaping me right now. Note that JIT in the WASM world doesn't quite mean the same thing as for eg Java. Almost all runtimes compile a whole module at once, not individual functions.
- pjmlp 6y agoCompiling individual functions was something in Java JITs like 20 years ago, they are a little bit more clever nowadays.
- nicoburns 6y agoI think the model is typically that optimisation applied in "normal compilation" are applied when the source language is translated to WASM, meaning that the WASM -> native translation can be quite straightforward and still performant.
- Others 6y agoI worked on one as an undergrad: https://github.com/gwsystems/aWsm https://github.com/gwsystems/aWsm Full AoT compilation, C programs run within 10% of native
- heavyset_go 6y ago10% difference from native, or 10% of native performance?
- Others 6y ago10% worse performance. Not 10x worse :D
- moonchild 6y agoSince it compiles using llvm, what's the reason for the discrepancy? Lost information about e.g. aliasing?
- Others 6y agoBounds checking, dynamic dispatch is more expensive, wasm semantics have no undefined behavior to exploit for optimizations. And there is bloat in the process. I’m sure losing information effects things as well
- vidarh 6y agowasm2c/WasmBoxC springs to mind: https://kripken.github.io/blog/wasm/2020/07/27/wasmboxc.html https://kripken.github.io/blog/wasm/2020/07/27/wasmboxc.html
- haberman 6y agoWasm3 calls its interpreter design a "meta machine": https://github.com/wasm3/wasm3/blob/main/docs/Interpreter.md#m3-massey-meta-machine https://github.com/wasm3/wasm3/blob/main/docs/Interpreter.md... It is heavily based around tail calls. I recently wrote a blog article about how we applied a similar tail-call-oriented strategy to accelerate protobuf parsing to 2+GB/s: https://blog.reverberate.org/2021/04/21/musttail-efficient-interpreters.html https://blog.reverberate.org/2021/04/21/musttail-efficient-i... I also recently landed a change in Clang trunk that offers guaranteed tail calls, so that this tail call design is safe in non-opt builds: https://reviews.llvm.org/D99517 https://reviews.llvm.org/D99517 I think wasm3 could benefit from using this attribute when it is available.
- glandium 6y agoHow do guaranteed tail calls work when you needs to put arguments on the stack, which usually prevents tail call? (and whether a specific call needs to pass arguments on the stack depends on the platform, and the number and type of arguments)
- haberman 6y agoThe musttail attribute performs several checks to ensure that a tail call can be guaranteed, and rejects the program if these rules are violated. The documentation has the details: https://github.com/llvm/llvm-project/blob/3b8ec86fd576b9808dc63da620d9a4f7bbe04372/clang/include/clang/Basic/AttrDocs.td#L449-L468 https://github.com/llvm/llvm-project/blob/3b8ec86fd576b9808d... These rules guarantee that the tail call is possible to perform on every platform. They end up being more strict than is necessary on some platforms and calling conventions.
- rubatuga 6y agoI feel the need to repost this link: https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death...
- tyingq 6y agoI'm somewhat curious at what pace WASM will gain "market share" while it's only practical to target it with C, C++, Rust, etc. Supposedly, there are plans to expand it to where it looks more like a virtual machine and less like ASM. Adding things like garbage collection, direct DOM manipulation, polymorphic inline cache, etc. Things that would make it possible to run a decent scripting language without pulling in some huge runtime. Or forgetting the scripting languages, just the direct DOM access might make it less tedious to use with compiled languages. It's sort of a second run at getting "applets" right, but in a cross-language way.
- jedimastert 6y agoit seems more like it's it's trying to be LLVM but for the web?
- cwalv 6y agoGraalvm native image has the ability to target llvm with a goal of making it easier "to target a new architecture without having to implement a complete new backend for Native Image." https://www.graalvm.org/reference-manual/native-image/LLVMBackend/ https://www.graalvm.org/reference-manual/native-image/LLVMBa...
- nicoburns 6y ago> while it's only practical to target it with C, C++, Rust, etc. You might not want to bundle Rust in with C and C++ in this context. Rust is pretty accessible to developers coming from scripting languages (JavaScript, Python, etc). Direct DOM manipulation (and the ability to send reference types across the JS-wasm boundary) is a pretty big deal though I think. Currently it doesn't make an awful lot of sense to build code that interacts with the DOM in WASM (although in Rust there are automatically generated bindings that go through a JavaScript shim).
- kindofajsdev 6y agoI'm a newer dev so pardon my ignorance. Is webassembly the new, more open JVM? Feels like that write once, run everywhere but this time with any(most) mainstream language.
- brokensegue 6y agoWasm lacks a lot of lower level system specific operations that the jvm offers.
- tyingq 6y agoIt seems like it's headed that way. At the moment, it's only really practical to generate WASM from compiled languages, or really small languages, like Lua. And your interfaces to the browser (and thus, the outside world) itself are pretty limited, so still lots of javascript involved.
- vvanders 6y agoCan confirm that Lua does run in WASM in the browser: https://github.com/vvanders/wasm_lua https://github.com/vvanders/wasm_lua I sometimes wonder what the world would be like if Lua had zero based indices and saw a bit broader adoption. It's such an easy language to embed/extends although QuickJS looks like it's going to give it a run for the money in the long term given the similar goals on ease of embedding. I wonder if anyone has compiled QuickJS to WASM and run it in the browser.
- nathcd 6y ago> I wonder if anyone has compiled QuickJS to WASM and run it in the browser. Fabrice Bellard actually did it himself here: http://numcalc.com/ http://numcalc.com/
- heavyset_go 6y agoIf your language is supported by LLVM, then Emscripten can compile it to WASM.
- conradev 6y agoHas anyone built a compiler extension and libffi-based runtime that supports compiling otherwise normal native code to WASM (eschewing hard linking for runtime resolution)? Would be fun to make it so that native apps for any platform can be run from an interpreter
- zozbot234 6y agoEmscripten?
- westurner 6y agoSelf-hosting (compilers) https://en.wikipedia.org/wiki/Self-hosting_(compilers) https://en.wikipedia.org/wiki/Self-hosting_(compilers) : > In computer programming, self-hosting is the use of a program as part of the toolchain or operating system that produces new versions of that same program—for example, a compiler that can compile its own source code
- aitchnyu 6y agoHow big of a milestone is self-hosting for a language?
- westurner 6y agoThe wikipedia article lists quite a few languages for which there are self-hosting compilers. JS can already write more JS. Are there advantages and risks introduced by this new capability for browser-hosted (?) WASM LLVM to compile WASM?
- aphextron 6y agoWhat is the state of the world with wasm at this point? It seems like it's been this huge tease for 10 years now, promising that we'll be able to build web apps in any language. But as far as I'm aware, garbage collection and DOM access is still nonexistent.
- arthurcolle 6y agoWish I had a RemindMe! bot on HN for comments like these.
- Hydraulix989 6y agoThere's no garbage collection in WASM. Why do you think that there is garbage collection in WASM? And WASM has been around since 2017, which is only 4 years, not 10 years... Bitcoin has been around much longer (~12 years), for example.
- sound_and_form 6y agoIt's a spec proposal. https://github.com/WebAssembly/gc/blob/master/proposals/gc/Overview.md https://github.com/WebAssembly/gc/blob/master/proposals/gc/O...
- Hydraulix989 6y agoYes, and this is just an extension. Existing WASM compiler front-ends for non-GC languages don't have to necessarily use it. Wasm itself still won't take a performance hit, unless individual devs opt in by using front-ends for GC languages.
- ReactiveJelly 6y agoEmscripten hit 1.0.1 around November 2012 (looked it up on GitHub) So if wasm is the spiritual successor to Emscripten's "Compile C to JavaScript and run C in a browser" then it's only 8 years.
- Hydraulix989 6y ago
- kumarski 6y agoanybody wasmer.io
- diimdeep 6y agoLooks like wasm3 is created by embedded/hardware guy, that is really interesting.