19 ms·

Firefox’s new streaming and tiering compiler

912 points by markdog12 9y ago

zach43 9y ago
Interesting article...I did not realize that the WASM needs to be compiled into machine code on the client system, I just assumed it would be directly interpreted by the JS engine. As a side note, it is interesting to see that multithreaded compilation of a single page provides significant performance benefits here...this is usually not done with C/C++ code compilation from what I understand about it
- Yoric 9y ago
  Well, the difference between "interpreted" and "compiled" has become very blurry during the last 20 years. These days, most "interpreted" programming languages are actually compiled to machine code on the client system. This includes the JVML, of course, but also JavaScript, Python (with PyPy), etc. PHP isn't quite there yet, but it's coming. > As a side note, it is interesting to see that multithreaded compilation of a single page provides significant performance benefits here...this is usually not done with C/C++ code compilation from what I understand about it It's slightly different, but native code is typically compiled concurrently, too. The meat of it is often handled by the build system rather than the compiler itself, but that's not so different.
  - lclarkmichalek 9y ago
    PHP is there with HHVM
    - Yoric 9y ago
      My bad, I hadn't looked at that project in some time and I hadn't realized they had advanced that far.
    - noir_lord 9y ago
      Which ironically isn't really any faster than PHP7 in the real world outside of benchmarks. 7 was a phenomenal release, I saw 50% reductions in processing time across the board and on old array heavy systems 5-10x memory reduction.
      - solidr53 9y ago
        Thanks to pressure from HHVM I assume. Nothing was happening in the PHP language for freaking three years. To be fair the benchmarks usually take a wordpress or drupal installation and do a requests per second measurement, which IMO is a real world benchmark. No hate, I just don't get why hhvm doesn't get any love for what they did. Maybe because from HPHPc to HHVM they seriously gave the PHPc a competition and people kind of got mad. https://kinsta.com/blog/the-definitive-php-7-final-version-hhvm-benchmark/ https://kinsta.com/blog/the-definitive-php-7-final-version-h...
        krapp 9y ago
        >No hate, I just don't get why hhvm doesn't get any love for what they did. I don't know - I expected to see a ton of Hack projects show up here but it's like no one cared about the language except as a wake-up call to PHP. Maybe the involvement of Facebook put people off.
        deleted 9y ago
        [deleted]
        littlestymaar 9y ago
        > Nothing was happening in the PHP language for freaking three years. To be fair that's not because they where sleeping, but because they attempted to do something that proved too hard (unicode support) and they had to abandon it. That's why PHP skipped version 6.
        deleted 9y ago
        [deleted]
  - nabla9 9y ago
    And then modern microprocessors receive x86-64 instructions and decode them into underlying microcode.
  - pjmlp 9y ago
    It was always like that on the mainframe world. Assembly was actually bytecode, with a micro-coded CPU doing the actual execution. All Xerox computers were like that. The first boot step was to load the right kind of micro-code for the environment being started. The AS/400 native environment (nowadays known as IBM i), is based on bytecode TIMI, which gets AOT compiled via a kernel level JIT.
- pcwalton 9y ago
  > As a side note, it is interesting to see that multithreaded compilation of a single page provides significant performance benefits here...this is usually not done with C/C++ code compilation from what I understand about it Well, that's because typically all cores are maxed out during a parallel build of large-scale C++ software, so there's no need to go any further. With link-time optimization it's a different story…hence the work some compilers (like rustc for Rust) are doing to parallelize builds of single compilation units.
- callahad 9y ago
  There's a simple-but-useful WebAssembly Explorer at https://mbebenita.github.io/WasmExplorer/ https://mbebenita.github.io/WasmExplorer/ that interactively shows the C/C++ -> WASM Text -> x86 ASM path that WebAssembly takes. I wrote up a short article and video demonstrating it last year at https://hacks.mozilla.org/2017/03/previewing-the-webassembly-explorer/ https://hacks.mozilla.org/2017/03/previewing-the-webassembly...
- chrisseaton 9y ago
  > I did not realize that the WASM needs to be compiled into machine code on the client system It doesn't need to be. This is a choice they've made. Other implementations of WASM could interpret it they wanted. The Church-Turing thesis tells us that any program you can compile to machine code can also be interpreted, so it is not possible that any language needs to be compiled into machine code.
- Rusky 9y ago
  You can interpret wasm if you want, but considering the motivation for wasm is performance that kind of defeats the point.
markdog12 9y ago
https://lukewagner.github.io/test-tanks-compile-time/ https://lukewagner.github.io/test-tanks-compile-time/ Firefox Nightly: WebAssembly.instantiate took 227.6ms (54.4mb/s) Chrome Canary: WebAssembly.instantiate took 8576ms (1.4mb/s) Wow. (Edit: And I believe that's not even using the streaming compilation mentioned in the article, it's just the new baseline compiler in action)
- styfle 9y ago
  Where did you even find that repo? Was that mentioned in the article or is this your code?
  - callahad 9y ago
    Linked in the article under "give it a try" in the sixth paragraph. The repo itself is at https://github.com/lukewagner/test-tanks-compile-time https://github.com/lukewagner/test-tanks-compile-time
- callahad 9y ago
  > I believe that's not even using the streaming compilation mentioned in the article That's correct. Streaming compilation would finish earlier, but might actually benchmark more slowly because you'd be adding in the time that the compiler is idle and waiting for the network to catch up. Preloading the .wasm file in the test lets us measure just the speed of the compiler, independent of the network.
  - derefr 9y ago
    An interesting benchmark would be the total energy usage of the CPU while loading a page in each browser. The compiler managing to idle waiting for network packets should theoretically allow some of the CPU cores to enter sleep states. (Not all, since others are still busy rendering the page, and at least one is busy doing whatever non-DMA kernel bits are involved with receiving the network packets.)
    - 9y ago
stevemk14ebr 9y ago
Caching of compiled code! As i read it they want to cache the wasm bytecode at the client level. What if servers did the caching instead? Group clients by the architectures they use and serve the cached bytecode to the right 'groups' of clients.
- sjrd 9y ago
  That would assume you trust the server not to give you malicious machine code (which you of course cannot!). wasm is specified in such a way that it is still sandboxed by the VM that compiles it. If you fetch arbitrary machine code, you cannot verify it and that leads to huge security holes!
  - chrisseaton 9y ago
    > which you of course cannot! Didn't Google's NaCL implement verification of sandboxed machine code?
    - sjrd 9y ago
      Maybe, but at what cost? I wouldn't be surprised if the cost of verifying the machine code was higher than the cost of compiling wasm to machine code.
  - hateduser2 9y ago
    Why can’t you just cache it with a hash?
    - icebraining 9y ago
      How would the client know the hash is valid?
      - Zpalmtree 9y ago
        Perhaps he means you compile the code locally, hash it, and then next time you can fetch the compiled code from a server, and check the hash matches?
sjrd 9y ago
Nice article. Although, as always with articles on WebAssembly, it keeps repeating that wasm is faster than JavaScript, without ever mentioning the limitations of wasm wrt. JS (no GC, no interaction with the DOM or with JS libraries besides numbers, etc.). And that means there are zillions of developers who keep being misled in thinking stuff like "Why don't you compile to wasm to make your stuff faster?". That includes absurdities like "We should write a compiler from JavaScript to wasm to make all our JS faster!"
- gigatexal 9y ago
  Yeah I read the article too and thought: but what can I do with wasm?
  - deleted 9y ago
    [deleted]
  - kbumsik 9y ago
    You can import C (and many others) libraries without re-implementing it in JS for browsers. In my company's case, we compile C-written libopus almost directly to encode audio streams into Opus on browsers, not on servers. In this way we benefit much less data traffic and server CPU loads.
    - QAkICoU7IDNkpFu 9y ago
      >You can import C (and many others) libraries This means compiling the lib to wasm, right? At first I was thinking of running JS clientside that imports some C lib somehow, which confused me.
      - kbumsik 9y ago
        Yes. The C lib will be compiled into wasm. You can use the wasm-compiled code in a browser through JS wrappers. So you can say importing C lib on JS client side anyway. Note that wasm's main objective is to run non-JS code on browsers, not for a faster JS. Here is the best overview and tutorial I've ever seen in HN , if you are interested: https://news.ycombinator.com/item?id=15958827 https://news.ycombinator.com/item?id=15958827
Twirrim 9y ago
Out of curiosity, using just released versions of browsers on this 2015 mac pro: Firefox 57: WebAssembly.instantiate took 2990.2ms (4.1mb/s) Chrome 63: WebAssembly.instantiate took 8736.9ms (1.4mb/s) Safari 11.0.2: WebAssembly.instantiate took 10341ms (1.2mb/s) If more speed is about to arrive, wow. I'm curious what optimisations are needed / valuable for wasm files to improve streaming performance. I'm assuming if, e.g.: def foo(baz): bar(baz) ... def bar(baz): baz = baz +1 Then compilation would start and get stuck until it had a definition for bar? If so, presumably the next build time optimisations for a website will be to shuffle the code around in to as optimal an order as possible so as to improve streaming compilation speed?
- kllrnohj 9y ago
  Function declarations are independent from function bodies. So think C/C++ headers/source file splits. You don't need to know what code is in bar if you know it takes 1 argument of type int and returns an int. That's all you needed to know how to call it successfully, so you can compile foo in your example perfectly fine. You just need to patch up the call location later when bar is resolved to actual address (this is the "link" step in a typical AOT compilation chain, or done by the loader if it's a dynamic dependency)
  - jnordwick 9y ago
    Considering the major optimization in compiling is inlining, knowing the function body is very important to compilation, but I guess that can be pushed off until the next tier.
    - rcthompson 9y ago
      I imagine the first pass could inline functions that are already compiled and skip functions that have yet to be compiled. Maybe tools that generate wasm will start reordering the functions they send to allow optimal first-pass inlining.
    - callahad 9y ago
      Don't forget about the local compilation phase from C/C++/Rust to WebAssembly before you ever hit the browser. At that point, LLVM is free to optimize and inline just like with any other binary target.
nothrabannosir 9y ago
does someone here, familiar with webassembly semantics, know if it’s theoretically possible to start streaming execution of code? I.e. as soon as the “main” (?) function is in, and block on every function call which is not yet compiled, recursively? Or could the last block of webassembly bytecode potentially change the semantics of the first? Sooner or later, that’s an avenue people will want to explore, I assume?
- tsavola 9y ago
  Yes, it's possible. No, the last block won't change the semantics of the first.
JepZ 9y ago
Quoting Yehuda Katz, the co-creator of Ember.js, when it comes to JS-sizes is kinda hilarious (random google result): https://gist.github.com/Restuta/cda69e50a853aa64912d https://gist.github.com/Restuta/cda69e50a853aa64912d No offense to Yehuda in general (he is doing great work), but Ember.js so ignorant of any js-size recommendations, that it seems weird to quote Yehuda in that context.
MaxBarraclough 9y ago
A two-tier JIT. Interesting to see tiered JIT compilation catch on the way it has. I seem to remember a few years ago reading that the Java HotSpot team had given up on tiered JIT compilation as being not worthwhile. How far we've come. A whirlwind tour of todays JITs (apologies for the million links): .Net Core seems not to use tiered compilation. It never interprets the IR; everything is run through the same JIT compiler. https://github.com/dotnet/coreclr/issues/4331 https://github.com/dotnet/coreclr/issues/4331 HotSpot uses three tiers these days (counting direct interpretation as a tier) - https://docs.oracle.com/javase/8/docs/technotes/guides/vm/performance-enhancements-7.html#tieredcompilation https://docs.oracle.com/javase/8/docs/technotes/guides/vm/pe... JavaScriptCore/Nitro seems to use four - https://webkit.org/blog/3362/introducing-the-webkit-ftl-jit/ https://webkit.org/blog/3362/introducing-the-webkit-ftl-jit/ Edge's Chakra engine has two - https://blogs.msdn.microsoft.com/ie/2014/10/09/announcing-key-advances-to-javascript-performance-in-windows-10-technical-preview/ https://blogs.msdn.microsoft.com/ie/2014/10/09/announcing-ke... V8 seems to use two - https://v8project.blogspot.co.uk/2017/05/launching-ignition-and-turbofan.html https://v8project.blogspot.co.uk/2017/05/launching-ignition-... Firefox's SpiderMonkey JS engine uses two - https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/Internals https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Sp...
- pjmlp 9y ago
  .NET focus always was native code, either AOT with NGEN or JIT on load. The only variants of .NET with interpreter support were from 3rd party implementations, and the .NET Micro Framework, used in NETduino. And now their focus seems to be to improve their AOT story. Another interesting evolution was Android, with Dalvik and its basic JIT, ART with AOT on installation, to ART reboot with an interpreter in Assembly, followed by JIT and AOT code cache with PGO.
  - bad_user 9y ago
    On .NET Core's focus, that's not actually true: http://mattwarren.org/2017/12/15/How-does-.NET-JIT-a-method-and-Tiered-Compilation/ http://mattwarren.org/2017/12/15/How-does-.NET-JIT-a-method-... Android optimizes for battery life, but it's also worth noting that Dalvik was a really rudimentary JIT, having no benefits from JIT compilation, only drawbacks, the ART with AOT being a good upgrade. But tiered compilation is in a different league, being about speculating what's going to happen depending on what the process has witnessed thus far. The point of tiered compilation is to profile/guard stuff at runtime and recompile pieces of code based on changing conditions, which is how you can optimize virtual call sites or other dynamic pieces, which you can't do ahead of time because the missing part is the decompiler which can revert optimizations based on invalidated conditions. It's really interesting actually, because you can profile a C++ app and use that to optimize your AOT compilation, but the compiler is still limited by the things it can prove ahead of time, or otherwise it would be memory unsafe.
gkya 9y ago
I'm not familiar with Web Assembly, but the recent trend is that as the downloads become faster, web performance in a vanilla browser becomes slower, because websites just send more stuff to you. Pages grow toward infinity. Also, if, like @sjrd mentioned, this code can't manipulate DOM or can use only a restricted set of JS objects, the where will the gain be? Is this intended to be used for number crunching code in the browser runtime? Help bitcoin miner scripts? What's the purpose then?
- seren 9y ago
  Running a native app compiled to wasm into the browser ? So the opposite of running an Electron app?
  - wlesieutre 9y ago
    Calling it now: 1) Write a Windows app 2) Run it in the browser with wasm 3) Stuff that into Electron and distribute to Mac/Linux/Windows Why distribute the electron wrapped wasm on Windows instead of using the real native Windows app? It's more consistent this way! Single codebase! Developer efficiencies!
    - pjmlp 9y ago
      Start here, http://www.mono-project.com/news/2018/01/16/mono-static-webassembly-compilation/ http://www.mono-project.com/news/2018/01/16/mono-static-weba...
    - swsieber 9y ago
      Small deviation - write a native linux or mac app (instead of targeting Windows for the initial app)... mostly because I feel like developing on those platforms is so much more enjoyable.
      - wlesieutre 9y ago
        Good point. Linux is probably the easier one because you'd need to build your UI toolkit into it and we don't have the source for Cocoa or whatever this year's Windows UI toolkit is called.
jnordwick 9y ago
I guess i don't understand the push to wasm. Why not just embed hotspot, or a branch of it. Is there any difference? Or going the other way, could hotspot be replaced with a wasm jit by compiling java to wasm? I know they have slightly different memory models, but I don't understand why they seem to be treated so separately.
- vurpo 9y ago
  wasm is not a browser plugin, it is part of the browser. That means the plugin architechture does not need to exist (which has already been removed by all major browsers). Building wasm support into browsers was not an easy task, and "just embed hotspot" is not any easier.
- filereaper 9y ago
  I'm not sure that JVM allows for streaming code, its centred around class-loading and classes in general. Shipping in fragments of code will require quite an overhaul of that entire architecture. Lambdas and other constructs were added much later via JSR-292 and invokedynamic, streaming stuff in will require quite a bit of shoehorning.
  - pjmlp 9y ago
    That is implementation specific and not defined as part of the JVM specification. I don't know all the JVM implementations out there, but it wouldn't surprise me if there was one with it implemented. By the way, RMI and Jini worked by streaming code across the network.
    - dfox 9y ago
      Streaming in this case means that you can start generating (or even executing) code before you have complete input. There are some mostly ignorable reasons (having to do with bytecode verification) why you should not stream-JIT Java bytecode. On the other hand in the whole ecosystem you will not gain anything wothwhile given the fact that you need the JVM state to be essentially complete before you start executing anything and on slightly lower-level the .class file format is designed to be compact and not meaningfully streamable.
axelfontaine 9y ago
Using https://lukewagner.github.io/test-tanks-compile-time/ https://lukewagner.github.io/test-tanks-compile-time/ Chrome 63: 3143.7ms (3.9mb/s) Firefox 57: 1499ms (8.3mb/s) Edge 41: 97.3ms (127.2mb/s) !!!
- MrMid 9y ago
  Wow. Firefox 59: 474 ms Edge 41: 164 ms The difference is smaller with newer FF, but that is amazing from Edge!
  - computerphage 9y ago
    Edge is doing something very different, see the sibling comments for details.
- pimeys 9y ago
  Firefox nightly on Linux: 165.6 ms (74.8 MB/s) Chromium: 1835 ms (6.7 MB/s)
- isaacaggrey 9y ago
  Could someone explain how Edge is performing so well or any references to what they have done in this regard? Has the Edge team already implemented this streaming and tiering compiler technique?
  - DiThi 9y ago
    You tell Edge, "compile this", Edge replies "done!" when what it has done is just verifying it's valid WASM. When you then call a function, it's compiled.
    - qznc 9y ago
      So it is actually sad. It means you just downloaded a lot of stuff and only used a fraction of it.
- luke_wagner 9y ago
sehugg 9y ago
That's great news. On http://8bitworkshop.com/ http://8bitworkshop.com/ I'd like to offer some additional WASM modules on-demand but they take 15+ seconds to load. (It seems 50% of the time is parsing and 50% module instantiation)
filereaper 9y ago
Total Aside. As a compiler and runtimes guy, I'm super excited for streaming compilation. I think stuff like this and ethereum for distributed computation is really cool stuff! :D
- dfox 9y ago
  Streaming compilation is the way it was always historically done. One reason is that computers used to not have enough RAM to store whole non-trivial programs in it in AST or another intermediate form. Second reason is that this approach matches how the underlying theory of languages and automatons works. One can view modern AST producing compiler frontend as compiler that compiles it's input into program that builds the resulting AST. On the other hand many modern optimalization passes simply cannot be done in streaming manner or even by any pushdown automaton.
  - caf 9y ago
    Yes, which is likely why in older languages like C you must declare symbols before you use them etc.
bsimpson 9y ago
Would be nice if it mentioned WebAssembly in the title - I presumed this was a new feature for JS in Firefox Quantum.
- markdog12 9y ago
  I originally put WASM in the title, not sure what happened.
bzadoroz 9y ago
Using https://lukewagner.github.io/test-tanks-compile-time/ https://lukewagner.github.io/test-tanks-compile-time/ Pale Moon 27.7.0: "fetch is not defined"
breatheoften 9y ago
Does wasm do runtime code specialization? I wonder if there will end up being a way to do to timing attacks against the optimizing wasm compiler/linker step ... Is it possible to setup code such that the optimization time depends on the runtime inferred type of an 'x' that you aren't supposed to have access to ...?
- yoklov 9y ago
  The term you’re looking for is speculation, not specialization, but no, I don’t think it does either. C++ and other languages targeting WASM often do type specialization, but it’s entirely done before the browser sees WASM, and has nothing to do with what you’re describing. (Which would be speculative compilation). I’d imagine that nobody does speculative compilation since the benefit is too low given how fast the network is. Also, yes, there would be security concerns.
microcolonel 9y ago
On a desktop, we compile 30-60 megabytes of WebAssembly code per second. That’s faster than the network delivers the packets. Funny enough, on my workstation it seems to compile something more like 60-80 MiB/s, to keep up with my network which was recently upgraded to gigabit. Very impressive stuff, I hope workstation CPUs can keep pace with networks.
bobsc123 9y ago
This cracks me up. Modern web browsers really started to evolve in the 90's when security problems really ramped up. You used to just download excecutables and run them on your computer because the functionality wasn't there otherwise. Flash and Java applets were the initial answer to that before Javascript and HTML evolved. We've come almost full circle to browsers basically being little VM's that can do anything again, the main reason they were developed in the first place. Most people's entire computer experience is now in the browser and here come executables again which will require another internal layer to mitigate problems.
- NanoWar 9y ago
  Actually, browsers are designed from the ground up to handle insecure code! That's pretty awesome, but comes at a cost: The mentioned additional layer, battery, speed? And it's platform independent!
- ajkjk 9y ago
  The end state ends up looking a lot like (the user-facing side of) an operating system, except that: * the filesystem is cloud storage (Drive/Dropbox/what have you -- the Unhosted (https://unhosted.org/ https://unhosted.org/) architecture) * the apps are insecure but open-source by requirement (interpreted jS) * ... running in a controlled sandbox (the browser) * ... using a standard UI language (HTML/CSS) * with functionality modifiable/overridable by user preference (extensions) It's pretty much the ecosystem you would want if you were building this from scratch! Except you'd want Html/CSS/JS to be much more intelligently designed from the start (I'm waiting so eagerly for the day that browsers natively run more scripting languages than just JS...) It never could be done in the 90s because everything ran too slowly, but it's feasible now.
  - pjmlp 9y ago
    > It never could be done in the 90s because everything ran too slowly, but it's feasible now. It used to be called Lisp Machines, Smalltalk, Oberon Juice, Java Jini, Inferno.
- hathawsh 9y ago
jokoon 9y ago
That's cool, but until I see it coming to smartphones, that won't really be useful, except for gaming. In theory this could really be the universal vm for the web everyone needed, but it's still lacking real sockets and dom support.
- aidenn0 9y ago
  It's available on android now...
  - jokoon 9y ago
    I mean as an universal way to make apps
StreamBright 9y ago
What are the security implications of wasm?
- markdog12 9y ago
  Good writeup here: http://webassembly.org/docs/security/ http://webassembly.org/docs/security/ It runs in existing browser VMs, which have been pretty battle tested. Another interesting note is that threads are now on hold for WebAssembly due to Spectre, that is, SharedBufferArray has been disabled. Hopefully it can be enabled in the future.
mrmondo 9y ago
Firefox Beta (58) macOS 10.13.2: WebAssembly.instantiate took 151.9 ms (81.5 MB/s)
Dolores12 9y ago
Guess what, downloading compiled executable code is even faster. Is that where we are heading to? Flash 2.0? Wouldn't it be great to save all the electric power that was used to compile very same code on millions of computers every day?
- __s 9y ago
  Sigh... let's have this thread again. Needs to be effectively sandboxed. Modern JS as a compiler target is just as opaque. JIT compilation is a win on energy relative to the energy spent in a slow interpreter. JITing WASM will take less energy than JITing emscripten-js (nevermind the energy to send over the wire)
- littlestymaar 9y ago
  Downloading a bigger executable wouldn't necessarily be faster actually, it depends on the size difference and the client bandwidth.[1] Additionnally, it wouldn't be portable (executable compiled for desktop wouldn't run on mobile). [1] See this comment : https://news.ycombinator.com/item?id=16171133 https://news.ycombinator.com/item?id=16171133
  - Dolores12 9y ago
    What you are saying is that source code is less in size than compiled native code which is nonsense.
    - littlestymaar 9y ago
      First of all I wasn't talking about source code, I was comparing the output of a C/Rust->wasm compiler to the one of a C/Rust->x86 compiler. Since the wasm virtual machine has a JIT, I believe the compilation to wasm isn't too aggressive with optimizations. And since those make the binary bigger, I assume a wasm output would be lighter than an x86 one. I didn't benchmark it though. And if you compare the size of the binary output with the size of the source code, the binary is bigger in many cases because of optimizations (and runtime size, for small programs). Additionally, the source code can be gziped with a good compression factor whereas the binary cannot. Then 99% of the time, the source code is lighter to send over the internet than the compiled binary.
rmrfrmrf 9y ago
> But there’s no good reason to keep the compiler waiting. It’s technically possible to compile WebAssembly line by line. This means you should be able to start as soon as the first chunk comes in. Maybe they can optimize further by speculating what the next line will be...
daveheq 9y ago
Where do I buy the Firefox coin?
deleted 9y ago
[deleted]

912 points by markdog12 9y ago

Interesting article...I did not realize that the WASM needs to be compiled into machine code on the client system, I just assumed it would be directly interpreted by the JS engine. As a side note, it is interesting to see that multithreaded compilation of a single page provides significant performance benefits here...this is usually not done with C/C++ code compilation from what I understand about it

Yoric 9y ago
Well, the difference between "interpreted" and "compiled" has become very blurry during the last 20 years. These days, most "interpreted" programming languages are actually compiled to machine code on the client system. This includes the JVML, of course, but also JavaScript, Python (with PyPy), etc. PHP isn't quite there yet, but it's coming. > As a side note, it is interesting to see that multithreaded compilation of a single page provides significant performance benefits here...this is usually not done with C/C++ code compilation from what I understand about it It's slightly different, but native code is typically compiled concurrently, too. The meat of it is often handled by the build system rather than the compiler itself, but that's not so different.
- lclarkmichalek 9y ago
  PHP is there with HHVM
  - Yoric 9y ago
    My bad, I hadn't looked at that project in some time and I hadn't realized they had advanced that far.
  - noir_lord 9y ago
    Which ironically isn't really any faster than PHP7 in the real world outside of benchmarks. 7 was a phenomenal release, I saw 50% reductions in processing time across the board and on old array heavy systems 5-10x memory reduction.
    - solidr53 9y ago
      Thanks to pressure from HHVM I assume. Nothing was happening in the PHP language for freaking three years. To be fair the benchmarks usually take a wordpress or drupal installation and do a requests per second measurement, which IMO is a real world benchmark. No hate, I just don't get why hhvm doesn't get any love for what they did. Maybe because from HPHPc to HHVM they seriously gave the PHPc a competition and people kind of got mad. https://kinsta.com/blog/the-definitive-php-7-final-version-hhvm-benchmark/ https://kinsta.com/blog/the-definitive-php-7-final-version-h...
      - krapp 9y ago
        >No hate, I just don't get why hhvm doesn't get any love for what they did. I don't know - I expected to see a ton of Hack projects show up here but it's like no one cared about the language except as a wake-up call to PHP. Maybe the involvement of Facebook put people off.
        deleted 9y ago
        [deleted]
      - littlestymaar 9y ago
        > Nothing was happening in the PHP language for freaking three years. To be fair that's not because they where sleeping, but because they attempted to do something that proved too hard (unicode support) and they had to abandon it. That's why PHP skipped version 6.
      - deleted 9y ago
        [deleted]
- nabla9 9y ago
  And then modern microprocessors receive x86-64 instructions and decode them into underlying microcode.
- pjmlp 9y ago
  It was always like that on the mainframe world. Assembly was actually bytecode, with a micro-coded CPU doing the actual execution. All Xerox computers were like that. The first boot step was to load the right kind of micro-code for the environment being started. The AS/400 native environment (nowadays known as IBM i), is based on bytecode TIMI, which gets AOT compiled via a kernel level JIT.
pcwalton 9y ago
> As a side note, it is interesting to see that multithreaded compilation of a single page provides significant performance benefits here...this is usually not done with C/C++ code compilation from what I understand about it Well, that's because typically all cores are maxed out during a parallel build of large-scale C++ software, so there's no need to go any further. With link-time optimization it's a different story…hence the work some compilers (like rustc for Rust) are doing to parallelize builds of single compilation units.
callahad 9y ago
There's a simple-but-useful WebAssembly Explorer at https://mbebenita.github.io/WasmExplorer/ https://mbebenita.github.io/WasmExplorer/ that interactively shows the C/C++ -> WASM Text -> x86 ASM path that WebAssembly takes. I wrote up a short article and video demonstrating it last year at https://hacks.mozilla.org/2017/03/previewing-the-webassembly-explorer/ https://hacks.mozilla.org/2017/03/previewing-the-webassembly...
chrisseaton 9y ago
> I did not realize that the WASM needs to be compiled into machine code on the client system It doesn't need to be. This is a choice they've made. Other implementations of WASM could interpret it they wanted. The Church-Turing thesis tells us that any program you can compile to machine code can also be interpreted, so it is not possible that any language needs to be compiled into machine code.
Rusky 9y ago
You can interpret wasm if you want, but considering the motivation for wasm is performance that kind of defeats the point.

markdog12 9y ago

https://lukewagner.github.io/test-tanks-compile-time/ https://lukewagner.github.io/test-tanks-compile-time/ Firefox Nightly: WebAssembly.instantiate took 227.6ms (54.4mb/s) Chrome Canary: WebAssembly.instantiate took 8576ms (1.4mb/s) Wow. (Edit: And I believe that's not even using the streaming compilation mentioned in the article, it's just the new baseline compiler in action)

styfle 9y ago
Where did you even find that repo? Was that mentioned in the article or is this your code?
- callahad 9y ago
  Linked in the article under "give it a try" in the sixth paragraph. The repo itself is at https://github.com/lukewagner/test-tanks-compile-time https://github.com/lukewagner/test-tanks-compile-time
callahad 9y ago
> I believe that's not even using the streaming compilation mentioned in the article That's correct. Streaming compilation would finish earlier, but might actually benchmark more slowly because you'd be adding in the time that the compiler is idle and waiting for the network to catch up. Preloading the .wasm file in the test lets us measure just the speed of the compiler, independent of the network.
- derefr 9y ago
  An interesting benchmark would be the total energy usage of the CPU while loading a page in each browser. The compiler managing to idle waiting for network packets should theoretically allow some of the CPU cores to enter sleep states. (Not all, since others are still busy rendering the page, and at least one is busy doing whatever non-DMA kernel bits are involved with receiving the network packets.)
  - 9y ago

stevemk14ebr 9y ago

Caching of compiled code! As i read it they want to cache the wasm bytecode at the client level. What if servers did the caching instead? Group clients by the architectures they use and serve the cached bytecode to the right 'groups' of clients.

sjrd 9y ago
That would assume you trust the server not to give you malicious machine code (which you of course cannot!). wasm is specified in such a way that it is still sandboxed by the VM that compiles it. If you fetch arbitrary machine code, you cannot verify it and that leads to huge security holes!
- chrisseaton 9y ago
  > which you of course cannot! Didn't Google's NaCL implement verification of sandboxed machine code?
  - sjrd 9y ago
    Maybe, but at what cost? I wouldn't be surprised if the cost of verifying the machine code was higher than the cost of compiling wasm to machine code.
- hateduser2 9y ago
  Why can’t you just cache it with a hash?
  - icebraining 9y ago
    How would the client know the hash is valid?
    - Zpalmtree 9y ago
      Perhaps he means you compile the code locally, hash it, and then next time you can fetch the compiled code from a server, and check the hash matches?

sjrd 9y ago

Nice article. Although, as always with articles on WebAssembly, it keeps repeating that wasm is faster than JavaScript, without ever mentioning the limitations of wasm wrt. JS (no GC, no interaction with the DOM or with JS libraries besides numbers, etc.). And that means there are zillions of developers who keep being misled in thinking stuff like "Why don't you compile to wasm to make your stuff faster?". That includes absurdities like "We should write a compiler from JavaScript to wasm to make all our JS faster!"

gigatexal 9y ago
Yeah I read the article too and thought: but what can I do with wasm?
- deleted 9y ago
  [deleted]
- kbumsik 9y ago
  You can import C (and many others) libraries without re-implementing it in JS for browsers. In my company's case, we compile C-written libopus almost directly to encode audio streams into Opus on browsers, not on servers. In this way we benefit much less data traffic and server CPU loads.
  - QAkICoU7IDNkpFu 9y ago
    >You can import C (and many others) libraries This means compiling the lib to wasm, right? At first I was thinking of running JS clientside that imports some C lib somehow, which confused me.
    - kbumsik 9y ago
      Yes. The C lib will be compiled into wasm. You can use the wasm-compiled code in a browser through JS wrappers. So you can say importing C lib on JS client side anyway. Note that wasm's main objective is to run non-JS code on browsers, not for a faster JS. Here is the best overview and tutorial I've ever seen in HN , if you are interested: https://news.ycombinator.com/item?id=15958827 https://news.ycombinator.com/item?id=15958827

Twirrim 9y ago

Out of curiosity, using just released versions of browsers on this 2015 mac pro: Firefox 57: WebAssembly.instantiate took 2990.2ms (4.1mb/s) Chrome 63: WebAssembly.instantiate took 8736.9ms (1.4mb/s) Safari 11.0.2: WebAssembly.instantiate took 10341ms (1.2mb/s) If more speed is about to arrive, wow. I'm curious what optimisations are needed / valuable for wasm files to improve streaming performance. I'm assuming if, e.g.: def foo(baz): bar(baz) ... def bar(baz): baz = baz +1 Then compilation would start and get stuck until it had a definition for bar? If so, presumably the next build time optimisations for a website will be to shuffle the code around in to as optimal an order as possible so as to improve streaming compilation speed?

kllrnohj 9y ago
Function declarations are independent from function bodies. So think C/C++ headers/source file splits. You don't need to know what code is in bar if you know it takes 1 argument of type int and returns an int. That's all you needed to know how to call it successfully, so you can compile foo in your example perfectly fine. You just need to patch up the call location later when bar is resolved to actual address (this is the "link" step in a typical AOT compilation chain, or done by the loader if it's a dynamic dependency)
- jnordwick 9y ago
  Considering the major optimization in compiling is inlining, knowing the function body is very important to compilation, but I guess that can be pushed off until the next tier.
  - rcthompson 9y ago
    I imagine the first pass could inline functions that are already compiled and skip functions that have yet to be compiled. Maybe tools that generate wasm will start reordering the functions they send to allow optimal first-pass inlining.
  - callahad 9y ago
    Don't forget about the local compilation phase from C/C++/Rust to WebAssembly before you ever hit the browser. At that point, LLVM is free to optimize and inline just like with any other binary target.

nothrabannosir 9y ago

does someone here, familiar with webassembly semantics, know if it’s theoretically possible to start streaming execution of code? I.e. as soon as the “main” (?) function is in, and block on every function call which is not yet compiled, recursively? Or could the last block of webassembly bytecode potentially change the semantics of the first? Sooner or later, that’s an avenue people will want to explore, I assume?

tsavola 9y ago
Yes, it's possible. No, the last block won't change the semantics of the first.

JepZ 9y ago

Quoting Yehuda Katz, the co-creator of Ember.js, when it comes to JS-sizes is kinda hilarious (random google result): https://gist.github.com/Restuta/cda69e50a853aa64912d https://gist.github.com/Restuta/cda69e50a853aa64912d No offense to Yehuda in general (he is doing great work), but Ember.js so ignorant of any js-size recommendations, that it seems weird to quote Yehuda in that context.

MaxBarraclough 9y ago

A two-tier JIT. Interesting to see tiered JIT compilation catch on the way it has. I seem to remember a few years ago reading that the Java HotSpot team had given up on tiered JIT compilation as being not worthwhile. How far we've come. A whirlwind tour of todays JITs (apologies for the million links): .Net Core seems not to use tiered compilation. It never interprets the IR; everything is run through the same JIT compiler. https://github.com/dotnet/coreclr/issues/4331 https://github.com/dotnet/coreclr/issues/4331 HotSpot uses three tiers these days (counting direct interpretation as a tier) - https://docs.oracle.com/javase/8/docs/technotes/guides/vm/performance-enhancements-7.html#tieredcompilation https://docs.oracle.com/javase/8/docs/technotes/guides/vm/pe... JavaScriptCore/Nitro seems to use four - https://webkit.org/blog/3362/introducing-the-webkit-ftl-jit/ https://webkit.org/blog/3362/introducing-the-webkit-ftl-jit/ Edge's Chakra engine has two - https://blogs.msdn.microsoft.com/ie/2014/10/09/announcing-key-advances-to-javascript-performance-in-windows-10-technical-preview/ https://blogs.msdn.microsoft.com/ie/2014/10/09/announcing-ke... V8 seems to use two - https://v8project.blogspot.co.uk/2017/05/launching-ignition-and-turbofan.html https://v8project.blogspot.co.uk/2017/05/launching-ignition-... Firefox's SpiderMonkey JS engine uses two - https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/Internals https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Sp...

pjmlp 9y ago
.NET focus always was native code, either AOT with NGEN or JIT on load. The only variants of .NET with interpreter support were from 3rd party implementations, and the .NET Micro Framework, used in NETduino. And now their focus seems to be to improve their AOT story. Another interesting evolution was Android, with Dalvik and its basic JIT, ART with AOT on installation, to ART reboot with an interpreter in Assembly, followed by JIT and AOT code cache with PGO.
- bad_user 9y ago
  On .NET Core's focus, that's not actually true: http://mattwarren.org/2017/12/15/How-does-.NET-JIT-a-method-and-Tiered-Compilation/ http://mattwarren.org/2017/12/15/How-does-.NET-JIT-a-method-... Android optimizes for battery life, but it's also worth noting that Dalvik was a really rudimentary JIT, having no benefits from JIT compilation, only drawbacks, the ART with AOT being a good upgrade. But tiered compilation is in a different league, being about speculating what's going to happen depending on what the process has witnessed thus far. The point of tiered compilation is to profile/guard stuff at runtime and recompile pieces of code based on changing conditions, which is how you can optimize virtual call sites or other dynamic pieces, which you can't do ahead of time because the missing part is the decompiler which can revert optimizations based on invalidated conditions. It's really interesting actually, because you can profile a C++ app and use that to optimize your AOT compilation, but the compiler is still limited by the things it can prove ahead of time, or otherwise it would be memory unsafe.

gkya 9y ago

I'm not familiar with Web Assembly, but the recent trend is that as the downloads become faster, web performance in a vanilla browser becomes slower, because websites just send more stuff to you. Pages grow toward infinity. Also, if, like @sjrd mentioned, this code can't manipulate DOM or can use only a restricted set of JS objects, the where will the gain be? Is this intended to be used for number crunching code in the browser runtime? Help bitcoin miner scripts? What's the purpose then?

seren 9y ago
Running a native app compiled to wasm into the browser ? So the opposite of running an Electron app?
- wlesieutre 9y ago
  Calling it now: 1) Write a Windows app 2) Run it in the browser with wasm 3) Stuff that into Electron and distribute to Mac/Linux/Windows Why distribute the electron wrapped wasm on Windows instead of using the real native Windows app? It's more consistent this way! Single codebase! Developer efficiencies!
  - pjmlp 9y ago
    Start here, http://www.mono-project.com/news/2018/01/16/mono-static-webassembly-compilation/ http://www.mono-project.com/news/2018/01/16/mono-static-weba...
  - swsieber 9y ago
    Small deviation - write a native linux or mac app (instead of targeting Windows for the initial app)... mostly because I feel like developing on those platforms is so much more enjoyable.
    - wlesieutre 9y ago
      Good point. Linux is probably the easier one because you'd need to build your UI toolkit into it and we don't have the source for Cocoa or whatever this year's Windows UI toolkit is called.

jnordwick 9y ago

I guess i don't understand the push to wasm. Why not just embed hotspot, or a branch of it. Is there any difference? Or going the other way, could hotspot be replaced with a wasm jit by compiling java to wasm? I know they have slightly different memory models, but I don't understand why they seem to be treated so separately.

vurpo 9y ago
wasm is not a browser plugin, it is part of the browser. That means the plugin architechture does not need to exist (which has already been removed by all major browsers). Building wasm support into browsers was not an easy task, and "just embed hotspot" is not any easier.
filereaper 9y ago
I'm not sure that JVM allows for streaming code, its centred around class-loading and classes in general. Shipping in fragments of code will require quite an overhaul of that entire architecture. Lambdas and other constructs were added much later via JSR-292 and invokedynamic, streaming stuff in will require quite a bit of shoehorning.
- pjmlp 9y ago
  That is implementation specific and not defined as part of the JVM specification. I don't know all the JVM implementations out there, but it wouldn't surprise me if there was one with it implemented. By the way, RMI and Jini worked by streaming code across the network.
  - dfox 9y ago
    Streaming in this case means that you can start generating (or even executing) code before you have complete input. There are some mostly ignorable reasons (having to do with bytecode verification) why you should not stream-JIT Java bytecode. On the other hand in the whole ecosystem you will not gain anything wothwhile given the fact that you need the JVM state to be essentially complete before you start executing anything and on slightly lower-level the .class file format is designed to be compact and not meaningfully streamable.

axelfontaine 9y ago

Using https://lukewagner.github.io/test-tanks-compile-time/ https://lukewagner.github.io/test-tanks-compile-time/ Chrome 63: 3143.7ms (3.9mb/s) Firefox 57: 1499ms (8.3mb/s) Edge 41: 97.3ms (127.2mb/s) !!!

MrMid 9y ago
Wow. Firefox 59: 474 ms Edge 41: 164 ms The difference is smaller with newer FF, but that is amazing from Edge!
- computerphage 9y ago
  Edge is doing something very different, see the sibling comments for details.
pimeys 9y ago
Firefox nightly on Linux: 165.6 ms (74.8 MB/s) Chromium: 1835 ms (6.7 MB/s)
isaacaggrey 9y ago
Could someone explain how Edge is performing so well or any references to what they have done in this regard? Has the Edge team already implemented this streaming and tiering compiler technique?
- DiThi 9y ago
  You tell Edge, "compile this", Edge replies "done!" when what it has done is just verifying it's valid WASM. When you then call a function, it's compiled.
  - qznc 9y ago
    So it is actually sad. It means you just downloaded a lot of stuff and only used a fraction of it.
luke_wagner 9y ago

sehugg 9y ago

That's great news. On http://8bitworkshop.com/ http://8bitworkshop.com/ I'd like to offer some additional WASM modules on-demand but they take 15+ seconds to load. (It seems 50% of the time is parsing and 50% module instantiation)

filereaper 9y ago

Total Aside. As a compiler and runtimes guy, I'm super excited for streaming compilation. I think stuff like this and ethereum for distributed computation is really cool stuff! :D

dfox 9y ago
Streaming compilation is the way it was always historically done. One reason is that computers used to not have enough RAM to store whole non-trivial programs in it in AST or another intermediate form. Second reason is that this approach matches how the underlying theory of languages and automatons works. One can view modern AST producing compiler frontend as compiler that compiles it's input into program that builds the resulting AST. On the other hand many modern optimalization passes simply cannot be done in streaming manner or even by any pushdown automaton.
- caf 9y ago
  Yes, which is likely why in older languages like C you must declare symbols before you use them etc.

bsimpson 9y ago

Would be nice if it mentioned WebAssembly in the title - I presumed this was a new feature for JS in Firefox Quantum.

markdog12 9y ago
I originally put WASM in the title, not sure what happened.

bzadoroz 9y ago

Using https://lukewagner.github.io/test-tanks-compile-time/ https://lukewagner.github.io/test-tanks-compile-time/ Pale Moon 27.7.0: "fetch is not defined"

breatheoften 9y ago

Does wasm do runtime code specialization? I wonder if there will end up being a way to do to timing attacks against the optimizing wasm compiler/linker step ... Is it possible to setup code such that the optimization time depends on the runtime inferred type of an 'x' that you aren't supposed to have access to ...?

yoklov 9y ago
The term you’re looking for is speculation, not specialization, but no, I don’t think it does either. C++ and other languages targeting WASM often do type specialization, but it’s entirely done before the browser sees WASM, and has nothing to do with what you’re describing. (Which would be speculative compilation). I’d imagine that nobody does speculative compilation since the benefit is too low given how fast the network is. Also, yes, there would be security concerns.

microcolonel 9y ago

On a desktop, we compile 30-60 megabytes of WebAssembly code per second. That’s faster than the network delivers the packets. Funny enough, on my workstation it seems to compile something more like 60-80 MiB/s, to keep up with my network which was recently upgraded to gigabit. Very impressive stuff, I hope workstation CPUs can keep pace with networks.

bobsc123 9y ago

This cracks me up. Modern web browsers really started to evolve in the 90's when security problems really ramped up. You used to just download excecutables and run them on your computer because the functionality wasn't there otherwise. Flash and Java applets were the initial answer to that before Javascript and HTML evolved. We've come almost full circle to browsers basically being little VM's that can do anything again, the main reason they were developed in the first place. Most people's entire computer experience is now in the browser and here come executables again which will require another internal layer to mitigate problems.

NanoWar 9y ago
Actually, browsers are designed from the ground up to handle insecure code! That's pretty awesome, but comes at a cost: The mentioned additional layer, battery, speed? And it's platform independent!
ajkjk 9y ago
The end state ends up looking a lot like (the user-facing side of) an operating system, except that: * the filesystem is cloud storage (Drive/Dropbox/what have you -- the Unhosted (https://unhosted.org/ https://unhosted.org/) architecture) * the apps are insecure but open-source by requirement (interpreted jS) * ... running in a controlled sandbox (the browser) * ... using a standard UI language (HTML/CSS) * with functionality modifiable/overridable by user preference (extensions) It's pretty much the ecosystem you would want if you were building this from scratch! Except you'd want Html/CSS/JS to be much more intelligently designed from the start (I'm waiting so eagerly for the day that browsers natively run more scripting languages than just JS...) It never could be done in the 90s because everything ran too slowly, but it's feasible now.
- pjmlp 9y ago
  > It never could be done in the 90s because everything ran too slowly, but it's feasible now. It used to be called Lisp Machines, Smalltalk, Oberon Juice, Java Jini, Inferno.
hathawsh 9y ago

jokoon 9y ago

That's cool, but until I see it coming to smartphones, that won't really be useful, except for gaming. In theory this could really be the universal vm for the web everyone needed, but it's still lacking real sockets and dom support.

aidenn0 9y ago
It's available on android now...
- jokoon 9y ago
  I mean as an universal way to make apps

StreamBright 9y ago

What are the security implications of wasm?

markdog12 9y ago
Good writeup here: http://webassembly.org/docs/security/ http://webassembly.org/docs/security/ It runs in existing browser VMs, which have been pretty battle tested. Another interesting note is that threads are now on hold for WebAssembly due to Spectre, that is, SharedBufferArray has been disabled. Hopefully it can be enabled in the future.

mrmondo 9y ago

Firefox Beta (58) macOS 10.13.2: WebAssembly.instantiate took 151.9 ms (81.5 MB/s)

Dolores12 9y ago

Guess what, downloading compiled executable code is even faster. Is that where we are heading to? Flash 2.0? Wouldn't it be great to save all the electric power that was used to compile very same code on millions of computers every day?

__s 9y ago
Sigh... let's have this thread again. Needs to be effectively sandboxed. Modern JS as a compiler target is just as opaque. JIT compilation is a win on energy relative to the energy spent in a slow interpreter. JITing WASM will take less energy than JITing emscripten-js (nevermind the energy to send over the wire)
littlestymaar 9y ago
Downloading a bigger executable wouldn't necessarily be faster actually, it depends on the size difference and the client bandwidth.[1] Additionnally, it wouldn't be portable (executable compiled for desktop wouldn't run on mobile). [1] See this comment : https://news.ycombinator.com/item?id=16171133 https://news.ycombinator.com/item?id=16171133
- Dolores12 9y ago
  What you are saying is that source code is less in size than compiled native code which is nonsense.
  - littlestymaar 9y ago
    First of all I wasn't talking about source code, I was comparing the output of a C/Rust->wasm compiler to the one of a C/Rust->x86 compiler. Since the wasm virtual machine has a JIT, I believe the compilation to wasm isn't too aggressive with optimizations. And since those make the binary bigger, I assume a wasm output would be lighter than an x86 one. I didn't benchmark it though. And if you compare the size of the binary output with the size of the source code, the binary is bigger in many cases because of optimizations (and runtime size, for small programs). Additionally, the source code can be gziped with a good compression factor whereas the binary cannot. Then 99% of the time, the source code is lighter to send over the internet than the compiled binary.

rmrfrmrf 9y ago

> But there’s no good reason to keep the compiler waiting. It’s technically possible to compile WebAssembly line by line. This means you should be able to start as soon as the first chunk comes in. Maybe they can optimize further by speculating what the next line will be...

daveheq 9y ago

Where do I buy the Firefox coin?

deleted 9y ago

[deleted]