20 ms·
Not So Fast: Analyzing the Performance of WebAssembly vs. Native Code
- mikece 7y agoSo... don't completely discard native desktop apps just yet? That said, I suspect we'll soon see something akin to Eelctron for WebAssembly: a toolkit for making UIs that acts as a permissions context/bridge to making calls into the host operating system's API. Or maybe WebAssembly won't be the revolutionary tidal wave of change that many of its proponents hope it will be?
- jtbayly 7y agoWon’t that be a security nightmare?
- kowdermeister 7y agoElectron apps can already access the host OS, spawn child processes: https://github.com/martinjackson/electron-run-shell-example https://github.com/martinjackson/electron-run-shell-example
- mikece 7y agoDepends on how well the sandboxing it set up, I suppose.
- Sean1708 7y agoSurely it wouldn't be any more of a security than a native app, since this hypothetical Electrasm would itself be a native app?
- kowdermeister 7y agoIt can already make use of WebAssembly: https://github.com/anderejd/electron-wasm-rust-example https://github.com/anderejd/electron-wasm-rust-example
- Octoth0rpe 7y agoTl;dr: 2x slower in FF, 2.5x slower in chrome. Depending on the application, that doesn't seem to be a huge tradeoff. Certainly many developers are comfortable writing things in python, which is likely hundreds of times slower than C++ in many problem spaces. TBH though my big hope for web assembly has nothing to do with the web at all, but as a distribution format for drivers that can be installed on any cpu arch/kernel. Finally, my dream of BSD on risc-v is achieved! ....or more likely, just running linux on arm.
- CGamesPlay 7y agoI hate to break it to you, but I Java didn’t do this for you I don’t think WebAssembly is going to either. :-/
- royjacobs 7y agoYou don't mind drivers being half as performant as they were before, with the only benefit being they're easier to distribute? Why? How is that beneficial to anyone except the person packaging the driver?
- bad_user 7y agoThe browser is a really good sandbox. The security of browsers is also what makes web apps easy to distribute. That said I don't see the point of writing drivers in webassembly. If you want better security for drivers, there are better endeavors like microkernel architectures which Linux still isn't using.
- Octoth0rpe 7y ago> You don't mind drivers being half as performant as they were before, with the only benefit being they're easier to distribute? A different way of phrasing this would be "would you prefer a driver that uses 2% of your cpu vs no driver at all? Note that if the hypothetical non-existent driver was written in c, it would only take 1% of your cpu" > Why? Because the alternative for many alternative OSes is no driver at all. > How is that beneficial to anyone except the person packaging the driver? Because if it's more convenient for the packager, then there's a better chance that they'll package a driver that way, which means I'm more likely to actually get the driver in the first place.
- BrissyCoder 7y agoIt's early days. Calm down. It will speed up.
- inlined 7y agoThe reserved registers may always be a problem. It reminds me of the people who complained about massive slowdown in games when Vista released. The games were so optimized to fit in a single page that the few new pieces of state in the window manager caused page thrashing.
- titzer 7y agoFor single-instance scenarios with a single memory, I think using a 0-based memory will be a lot more efficient than reserving a register for the memory base. That doesn't fly when a Wasm engine is embedded into an arbitrary C++ program, but for standalone scenarios this may be feasible.
- iTokio 7y ago> an offline compiler like Clang can spend considerably more time to generate better code, whereas WebAssembly compilers must be fast enough to run online > [security] checks are necessary for WebAssembly’s safety > current JavaScript implementations reserve a few registers for their own use, which increases register pressure on WebAssembly
- 0815test 7y agoRuntime security checks are not necessary for WebAssembly to be safe. You could compile from a memory-safe source language to start with and add support in the WASM format for proof-carrying code. Then all that's needed in the WASM runtime is a static check of the added proof information, which could be very quick. Runtime checks could still be used to run WASM that doesn't carry proofs of safety, keeping the same performance cost. (Some occasional runtime checks would nonetheless be needed, since safety against e.g. excessive stack use could depend on what the program is doing at runtime; but they would be quite rare and come with very little overhead.)
- PudgePacket 7y ago> > an offline compiler like Clang can spend considerably more time to generate better code, whereas WebAssembly compilers must be fast enough to run online https://hacks.mozilla.org/2018/01/making-webassembly-even-faster-firefoxs-new-streaming-and-tiering-compiler/ https://hacks.mozilla.org/2018/01/making-webassembly-even-fa... > One of these speedups is streaming compilation, where the browser compiles the code while the code is still being downloaded. Up until now, this was just a potential future speedup. But with the release of Firefox 58 next week, it becomes a reality. > Firefox 58 also includes a new 2-tiered compiler. The new baseline compiler compiles code 10–15 times faster than the optimizing compiler. Seems to deal with the "whereas WebAssembly compilers must be fast enough to run online"
- azakai 7y agoStreaming and tiering do help, and I believe all browsers do that. Still, the article's point is valid that offline compilers can spend more time compiling, because even if you have a faster tier before you, running a slower compiler eats up battery and increases the time before the code runs at full speed.
- FpUser 7y agoOk. At some point in the future the browser/wasm will have won. So instead of Linux/Windows/OSX we will have Firefox/Safari/Chrome/Whatever all with their own implementation gotchas, missing features and performance loss. All for the sake of browser vendors trying to displace the existing native deployment platforms. They sure have nice financial incentive. Why do we - developers fall into this trap? We've been there quite a few times already, with cross platform standards. No matter what vendors will fsck'em up with proprietary things.
- kowdermeister 7y ago> Why do we - developers fall into this trap? Regardless of is this a trap or not, it's obvious to me that developers want simplicity and consistency. WA provides both so don't wonder why it's taking off. > So instead of Linux/Windows/OSX we will have Firefox/Safari/Chrome/Whatever all with their own implementation gotchas I let the compilers worry about this :)
- FpUser 7y agoYeah, right. Compilers do not deal with the difference in presence / absence and different implementations of various features on different platforms, be it native OS or Browser OS.
- kowdermeister 7y agoIs that so? Worst case I can imagine is to have the same compiler emit app-firefox.wasm and app-chrome.wasm binaries from the same codebase, but I think this is highly unlikely. But there probably always will be a safe flag that generates everywhere running binaries.
- FpUser 7y agoAnd by then you will have discovered that your software can not do this and that after being relegated to a lowest common denominator state.
- 7y ago
- jacknews 7y agoI would say even 50% slow down is a fair trade-off for very many applications, for the benefit or running almost anywhere, without installation etc. That was java's claim (but not the reality), what's the slowdown there?
- eitland 7y agoIn a lot of cases Java is faster than what average developers would write in C. That is except startup time.
- pulse7 7y agoNot So Fast... yet... (implement first, optimize later)
- jakobegger 7y agoCan someone explain a concrete use case for web assembly? Are there projects out there using web assembly in production? What kind of application needs more performance than Javascript can offer, but doesn't need full native performance?
- jacknews 7y agoI think it's more a case of wasm being a more natural target for compilers, meaning you can write or port applications in other languages to the web without the much-worse performance (and size) implications of compiling to interpreted javascript.
- aldanor 7y agoYou can, for instance, run Python in your browser by compiling RustPython to WASM
- dakom 7y agogamedev comes to mind, where performance isn't only about raw power but also avoiding garbage-collecting hiccups which are unavoidable in JS
- MikusR 7y agoBeta of Google Earth is Wasm.
- josalhor 7y agoUnity 3D targets Web Assembly now for the web, which makes much more sense than JavaScript, even if asm
- DonHopkins 7y agoYes, Unity3D's WebAssembly back end is a very widely used, non-trivial, practical, and successful example.
- sp332 7y agoThe Internet Archive has a ton of in-browser emulators that can boot many old computers and arcade cabinets in the browser. https://archive.org/details/msdos_Oregon_Trail_The_1990 https://archive.org/details/msdos_Oregon_Trail_The_1990 Not all of these will boot, but most will. https://archive.org/details/softwarelibrary https://archive.org/details/softwarelibrary & https://archive.org/details/internetarcade https://archive.org/details/internetarcade
- danieldk 7y agoSlower at an average of 50% looks acceptable to me for many applications. However, for me personally I wonder: what's the point? I would go from Linux -> glibc -> native application to Linux -> glibc -> Web Browser -> WebAssembly application. Yet another layer of abstraction. Sure, WebAssembly can be run directly on any platform. But precompiling an application is not a problem (my Linux distribution provides binary builds). So, this seems to be interesting to (primarily) web applications. However, I try to avoid web apps, because they are typically proprietary, a moving target, the owner can randomly decide to charge me for things, or (ab)use my data, and they feel non-native. (I do see the benefit for people who use web apps, since WebAssembly is typically faster than Javascript and opens up web app programming to more languages.)
- cjfd 7y agoPeople might start baking chips that can natively run web assembly at some point. If that happens we have found a very expensive path from running native binaries to running native binaries for the purpose of making this independent of the platform.
- not_a_cop75 7y agoLol. Well, I mean nothing is outside of the realm of possibility, but remember those native Java chips! Some thought those were going to be a big win. And then you had companies like Transmeta trying to do the exact opposite - interpretively optimize all the things....or something like that.
- microcolonel 7y agoUltimately the features involved in optimizing WebAssembly are even more variable than they are for Java, so a Jazelle approach is probably even less useful.
- CalChris 7y agoTransmeta JIT'd x86 into VLIW. That was right before the Power Wall became an obstacle and CPUs were already burning up. Transmeta in a sense invented low power computing but then Intel said thank you very much; we'll take it from here. It wasn't that they didn't deliver. It's just that they were competing against Intel in the x86 space. A lot of the Transmeta team moved to nVidia and work on Denver but that's different and they got an architecture license from ARM to do that.
- flohofwoe 7y agoFWIW: in my 8-bit emulators I'm closer to the 10% performance difference for code compiled in clang with -O3 (the native version uses latest Xcode's clang, the WASM the latest LLVM WASM backend via emscripten): https://floooh.github.io/tiny8bit/cpc-ui.html https://floooh.github.io/tiny8bit/cpc-ui.html The time in the top-right is the time spent per frame in the emulator, in the WASM version that's somewhere around 2.5 to 2.8ms on my laptop. The native version is somewhere between 2.0 and 2.2ms. From my experiments, it's quite easy to accidentally hit a "slow path" on the WASM side, for instance registers are spilling earlier (AFAIK because WASM has one or two registers set aside). Things are getting better though. About a year ago, the same emulator code seemed to have triggered some specific slow-path and the WASM version was nearly 4x slower. Of course it's always possible to put more time into optimizing the native version of a piece of code when targeting a specific compiler version, compiler-specific intrinsics and a specific CPU model. This is where WASM will always lag behind, with a portable ISA you'll always hit a wall where further optimization simply isn't possible. So from my experience: somewhere between 10% slower and 3x slower is all within the "expected" range, the more important part is that performance is predictable for a given piece of code compiled with a given compiler version (e.g. you won't have the GC come in and do its thing for a few hundred milliseconds). And at least the emscripten team is very responsive and interested in such pathological cases.
- hinkley 7y agoBut 10% is something I can easily sell to my boss. I only need one or two compensating qualities for that argument to succeed. If I'm getting work done faster, prioritizing a couple more perf stories is a reasonable tax to pay.
- tracker1 7y agoIt depends on what you are trying to do... you can only spend so much CPU time on a blinking cursor waiting for a user to click/type something.
- titzer 7y agoPlease actually read the paper. As usual, the mean is a poor approximation of a distribution. This paper states repeatedly that it was able to reproduce the experimental results from the 2017 PLDI paper, which primarily used PolyBenchC as the suite. My team wrote that paper. We didn't cherry pick. The issue is mostly that we suck more on Spec2k6 than we realized. Since publishing that paper we do track Spec2k6 internally but we don't spend time tuning for it. /v8 wasm runtime TL
- zenlibs 7y agoThank you for the clarification. Some questions related to V8 perf: 1. WASM <--> JS/Host calls seem to have a higher cost on chrome than FF [1][2]. Is that in your optimization roadmap? 2. Wasmer tracks V8 perf against other backbends, and the LLVM backend consistently outperforms V8 [3]. Can anything be done to address this, especially for node, where the compile time cost of a super optimizing compiler might be beneficial in certain applications? [1]: https://bnjbvr.github.io/perf-wasm-calls/ https://bnjbvr.github.io/perf-wasm-calls/ [2]: https://hacks.mozilla.org/2018/10/calls-between-javascript-and-webassembly-are-finally-fast-/ https://hacks.mozilla.org/2018/10/calls-between-javascript-a... [3]: http://speed.wasmer.io/comparison/ http://speed.wasmer.io/comparison/
- tom_mellior 7y agoAre you looking into some of the issues raised in the paper? Better use of register-memory instructions seems like it might be a relatively big win for relatively little effort. The code in lines 35-37 in Figure 7(c) looks like you should be able to fold the memory operand even in a simple peephole pass before register allocation, no? (I know, I know, patches are probably welcome ;-))
- hellofunk 7y agoThe article claims that this is a revelation, but I’ve been hearing for at least one year already that webassembly comes at best around half the speed of native.
- mises 7y ago10% slower is actually not that bad, when compared to the safety and portability not given by C/C++. Past 10% performance parity is the stuff of performance optimization challenges; we may have to accept that. We're all putting up with regular 10% drops in our CPU performance because intel was stupid about alot of things; maybe we could give 10% up for common applications (let's face it, most aren't CPU-bound) having more safety. Of course, I don't know if wasm is the answer. Jury's still out for whether it's great in native or not; honestly, GUI is still the issue.
- bastawhiz 7y agoSo many comments here missing the point. As a developer, I can use wasm to ship a single binary that works on Windows, Mac, Linux, Android, and iOS. I can use the language that I want. I don't need to have Xcode, visual studio, android studio, or any other set of tools: you can reasonably build an app with your text editor and LLVM. And then, you can use my app by typing its address into your address bar. No installation needed. No polluting your home directory. No registry entries. It's sandboxed, you're never giving the app permissions that you don't intend. A service worker can be used to make the app available offline. My dear grandmother could use this without confusion. My dad can use these apps and feel confident he's not not infesting himself with malware or bundled software. The complaints about "the web doesn't feel native!" and "it's less secure!" and "it's just as easy to ship a native app [for my platform of choice]!" are tired arguments. We've finally got a simple way to distribute apps fully cross platform in a way that CAN respect privacy and guarantees a reasonable level of security. And, it's pretty darn fast.
- snek 7y agoThe thing I'm most excited about right now is WASI. It provides abstractions for system calls like fs, sockets, graphics, etc. But the best part is that the entire API is being designed with security and capabilities in mind. WASM binaries have to explicitly declare resources they want access to, or risk not being allowed to access them. (The application could also choose to further prompt the user, but i think in most cases they will just be denied). Once resources are acquired, we use unforgable handles to refer to them. It is a super cool and declarative way to make sure people know exactly what a wasm bin will try to access before it's even run.
- nteon 7y ago(I'm one of the authors of Not So Fast) In the paper we build on previous work from our group, Browsix ( https://browsix.org https://browsix.org ) to provide a similar level of abstractions to WASI (the syscall layer). We've talked about having Browsix support running programs targeting WASI in the browser. One of the great things about supporting "Unix" in the browser is that it makes bringing a whole class of already written applications and library into the browser. I'm excited to see what people do with WASI, but its not clear to me what a program designed from the ground up around capabilities would look like!
- gok 7y agoIf anything this paper is unfair to the native compiler; they compare V8 and SpiderMonkey from April 2019 to a clang release from March 2017, and they turn off autovectorization. That being said, most of the slowdowns they identify seem to be deficiencies in the WASM engines rather than fundamental problems with WASM itself.
- wiremine 7y agoHow much of this performance gap is due to the inherent limitations of the WebAssembly technology, and how much is due to it being relatively new? I.e., do we expect to see performance improvements similar to what we've seen in JavaScript over the last two decades? Or are we going to need better hardware to realize better performance?
- Paul-ish 7y agoSection 6.4 tries to answer this question. > It is worth asking if the performance issues identified hereare fundamental. We believe that two of the identified issues are not: that is, they could be ameliorated by improved implementations. WebAssembly implementations today useregister allocators (§6.1.2) and code generators (§6.2.1) thatperform worse than Clang’s counterparts. However, an offlinecompiler like Clang can spend considerably more time togenerate better code, whereas WebAssembly compilers mustbe fast enough to run online. Therefore, solutions adoptedby other JITs, such as further optimizing hot code, are likely applicable here. > The four other issues that we have identified appear to arise from the design constraints of WebAssembly: the stackoverflow checks (§6.2.2), indirect call checks (§6.2.3), andreserved registers (§6.1.1) have a runtime cost and lead to in-creased code size (§6.3). Unfortunately, these checks are nec-essary for WebAssembly’s safety guarantees. A redesigned WebAssembly, with richer types for memory and function pointers [23], might be able to perform some of these checksat compile time, but that could complicate the implementation of compilers that produce WebAssembly. Finally, a Web-Assembly implementation in a browser must interoperate witha high-performance JavaScript implementation, which mayimpose its own constraints. For example, current JavaScript implementations reserve a few registers for their own use,which increases register pressure on WebAssembly.
- wiremine 7y ago> A redesigned WebAssembly, with richer types for memory and function pointers [23], might be able to perform some of these checks at compile time, but that could complicate the implementation of compilers that produce WebAssembly. Great info, thanks for sharing! So, to summarize: Performance will get a bit better for WebAssembly 1.0, but we'll need a WebAssembly 2.0 (for lack of a better term) to realize major performance improvements. And/Or we'll need advancements in JIT technology. Beyond that, there will always be some limitations because of the Javascript interop requirements. And, as other threads here have pointed out... maybe who cares? It's still pretty fast!
- devit 7y agoIt looks like there's another issue that is not quite mentioned: the Chrome code generator is using %rbx to point to the start of the WebAssembly memory, which means that the available addressing modes are severely reduced and a register is used up. It seems like that it could use a %fs or %gs segment override instead (one of which can be programmed arbitrarily at least on Linux), which should be cost-free on all x86-64 processors and would solve the issue. Once this is changed, the only remaining fundamental unfixable differences would be reduced cache efficiency from the extra bytes for the segment override, and the need to reserve an extra register for the wasm stack pointer. However, running WebAssembly code in a separate address space would allow to run arbitrary code, although this would have the drawback of higher memory usage and IPC slowdown. Is there any article discussing the rationale between Firefox and Chrome's current approach and whether they plan to use either segment overrides or process isolation?
- titzer 7y ago> the Chrome code generator is using %rbx The memory base is actually loaded from the instance pointer, and neither of these are pinned to specific registers. They're both allocatable by the register allocator. > %fs or %gs segment override instead This is tricky to do for all platforms that V8 supports. I'm not sure it's free; it's still at the very least another prefix.
- devit 7y ago> This is tricky to do for all platforms that V8 supports It can be done only where supported. Ivy Bridge and later have instructions to set them in user space (WRFSBASE/WRGSBASE), and although it needs the OS to enable it, I would expect it can be done on all significant x86-64 OSes. > I'm not sure it's free; it's still at the very least another prefix. There are benchmarks that show TLS is as fast as global variable, so it's probably free (aside from the impact on the I-cache due to increased code size, but using a general purpose register is probably much worse due to additional instructions and extra SIB bytes).
- leshow 7y agoI didn't watch the video, does it say how much of the perf gap is due to the unix API's they tried to replicate and how much is due to wasm actually being 'slower'? 50% the speed of native is really not bad, it's not like wasm is done being worked on. That's a very promising start.
- emeryberger 7y ago(I'm one of the paper authors.) The paper points out that the overhead of Browsix-WASM is under 1% across all benchmarks.
- dmitrygr 7y agoWatching wasm fans realize they've reinvented java is priceless
- jerp 7y agoComing form a win32 desktop background, how would I do a UI in web assembly?
- ScottFree 7y agoHave you seen Blazor? https://dotnet.microsoft.com/apps/aspnet/web-apps/client https://dotnet.microsoft.com/apps/aspnet/web-apps/client https://www.youtube.com/watch?v=uW-Kk7Qpv5U https://www.youtube.com/watch?v=uW-Kk7Qpv5U It packages the entire .NET framework into a single wasm app, so it's a little bloated, but it works.
- jedisct1 7y agoRelevant: benchmarking WebAssembly using libsodium: https://00f.net/2019/04/09/benchmarking-webassembly-using-libsodium/ https://00f.net/2019/04/09/benchmarking-webassembly-using-li...