5 ms·

Wasm vs. PNaCl Performance Benchmark

96 points by jjohansson 7y ago

jjohansson 7y ago
Here's a direct link to the client-side benchmark tool to test your own documents and pull numbers directly: https://www.pdftron.com/benchmarks/pnacl-vs-wasm/ https://www.pdftron.com/benchmarks/pnacl-vs-wasm/ Note that it only works in Chrome b/c it's the only browser that supported PNaCl (recently deprecated). Edge Beta (with chromium underneath) is buggy, so best not to use it for this tool.
- MuffinFlavored 7y ago
  > PNaCl's speed is 1.95 times faster than WASM > MacBook Pro (13-inch, 2017, Four Thunderbolt 3 Ports) I wonder why PNaCL was deprecated if it is faster in run-of-the-mill situations?
  - jfk13 7y ago
    https://blog.chromium.org/2017/05/goodbye-pnacl-hello-webassembly.html https://blog.chromium.org/2017/05/goodbye-pnacl-hello-webass...
  - leftyted 7y ago
    It only works in Chrome, which is a pretty big issue. Per wikipedia [0], other browsers rejected PNaCL because they don't like the idea of web apps running binaries as that represents a black box for most users. In comparison, wasm is "more open". Wasm code has a human-readable text equivalent (although, frankly, the number of people who will be able to make sense of it is not high and doesn't include me). [0] https://en.wikipedia.org/wiki/Google_Native_Client#Detractors https://en.wikipedia.org/wiki/Google_Native_Client#Detractor...
    - MuffinFlavored 7y ago
      What will it take to get WebAssembly performance more in line?
      - Rusky 7y ago
        I don't have a good overview but the last time I saw wasm-generated assembly it (e.g. firefox's compiler) was very very bad at actually using x86 addressing modes and other peephole-like "optimization" leading to high instruction counts and code size. This is something that will easily improve over time without a ton of work.
    - mintplant 7y ago
      (P)NaCl was also essentially defined by Chromium's implementation, so difficult to reproduce from scratch; and tightly intertwined with the surrounding Chromium codebase, so that implementation was difficult to port to other browsers. Its success depended on a total Chromium monoculture.
      - kevingadd 7y ago
        Another problem was compile times - getting an efficient emit and compile time for LLVM IR was difficult and involved having to heavily optimize/replace lots of parts of LLVM's pipeline that were never designed for turbo speed because it wasn't a priority.
    - vbezhenar 7y ago
      Machine code can be disassembled to text form. What's different?
      - deleted 7y ago
        [deleted]
    - snek 7y ago
      On the note of webassembly's text format (WAT), it's a fantastic introduction into a low level assembly. Wasm is a virtual stack machine, and it's strongly statically typed, so WAT is surprisingly readable. That being said, if the WAT you're reading wasn't hand-written, it's probably going to be unreadable anyway. Here's an example of some mutex operations in WAT: https://gist.github.com/devsnek/9274f1420b7641d1b6c34141921f0851 https://gist.github.com/devsnek/9274f1420b7641d1b6c34141921f...
      - erk__ 7y ago
        The livestreaming site twitch.tv uses wasm for their player, so you download a fifty something blob which is not too open I feel. I tried to have a look at it in wat and it was sadly pretty hard to figure out what was going on.
  - sunfish 7y ago
    WebAssembly is expected to be around a long time, so it's not enough to just measure its speed today, because engines aren't very mature yet. As engines mature, it will likely get faster. Also, performance is just one factor. Other criteria include security model, standards and governance, Web integration, nondeterminism.
    - robocop2018 7y ago
      No doubt, WebAssembly has lot's of going for it. I think the purpose of the article was to bring awareness around current performance and memory limitations so that WebAssembly can live to its full promise. Not so much an argument to keep PNaCl around.
  - snek 7y ago
    As someone who has worked with PNaCL before, I think its really important to stress how utterly terrible it is to work with. The intermediary representation for PNaCL was based off of LLVM IR, which is not (contrary to what a lot of people think) platform independent. The nail in the coffin though, was probably sandboxing. PNaCL is basically just running some opaque binary on your computer and crossing your fingers that chrome's sandboxing was up to snuff. Wasm, on the other hand, is built from the ground up as a mathematically verified sandboxed vm, and this works so well that modern wasm compilers don't even need to emit bounds checks on memory lookups in many cases. On a more organizational level, other browsers were not comfortable implementing something with so much control from a single company.
    - pjmlp 7y ago
      While the official representation is not 100% platform independent, there are some forks of it that are, e.g. Apple own variant for App Store. There was even a WWDC 2019 talk, on how it allowed a seamless 64 bit transition for watchOS.
  - kbumsik 7y ago
    PNaCL does not have much degree of ABI stability. So the compiled binary may not work in the future.
  - littlestymaar 7y ago
    Has chrome performance with wasm improved lately ? Last time I checked it was way slower than Firefox.
tonto 7y ago
Some of the things with javascript PDF renderers overlaps with what we do in bioinformatics (we make http range requests from large binary data files similar to this article's 7gb "linearized" pdf) so it's cool to see what techniques they're using on the bleeding edge
ReD_CoDE 7y ago
I'm not a programmer, so Wasm is DOM-based or Canvas-based? I mean does it support explicit DOMs?
- yoklov 7y ago
  Neither really. It doesn’t have direct access to a canvas API or to the DOM, but you can proxy calls through javascript. There’s a plan to allow avoiding this overhead, but it’s fairly early.
- snek 7y ago
  Wasm has no concept of a browser, or any outside-world interaction APIs. Wasm hosts can provide APIs, for example emscripten provides bindings to wasm for canvas and networking.
- scrungus 7y ago
  I think it depends on how you compile it i think. for example,you can use SDL or something with an emscripten compiler flag [1]. but i am not super sure how that works without canvas, i haven't tried it with normal sdl web assembly is not necessarily just for the browser though. you can have webassembly without any web[2], just to run binaries across different platforms. i think it's pretty cool [1] https://timhutton.github.io/sdl-canvas-wasm/ https://timhutton.github.io/sdl-canvas-wasm/ [2] https://hacks.mozilla.org/2019/03/standardizing-wasi-a-webassembly-system-interface/ https://hacks.mozilla.org/2019/03/standardizing-wasi-a-webas...
- ReD_CoDE 7y ago
  Thanks all
deleted 7y ago
[deleted]

96 points by jjohansson 7y ago

Here's a direct link to the client-side benchmark tool to test your own documents and pull numbers directly: https://www.pdftron.com/benchmarks/pnacl-vs-wasm/ https://www.pdftron.com/benchmarks/pnacl-vs-wasm/ Note that it only works in Chrome b/c it's the only browser that supported PNaCl (recently deprecated). Edge Beta (with chromium underneath) is buggy, so best not to use it for this tool.

MuffinFlavored 7y ago
> PNaCl's speed is 1.95 times faster than WASM > MacBook Pro (13-inch, 2017, Four Thunderbolt 3 Ports) I wonder why PNaCL was deprecated if it is faster in run-of-the-mill situations?
- jfk13 7y ago
  https://blog.chromium.org/2017/05/goodbye-pnacl-hello-webassembly.html https://blog.chromium.org/2017/05/goodbye-pnacl-hello-webass...
- leftyted 7y ago
  It only works in Chrome, which is a pretty big issue. Per wikipedia [0], other browsers rejected PNaCL because they don't like the idea of web apps running binaries as that represents a black box for most users. In comparison, wasm is "more open". Wasm code has a human-readable text equivalent (although, frankly, the number of people who will be able to make sense of it is not high and doesn't include me). [0] https://en.wikipedia.org/wiki/Google_Native_Client#Detractors https://en.wikipedia.org/wiki/Google_Native_Client#Detractor...
  - MuffinFlavored 7y ago
    What will it take to get WebAssembly performance more in line?
    - Rusky 7y ago
      I don't have a good overview but the last time I saw wasm-generated assembly it (e.g. firefox's compiler) was very very bad at actually using x86 addressing modes and other peephole-like "optimization" leading to high instruction counts and code size. This is something that will easily improve over time without a ton of work.
  - mintplant 7y ago
    (P)NaCl was also essentially defined by Chromium's implementation, so difficult to reproduce from scratch; and tightly intertwined with the surrounding Chromium codebase, so that implementation was difficult to port to other browsers. Its success depended on a total Chromium monoculture.
    - kevingadd 7y ago
      Another problem was compile times - getting an efficient emit and compile time for LLVM IR was difficult and involved having to heavily optimize/replace lots of parts of LLVM's pipeline that were never designed for turbo speed because it wasn't a priority.
  - vbezhenar 7y ago
    Machine code can be disassembled to text form. What's different?
    - deleted 7y ago
      [deleted]
  - snek 7y ago
    On the note of webassembly's text format (WAT), it's a fantastic introduction into a low level assembly. Wasm is a virtual stack machine, and it's strongly statically typed, so WAT is surprisingly readable. That being said, if the WAT you're reading wasn't hand-written, it's probably going to be unreadable anyway. Here's an example of some mutex operations in WAT: https://gist.github.com/devsnek/9274f1420b7641d1b6c34141921f0851 https://gist.github.com/devsnek/9274f1420b7641d1b6c34141921f...
    - erk__ 7y ago
      The livestreaming site twitch.tv uses wasm for their player, so you download a fifty something blob which is not too open I feel. I tried to have a look at it in wat and it was sadly pretty hard to figure out what was going on.
- sunfish 7y ago
  WebAssembly is expected to be around a long time, so it's not enough to just measure its speed today, because engines aren't very mature yet. As engines mature, it will likely get faster. Also, performance is just one factor. Other criteria include security model, standards and governance, Web integration, nondeterminism.
  - robocop2018 7y ago
    No doubt, WebAssembly has lot's of going for it. I think the purpose of the article was to bring awareness around current performance and memory limitations so that WebAssembly can live to its full promise. Not so much an argument to keep PNaCl around.
- snek 7y ago
  As someone who has worked with PNaCL before, I think its really important to stress how utterly terrible it is to work with. The intermediary representation for PNaCL was based off of LLVM IR, which is not (contrary to what a lot of people think) platform independent. The nail in the coffin though, was probably sandboxing. PNaCL is basically just running some opaque binary on your computer and crossing your fingers that chrome's sandboxing was up to snuff. Wasm, on the other hand, is built from the ground up as a mathematically verified sandboxed vm, and this works so well that modern wasm compilers don't even need to emit bounds checks on memory lookups in many cases. On a more organizational level, other browsers were not comfortable implementing something with so much control from a single company.
  - pjmlp 7y ago
    While the official representation is not 100% platform independent, there are some forks of it that are, e.g. Apple own variant for App Store. There was even a WWDC 2019 talk, on how it allowed a seamless 64 bit transition for watchOS.
- kbumsik 7y ago
  PNaCL does not have much degree of ABI stability. So the compiled binary may not work in the future.
- littlestymaar 7y ago
  Has chrome performance with wasm improved lately ? Last time I checked it was way slower than Firefox.

tonto 7y ago

Some of the things with javascript PDF renderers overlaps with what we do in bioinformatics (we make http range requests from large binary data files similar to this article's 7gb "linearized" pdf) so it's cool to see what techniques they're using on the bleeding edge

ReD_CoDE 7y ago

I'm not a programmer, so Wasm is DOM-based or Canvas-based? I mean does it support explicit DOMs?

yoklov 7y ago
Neither really. It doesn’t have direct access to a canvas API or to the DOM, but you can proxy calls through javascript. There’s a plan to allow avoiding this overhead, but it’s fairly early.
snek 7y ago
Wasm has no concept of a browser, or any outside-world interaction APIs. Wasm hosts can provide APIs, for example emscripten provides bindings to wasm for canvas and networking.
scrungus 7y ago
I think it depends on how you compile it i think. for example,you can use SDL or something with an emscripten compiler flag [1]. but i am not super sure how that works without canvas, i haven't tried it with normal sdl web assembly is not necessarily just for the browser though. you can have webassembly without any web[2], just to run binaries across different platforms. i think it's pretty cool [1] https://timhutton.github.io/sdl-canvas-wasm/ https://timhutton.github.io/sdl-canvas-wasm/ [2] https://hacks.mozilla.org/2019/03/standardizing-wasi-a-webassembly-system-interface/ https://hacks.mozilla.org/2019/03/standardizing-wasi-a-webas...
ReD_CoDE 7y ago
Thanks all

deleted 7y ago

[deleted]