4 ms·

What do you account for the perf win over Qemu? A bunch of micro optimizations, less abstraction layers, or something more systemic?

jart 4y ago
Blink is like a Tesla sports car whereas Qemu is like a locomotive. I think what may be happening, is Qemu has a lot of heavy hitting optimizations that benefit long-running compute intensive programs. But if you just want to run an big program like GCC ephemerally as part of your build system, the cost of the the locomotive gaining speed doesn't pay off, since there's nothing to amortize over. Blink's JIT also accelerates quickly because it uses a printf-style DSL and it doesn't relocate. The tradeoff is that JIT path construction sometimes fails and needs to be retried. Another great example of this tinier is better phenomenom, would be v8 vs. quickjs. Fabrice Bellard singlehandedly wrote a JavaScript interpreter that runs the Test262 suite something like 20x faster Google's flagship V8 software, because once again, tests are ephemeral. It's amazing how much quicker QuickJS is. But if you wanted to do something like write a JS MPEG decoder to show television advertisements without a <video> tag then v8 is going to be faster, since it's a locomotive. Fabrice Bellard wrote Qemu too. But I suspect his Tiny Code Generator has gotten a lot heftier over the years as so many people everywhere contributed to it. I really want to examine his original source code, since I'd imagine what he originally did probably looked a lot more like Blink than it looks like modern Qemu.
- deleted 4y ago
  [deleted]
- andai 4y ago
  Fascinating. Most JS code is ephemeral, i.e. rarely is something as intensive as video encoding done in the browser (and even then WebAssembly would usually be preferred). It seems to me like browsers would benefit from running most code in QuickJS, and then spinning up V8 only for those rare cases of long-running JS?
  - nightpool 4y ago
    "Ephemeral" is relative. Most JS code in the browser runs for at least 30 seconds, if not longer, as the user interacts with the page. That's plenty of time to spend spare cycles on JITing in the background to make responsiveness better without worrying about 100s of milliseconds of startup / shutdown latency.
  - saagarjha 4y ago
    V8 is optimized for real-world use cases, not benchmarks. Any modern browser will blow QuickJS out of the water for anything that's non-trivial.
- pwdisswordfish9 4y ago
  > Fabrice Bellard singlehandedly wrote a JavaScript interpreter that No he didn't.
  - deleted 4y ago
    [deleted]
  - googlryas 4y ago
    Charlie Gordon was involved too, I suppose would be a more constructive comment.
    - deleted 4y ago
      [deleted]
    - pwdisswordfish9 4y ago
      So we agree that he didn't write it singlehandedly, then. jonah_hill_fuck_me_right?.jpg (We don't agree on the appropriateness of the description that he "was involved", however. That sounds like an attempt to understate/diminish.)
- saagarjha 4y ago
  > Fabrice Bellard singlehandedly wrote a JavaScript interpreter that runs the Test262 suite something like 20x faster Google's flagship V8 software Something is wrong here. How did you test this? QuickJS might start up faster on very small testcases but V8 is not that slow; it needs to have very low latency on a webpage too. Did you run a debug build or something?
  - rcme 4y ago
    I have no knowledge of what allows QuickJS to run the tests faster, or if it even does run the tests faster, but QuickJS does have one big speed advantage over V8 in some circumstances: QuickJS allows ahead-of-time compilation of JS to byte code. This removes the need to parse the JS at execution time. It’s a pretty nifty feature.
    - sroussey 4y ago
      Once upon a time, Firefox used to do that for extensions.
    - fathyb 4y ago
      FWIW v8 does too through its "snapshot" feature. It's used by Chrome and Node.js to speed up the initialization of the standard library. It might even be faster because it includes both the bytecode and the heap.
- bonzini 4y ago
  Hi Justine, QEMU developer here. Great job on Blink! You have done a lot of cool work and it's been fun to follow. I enjoyed looking at different choices you made in the frontend, for example flags handling is very different from QEMU. QEMU's code generator is actually pretty fast and shouldn't really be expensive. It's a handful of passes that are run on individual basic blocks, certainly not optimal when a lot of code runs once as is the case for a very short compile but it's nothing like v8. I suspect an even more silly reason—startup time might even be the biggest factor, because I think qemu-user's startup has never been optimized. I assume both QEMU and blink binaries are statically linked (or both dynamically linked, alternatively)? Anyhow these theories should be pretty easy to disprove just by compiling something larger than hello world, so I will do it in case there's some low-hanging fruit left.
  - jvolkman 4y ago
    Is there a way to run linux binaries on macOS and Windows with qemu-user? As far as I can tell it only runs on Linux (and I guess BSD). Performance considerations are nice, but I think the most interesting thing about Blink is the promise of running small linux binaries on these other platforms.
    - bonzini 4y ago
      No, it would be easy to do on Darwin but only if you were willing to trade off emulation accuracy and QEMU wants to be able to run any Linux program. Even if you remain in the Unix world (so macOS but not Windows), system-call–level translation will let some of the differences between operating systems transpire to the program; and it will cause failures if a program has #ifdefs to distinguish between Linux and Darwin but then sees Darwin behavior under Linux. On Windows, you would basically be reimplementing cygwin—imagine what a mess it would be for a Linux program to see Windows paths with drives and backslashes. Even though blink can run any Linux program in principle, the main idea is to use it for the author's "cosmopolitan libc", to run programs that are essentially scripts compiled to machine code. So you'd write these programs portably but still be able to ship a single binary.
  - stuaxo 4y ago
- JoshTriplett 4y ago
  Would it be fair to describe Blink's JIT as more of a "baseline JIT" to QEMU's "optimized JIT", or does that analogy not accurately capture what you mean in the first paragraph?
  - bonzini 4y ago
    Both of them are pretty basic but Blink is closer to subroutine-threaded code than to a full JIT. IIRC it has no intermediate representation, liveness analysis or register allocation. QEMU has all of those but only within a basic block, so it does not do any complicated data flow analysis. It's what would be considered a baseline JIT in the JavaScript world. Rosetta is similar too as far as I understand.
- guiriduro 4y ago
  That's a tradeoff that might make sense for optimising single-shot execution environments - I'm thinking Serverless/Lambda.
saagarjha 4y ago
QEMU TCG is not particularly optimized for performance. It's not all that hard to do better than it, especially if you target only one architecture.
- bonzini 4y ago
  It's also not that easy though, and blink's code generation is comparable to QEMU in 2007 or so. Even if TCG remains relatively basic, 2x better code generation then QEMU is still a strong claim. I suspect that the reason why blink is faster in this experiment is not related to code generation, as I mentioned in another comment. Looking again at the screenshot, the 28 context switches (vs 0 for blink) might be a clue as well.
  - jart 4y ago
    Qemu generates pretty good code. I've skimmed through your codebase. Qemu knows all the ISA-specific operation constraints, that lets it work very similar to a compiler. All I'm saying is that Qemu generates that quality code slowly. That's what I mean by a locomotive. A train can go hundreds of miles per hour faster than a Tesla sports car. But which is quicker off the mark? Blink basically just glues together functions that were created by the compiler. So when it runs a program like GCC, which has an enormously long initialization ritual, the JIT path generator is able to plow through it at a speed comparable to memcpy. While I have your attention, may I ask if you've considered saving your JIT's output to a file, that can be rapidly loaded if GCC is launched a second time? Ephemeral commands tend to be executed independently many times, which would seem to favor an AOT approach. But I don't see any reason why a JIT can't persist its output to gain the advantages of AOT. I'd call it EOT.
    - bonzini 4y ago
      > may I ask if you've considered saving your JIT's output to a file, that can be rapidly loaded if GCC is launched a second time? I don't work that much on QEMU TCG actually, but it would be possible to do so. In fact recent (unrelated!) changes to QEMU might even make it possible to preserve ASLR with such a first-execution JIT compilation. That said I have rarely seen code generation in QEMU's profiles. In the end what matters is real world performance and I need to redo the test myself to understand what's going on and whether your benchmark is representative of e.g. building a small but nontrivial program (let's say blink itself) with both QEMU and blink. In that case there would be repeated cold-start recompilation, but also the compiled code would run at least once per function so QEMU would have an edge. To be honest blink is probably more like a bicycle than a sports car. It will start faster than a locomotive, but the cruise speed is definitely lower.

jart 4y ago

Blink is like a Tesla sports car whereas Qemu is like a locomotive. I think what may be happening, is Qemu has a lot of heavy hitting optimizations that benefit long-running compute intensive programs. But if you just want to run an big program like GCC ephemerally as part of your build system, the cost of the the locomotive gaining speed doesn't pay off, since there's nothing to amortize over. Blink's JIT also accelerates quickly because it uses a printf-style DSL and it doesn't relocate. The tradeoff is that JIT path construction sometimes fails and needs to be retried. Another great example of this tinier is better phenomenom, would be v8 vs. quickjs. Fabrice Bellard singlehandedly wrote a JavaScript interpreter that runs the Test262 suite something like 20x faster Google's flagship V8 software, because once again, tests are ephemeral. It's amazing how much quicker QuickJS is. But if you wanted to do something like write a JS MPEG decoder to show television advertisements without a <video> tag then v8 is going to be faster, since it's a locomotive. Fabrice Bellard wrote Qemu too. But I suspect his Tiny Code Generator has gotten a lot heftier over the years as so many people everywhere contributed to it. I really want to examine his original source code, since I'd imagine what he originally did probably looked a lot more like Blink than it looks like modern Qemu.

deleted 4y ago
[deleted]
andai 4y ago
Fascinating. Most JS code is ephemeral, i.e. rarely is something as intensive as video encoding done in the browser (and even then WebAssembly would usually be preferred). It seems to me like browsers would benefit from running most code in QuickJS, and then spinning up V8 only for those rare cases of long-running JS?
- nightpool 4y ago
  "Ephemeral" is relative. Most JS code in the browser runs for at least 30 seconds, if not longer, as the user interacts with the page. That's plenty of time to spend spare cycles on JITing in the background to make responsiveness better without worrying about 100s of milliseconds of startup / shutdown latency.
- saagarjha 4y ago
  V8 is optimized for real-world use cases, not benchmarks. Any modern browser will blow QuickJS out of the water for anything that's non-trivial.
pwdisswordfish9 4y ago
> Fabrice Bellard singlehandedly wrote a JavaScript interpreter that No he didn't.
- deleted 4y ago
  [deleted]
- googlryas 4y ago
  Charlie Gordon was involved too, I suppose would be a more constructive comment.
  - deleted 4y ago
    [deleted]
  - pwdisswordfish9 4y ago
    So we agree that he didn't write it singlehandedly, then. jonah_hill_fuck_me_right?.jpg (We don't agree on the appropriateness of the description that he "was involved", however. That sounds like an attempt to understate/diminish.)
saagarjha 4y ago
> Fabrice Bellard singlehandedly wrote a JavaScript interpreter that runs the Test262 suite something like 20x faster Google's flagship V8 software Something is wrong here. How did you test this? QuickJS might start up faster on very small testcases but V8 is not that slow; it needs to have very low latency on a webpage too. Did you run a debug build or something?
- rcme 4y ago
  I have no knowledge of what allows QuickJS to run the tests faster, or if it even does run the tests faster, but QuickJS does have one big speed advantage over V8 in some circumstances: QuickJS allows ahead-of-time compilation of JS to byte code. This removes the need to parse the JS at execution time. It’s a pretty nifty feature.
  - sroussey 4y ago
    Once upon a time, Firefox used to do that for extensions.
  - fathyb 4y ago
    FWIW v8 does too through its "snapshot" feature. It's used by Chrome and Node.js to speed up the initialization of the standard library. It might even be faster because it includes both the bytecode and the heap.
bonzini 4y ago
Hi Justine, QEMU developer here. Great job on Blink! You have done a lot of cool work and it's been fun to follow. I enjoyed looking at different choices you made in the frontend, for example flags handling is very different from QEMU. QEMU's code generator is actually pretty fast and shouldn't really be expensive. It's a handful of passes that are run on individual basic blocks, certainly not optimal when a lot of code runs once as is the case for a very short compile but it's nothing like v8. I suspect an even more silly reason—startup time might even be the biggest factor, because I think qemu-user's startup has never been optimized. I assume both QEMU and blink binaries are statically linked (or both dynamically linked, alternatively)? Anyhow these theories should be pretty easy to disprove just by compiling something larger than hello world, so I will do it in case there's some low-hanging fruit left.
- jvolkman 4y ago
  Is there a way to run linux binaries on macOS and Windows with qemu-user? As far as I can tell it only runs on Linux (and I guess BSD). Performance considerations are nice, but I think the most interesting thing about Blink is the promise of running small linux binaries on these other platforms.
  - bonzini 4y ago
    No, it would be easy to do on Darwin but only if you were willing to trade off emulation accuracy and QEMU wants to be able to run any Linux program. Even if you remain in the Unix world (so macOS but not Windows), system-call–level translation will let some of the differences between operating systems transpire to the program; and it will cause failures if a program has #ifdefs to distinguish between Linux and Darwin but then sees Darwin behavior under Linux. On Windows, you would basically be reimplementing cygwin—imagine what a mess it would be for a Linux program to see Windows paths with drives and backslashes. Even though blink can run any Linux program in principle, the main idea is to use it for the author's "cosmopolitan libc", to run programs that are essentially scripts compiled to machine code. So you'd write these programs portably but still be able to ship a single binary.
- stuaxo 4y ago
JoshTriplett 4y ago
Would it be fair to describe Blink's JIT as more of a "baseline JIT" to QEMU's "optimized JIT", or does that analogy not accurately capture what you mean in the first paragraph?
- bonzini 4y ago
  Both of them are pretty basic but Blink is closer to subroutine-threaded code than to a full JIT. IIRC it has no intermediate representation, liveness analysis or register allocation. QEMU has all of those but only within a basic block, so it does not do any complicated data flow analysis. It's what would be considered a baseline JIT in the JavaScript world. Rosetta is similar too as far as I understand.
guiriduro 4y ago
That's a tradeoff that might make sense for optimising single-shot execution environments - I'm thinking Serverless/Lambda.

saagarjha 4y ago

QEMU TCG is not particularly optimized for performance. It's not all that hard to do better than it, especially if you target only one architecture.

bonzini 4y ago
It's also not that easy though, and blink's code generation is comparable to QEMU in 2007 or so. Even if TCG remains relatively basic, 2x better code generation then QEMU is still a strong claim. I suspect that the reason why blink is faster in this experiment is not related to code generation, as I mentioned in another comment. Looking again at the screenshot, the 28 context switches (vs 0 for blink) might be a clue as well.
- jart 4y ago
  Qemu generates pretty good code. I've skimmed through your codebase. Qemu knows all the ISA-specific operation constraints, that lets it work very similar to a compiler. All I'm saying is that Qemu generates that quality code slowly. That's what I mean by a locomotive. A train can go hundreds of miles per hour faster than a Tesla sports car. But which is quicker off the mark? Blink basically just glues together functions that were created by the compiler. So when it runs a program like GCC, which has an enormously long initialization ritual, the JIT path generator is able to plow through it at a speed comparable to memcpy. While I have your attention, may I ask if you've considered saving your JIT's output to a file, that can be rapidly loaded if GCC is launched a second time? Ephemeral commands tend to be executed independently many times, which would seem to favor an AOT approach. But I don't see any reason why a JIT can't persist its output to gain the advantages of AOT. I'd call it EOT.
  - bonzini 4y ago
    > may I ask if you've considered saving your JIT's output to a file, that can be rapidly loaded if GCC is launched a second time? I don't work that much on QEMU TCG actually, but it would be possible to do so. In fact recent (unrelated!) changes to QEMU might even make it possible to preserve ASLR with such a first-execution JIT compilation. That said I have rarely seen code generation in QEMU's profiles. In the end what matters is real world performance and I need to redo the test myself to understand what's going on and whether your benchmark is representative of e.g. building a small but nontrivial program (let's say blink itself) with both QEMU and blink. In that case there would be repeated cold-start recompilation, but also the compiled code would run at least once per function so QEMU would have an edge. To be honest blink is probably more like a bicycle than a sports car. It will start faster than a locomotive, but the cruise speed is definitely lower.