3 ms·

Here's a conundrum. Fabrice Bellard's QuickJS engine takes 3 minutes to run the test262 ECMAScript conformance suite. d8 takes 37 minutes to run test262 accordi

by jart 5y ago

Here's a conundrum. Fabrice Bellard's QuickJS engine takes 3 minutes to run the test262 ECMAScript conformance suite. d8 takes 37 minutes to run test262 according to https://medium.com/compilers/testing-the-v8-javascript-engine-cbda7d9272e6 https://medium.com/compilers/testing-the-v8-javascript-engin... and it crashes for me in Chrome https://v8.github.io/test262/website/default.html https://v8.github.io/test262/website/default.html Has anyone else observed this performance disparity? Could it really be possible that a JavaScript engine written by one guy is 10x faster for everyday code than the flagship product of a flagship company? Because if that's the case it'd make Fabrice Bellard the Han Solo of programmers.

stock_toaster 5y ago
I think Fabrice Bellard (ffmpeg, qemu, Bellard's formula, quickjs, tinygl, tinycc), could _already_ be considered the Han Solo of programmers.
- pwdisswordfish9 5y ago
  Fabrice shot first!
vasco 5y ago
There's more chuck norris jokes about Fabrice than chuck norris. And in this case they are not jokes.
- jart 5y ago
  Fabrice Bellard’s tears solve P vs. NP. Too bad he's never cried. Fabrice Bellard once failed a Turing test when he correctly identified the 203rd digit of pi in less than a second. Fabrice Bellard once shifted a bit so hard, it ended up on another computer.
  - Teknoman117 5y ago
    > Fabrice Bellard once shifted a bit so hard, it ended up on another computer. And that, my friends, is essentially what networking is :)
kevingadd 5y ago
Are you sure they're running the same set of tests? There's over 10000 tests in the full set that webpage runs, but there are subsets and fabrice might be running one of the subsets. FWIW test262 falls over partway through in Firefox and I have to kill the tab, though it doesn't crash. There are a bunch of test failures as well for things that are probably not implemented by anyone (I'm curious how many of the tests QuickJS actually passes) My guess for any performance gap would be that the browser runner probably sets up an entirely separate execution context (iframe?) to run each test cleanly so they don't interfere with each other.
- jart 5y ago
  V8 has 89% coverage and QuickJS has 86%. https://test262.report/?date=2022-02-28 https://test262.report/?date=2022-02-28
  - jjice 5y ago
    To add, QuickJS is almost completely comparable in test coverage, except for internationalization (which makes sense it wasn't a priority, since it's not used in a browser).
deleted 5y ago
[deleted]
whizzter 5y ago
test262 is a conformance test and as such mostly runs small test-snippets once to verify, what counts here is setup, parsing and first-run execution. While V8 has added a fast first-stage interpreter there are probably a ton of other overheads when starting a V8 context as well as "inefficiencies" related to preparing code for later JIT optimizations. So for more CPU intensive code written in JS where the JIT activates the tables shifts radically, compare QuickJS with V8 (JIT-less) and V8 (JIT) on benchmarks, particularly Raytrace, Crypto and NavierStokes that should be pushing the computation performance. https://bellard.org/quickjs/bench.html https://bellard.org/quickjs/bench.html
- deleted 5y ago
  [deleted]
mort96 5y ago
I wrote a small lisp in Rust. I try to not do obviously stupid things, but it's just a basic parser + AST interpreter; its execution speed is slow compared to even a basic bytecode interpreter, and it's absurdly slow compared to a JIT. Yet it runs hello world in a fraction of a millisecond, whereas V8 needs tens of milliseconds. In executing one computationally intensive program, V8 would be many orders of magnitude faster than my program. But in a test which largely consists of running tens of thousands of small, uninteresting test case programs, my silly interpreter would outperform V8 by orders of magnitude. Essentially, performance is complicated, and improving throughput often has costs in other areas.
- deleted 5y ago
  [deleted]

by jart 5y ago

stock_toaster 5y ago

I think Fabrice Bellard (ffmpeg, qemu, Bellard's formula, quickjs, tinygl, tinycc), could _already_ be considered the Han Solo of programmers.

pwdisswordfish9 5y ago
Fabrice shot first!

vasco 5y ago

There's more chuck norris jokes about Fabrice than chuck norris. And in this case they are not jokes.

jart 5y ago
Fabrice Bellard’s tears solve P vs. NP. Too bad he's never cried. Fabrice Bellard once failed a Turing test when he correctly identified the 203rd digit of pi in less than a second. Fabrice Bellard once shifted a bit so hard, it ended up on another computer.
- Teknoman117 5y ago
  > Fabrice Bellard once shifted a bit so hard, it ended up on another computer. And that, my friends, is essentially what networking is :)

kevingadd 5y ago

Are you sure they're running the same set of tests? There's over 10000 tests in the full set that webpage runs, but there are subsets and fabrice might be running one of the subsets. FWIW test262 falls over partway through in Firefox and I have to kill the tab, though it doesn't crash. There are a bunch of test failures as well for things that are probably not implemented by anyone (I'm curious how many of the tests QuickJS actually passes) My guess for any performance gap would be that the browser runner probably sets up an entirely separate execution context (iframe?) to run each test cleanly so they don't interfere with each other.

jart 5y ago
V8 has 89% coverage and QuickJS has 86%. https://test262.report/?date=2022-02-28 https://test262.report/?date=2022-02-28
- jjice 5y ago
  To add, QuickJS is almost completely comparable in test coverage, except for internationalization (which makes sense it wasn't a priority, since it's not used in a browser).

deleted 5y ago

[deleted]

whizzter 5y ago

test262 is a conformance test and as such mostly runs small test-snippets once to verify, what counts here is setup, parsing and first-run execution. While V8 has added a fast first-stage interpreter there are probably a ton of other overheads when starting a V8 context as well as "inefficiencies" related to preparing code for later JIT optimizations. So for more CPU intensive code written in JS where the JIT activates the tables shifts radically, compare QuickJS with V8 (JIT-less) and V8 (JIT) on benchmarks, particularly Raytrace, Crypto and NavierStokes that should be pushing the computation performance. https://bellard.org/quickjs/bench.html https://bellard.org/quickjs/bench.html

deleted 5y ago
[deleted]

mort96 5y ago

I wrote a small lisp in Rust. I try to not do obviously stupid things, but it's just a basic parser + AST interpreter; its execution speed is slow compared to even a basic bytecode interpreter, and it's absurdly slow compared to a JIT. Yet it runs hello world in a fraction of a millisecond, whereas V8 needs tens of milliseconds. In executing one computationally intensive program, V8 would be many orders of magnitude faster than my program. But in a test which largely consists of running tens of thousands of small, uninteresting test case programs, my silly interpreter would outperform V8 by orders of magnitude. Essentially, performance is complicated, and improving throughput often has costs in other areas.

deleted 5y ago
[deleted]