5 ms·

There are some benchmarks against Deno here (as JSON files): https://github.com/elsaland/elsa/tree/master/benchmarks https://github.com/elsaland/elsa/tree/maste

There are some benchmarks against Deno here (as JSON files): https://github.com/elsaland/elsa/tree/master/benchmarks https://github.com/elsaland/elsa/tree/master/benchmarks

artagnon 6y ago
Ofcourse; Deno is V8-based, which is a lot more full-featured than quickjs. Perhaps this is a more informative benchmark: https://bellard.org/quickjs/bench.html https://bellard.org/quickjs/bench.html
- tomp 6y ago
  Wow,, this is seriously bad. Calling this fast is very misleading. QuickJS is about 0.5x as fast as V8 without JIT (and like 20x slower than V8+JIT).
  - tyingq 6y ago
    True for these benchmarks, but other than regex, most aren't what one would typically be doing with it. It would be nice if one of the benchmarks was more reflective of typical use.
  - filleduchaos 6y ago
    That a rocket is two orders of magnitude faster than a jetliner doesn't make a jetliner not fast.
    - seg_lol 6y ago
      True statement, but what does that metaphor-analogy have to do with server side JavaScript runtimes? They are also not the same thing, Dragonflys are fast, but they won't run my JS either. Our systems have absolute speeds that the client requests don't care about. They don't care if they are fast for an _x_ when that _x_ has nothing to do with the client. Minimal is a nice quality, but I don't think Elsa is fast in absolute terms or as secure as Deno. I did an analysis in a different comment, it looks like Elsa comes in at 3500 lines of code, of which Go is less than a thousand. What this does is supply an outlet for Go programmers to contribute to a JS runtime like Deno which is mostly developed in Rust and TS. From a sociological perspective, it will be interesting to see how Elsa evolves and whether Go or TS dominate future development. What I think is neat is that QuickJS comes in as more computationally efficient (benchmark score / executable_bytes_size). But that also makes sense, jits have to work pretty hard to extract that extra performance. One would have to do some sort of coverage analysis to compare quickjs to v8 --jitless, interpreter to interpreter. It would be interesting to track the commits in a codebase over time wrt speed for interpreters and jits across a set of languages. # from https://bellard.org/quickjs/bench.html # total score w/o regex In [2]: 41576 / 28000 Out[2]: 1.4848571428571429 In [3]: 1138 / 620 Out[3]: 1.835483870967742 Given Fabrice Bellards other projects, I wouldn't be surprised if a JIT landed in QuickJS.
      - filleduchaos 6y ago
        > True statement, but what does that metaphor-analogy have to do with server side JavaScript runtimes Seems pretty self-evident that there are more considerations than "literally the fastest" to be made when the decision is "how do I get from point A to point B", or in this case "how do I execute my javascript". Vehicles and runtimes can be considered fast [enough] regardless of the existence of faster options. Also, QuickJS is not a "server-side JavaScript runtime". Not all programming is web development. > Given Fabrice Bellards other projects, I wouldn't be surprised if a JIT landed in QuickJS. Personally I would be pretty surprised considering that the aim of the project seems in part to be as small as possible.
        seg_lol 6y ago
        I would like to deescalate this convo, I used overly harsh language and I apologize. Your use of fast, is in the definition fast enough for the task at hand. It is not fast in the absolute sense. I assert that Elsa being that much slower than Deno due to Deno having a JIT where as Elsa does not automatically puts them in different categories. But the stack that Elsa uses still needs Go and C, so long term for Elsa I could see it using a pure Go JS runtime in place of QuickJS.
  - jiofih 6y ago
    It’s not. V8 is incredibly fast, sometimes within 50% of C perf. This is probably on par with Ruby and other script runtimes.
pastrami_panda 6y ago
That is a horrible way to present benchmarks, but it's quite a big difference in favor of Elsa. I was under the impression Deno was built on Rust and V8 which should bode for good performance, Elsa must be doing something very good?
- hombre_fatal 6y ago
  Yeah, I've put more work into explaining the results of a benchmark I put on pastebin.
- coder543 6y ago
  Each of the benchmarks seem to take a single-digit number of milliseconds in Elsa/QuickJS. My guess is that these benchmarks were chosen because the startup time for QuickJS is probably less than the startup time for V8, but I struggle to imagine that V8 is slower once the code has warmed up. Any meaningful benchmark with JIT runtimes should take several seconds, at a minimum, in my opinion. 8 milliseconds is not enough. From what I can tell, the benchmark is including the startup time, and it isn't keeping the interpreter warm -- it starts a fresh instance for each iteration of the benchmark. This is one of the benchmarks: https://github.com/elsaland/elsa/blob/master/testing/bench_scripts/fs.js https://github.com/elsaland/elsa/blob/master/testing/bench_s... Literally a single line of JavaScript.
  - mkl 6y ago
    QuickJS's own benchmarks are here: https://bellard.org/quickjs/bench.html https://bellard.org/quickjs/bench.html V8, with or without its JIT, wins hands down. QuickJS may be small and fast to start, but fast actual computations seem not to be its strength (or intended target). Big discussion of QuickJS last year: https://news.ycombinator.com/item?id=20411154 https://news.ycombinator.com/item?id=20411154
  - Waterluvian 6y ago
    It’s amazing watching V8s JIT warm up. And eye opening on how to measure performance when profiling. A heavy call can take 200ms, then 100, then 80, 70. And suddenly where I thought the low hanging fruit is has vanished.
untog 6y ago
I'm very dubious about whether these actually match real world performance. I've done a bunch of work with QuickJS. It's awesome and the performance features you get in return for its tiny footprint is amazing. But compared to JIT-enabled V8 it's incredibly slow. The one area where it shines compared to V8 is startup time, because the engine is so much lighter. So if you're profiling "time taken to start up and execute one operation" then yes, QuickJS might look better. But that isn't how people use Node/Deno, even when you're creating a CLI tool you're loading a lot of JS and performing a lot of operations.

There are some benchmarks against Deno here (as JSON files): https://github.com/elsaland/elsa/tree/master/benchmarks https://github.com/elsaland/elsa/tree/master/benchmarks

artagnon 6y ago

Ofcourse; Deno is V8-based, which is a lot more full-featured than quickjs. Perhaps this is a more informative benchmark: https://bellard.org/quickjs/bench.html https://bellard.org/quickjs/bench.html

tomp 6y ago
Wow,, this is seriously bad. Calling this fast is very misleading. QuickJS is about 0.5x as fast as V8 without JIT (and like 20x slower than V8+JIT).
- tyingq 6y ago
  True for these benchmarks, but other than regex, most aren't what one would typically be doing with it. It would be nice if one of the benchmarks was more reflective of typical use.
- filleduchaos 6y ago
  That a rocket is two orders of magnitude faster than a jetliner doesn't make a jetliner not fast.
  - seg_lol 6y ago
    True statement, but what does that metaphor-analogy have to do with server side JavaScript runtimes? They are also not the same thing, Dragonflys are fast, but they won't run my JS either. Our systems have absolute speeds that the client requests don't care about. They don't care if they are fast for an _x_ when that _x_ has nothing to do with the client. Minimal is a nice quality, but I don't think Elsa is fast in absolute terms or as secure as Deno. I did an analysis in a different comment, it looks like Elsa comes in at 3500 lines of code, of which Go is less than a thousand. What this does is supply an outlet for Go programmers to contribute to a JS runtime like Deno which is mostly developed in Rust and TS. From a sociological perspective, it will be interesting to see how Elsa evolves and whether Go or TS dominate future development. What I think is neat is that QuickJS comes in as more computationally efficient (benchmark score / executable_bytes_size). But that also makes sense, jits have to work pretty hard to extract that extra performance. One would have to do some sort of coverage analysis to compare quickjs to v8 --jitless, interpreter to interpreter. It would be interesting to track the commits in a codebase over time wrt speed for interpreters and jits across a set of languages. # from https://bellard.org/quickjs/bench.html # total score w/o regex In [2]: 41576 / 28000 Out[2]: 1.4848571428571429 In [3]: 1138 / 620 Out[3]: 1.835483870967742 Given Fabrice Bellards other projects, I wouldn't be surprised if a JIT landed in QuickJS.
    - filleduchaos 6y ago
      > True statement, but what does that metaphor-analogy have to do with server side JavaScript runtimes Seems pretty self-evident that there are more considerations than "literally the fastest" to be made when the decision is "how do I get from point A to point B", or in this case "how do I execute my javascript". Vehicles and runtimes can be considered fast [enough] regardless of the existence of faster options. Also, QuickJS is not a "server-side JavaScript runtime". Not all programming is web development. > Given Fabrice Bellards other projects, I wouldn't be surprised if a JIT landed in QuickJS. Personally I would be pretty surprised considering that the aim of the project seems in part to be as small as possible.
      - seg_lol 6y ago
        I would like to deescalate this convo, I used overly harsh language and I apologize. Your use of fast, is in the definition fast enough for the task at hand. It is not fast in the absolute sense. I assert that Elsa being that much slower than Deno due to Deno having a JIT where as Elsa does not automatically puts them in different categories. But the stack that Elsa uses still needs Go and C, so long term for Elsa I could see it using a pure Go JS runtime in place of QuickJS.
- jiofih 6y ago
  It’s not. V8 is incredibly fast, sometimes within 50% of C perf. This is probably on par with Ruby and other script runtimes.

pastrami_panda 6y ago

That is a horrible way to present benchmarks, but it's quite a big difference in favor of Elsa. I was under the impression Deno was built on Rust and V8 which should bode for good performance, Elsa must be doing something very good?

hombre_fatal 6y ago
Yeah, I've put more work into explaining the results of a benchmark I put on pastebin.
coder543 6y ago
Each of the benchmarks seem to take a single-digit number of milliseconds in Elsa/QuickJS. My guess is that these benchmarks were chosen because the startup time for QuickJS is probably less than the startup time for V8, but I struggle to imagine that V8 is slower once the code has warmed up. Any meaningful benchmark with JIT runtimes should take several seconds, at a minimum, in my opinion. 8 milliseconds is not enough. From what I can tell, the benchmark is including the startup time, and it isn't keeping the interpreter warm -- it starts a fresh instance for each iteration of the benchmark. This is one of the benchmarks: https://github.com/elsaland/elsa/blob/master/testing/bench_scripts/fs.js https://github.com/elsaland/elsa/blob/master/testing/bench_s... Literally a single line of JavaScript.
- mkl 6y ago
  QuickJS's own benchmarks are here: https://bellard.org/quickjs/bench.html https://bellard.org/quickjs/bench.html V8, with or without its JIT, wins hands down. QuickJS may be small and fast to start, but fast actual computations seem not to be its strength (or intended target). Big discussion of QuickJS last year: https://news.ycombinator.com/item?id=20411154 https://news.ycombinator.com/item?id=20411154
- Waterluvian 6y ago
  It’s amazing watching V8s JIT warm up. And eye opening on how to measure performance when profiling. A heavy call can take 200ms, then 100, then 80, 70. And suddenly where I thought the low hanging fruit is has vanished.

untog 6y ago

I'm very dubious about whether these actually match real world performance. I've done a bunch of work with QuickJS. It's awesome and the performance features you get in return for its tiny footprint is amazing. But compared to JIT-enabled V8 it's incredibly slow. The one area where it shines compared to V8 is startup time, because the engine is so much lighter. So if you're profiling "time taken to start up and execute one operation" then yes, QuickJS might look better. But that isn't how people use Node/Deno, even when you're creating a CLI tool you're loading a lot of JS and performing a lot of operations.