9 ms·

Performance of ES6 features relative to ES5

130 points by shawndumas 11y ago

white-flame 11y ago
The thing about JIT languages is that there's no such thing as overall speed of a particular operation; you can only sample current speed on current implementations. New features pursue correctness, then as their place in the JITs mature interesting new approaches will change their speed characteristics later. Things like contrasting the speed of various ways of iterating an array are all semantically equivalent, and as the JIT learns the various semantics they should all theoretically approach similar speeds, even if the newer ones are slower now.
- deleted 11y ago
  [deleted]
- amelius 11y ago
  > The thing about JIT languages is that there's no such thing as overall speed of a particular operation Except for the ASM.js subset of operations perhaps.
- paulddraper 11y ago
  > The thing about JIT languages is that there's no such thing as overall speed of a particular operation; you can only sample current speed on current implementations. How in the world is that specific to JIT? Have you never heard of a Sufficiently Smart Compiler?
  - acdha 11y ago
    The main difference is that you the developer can test with a known compiler and know what the performance characteristics are before you release your code: when Microsoft releases VisualStudio 2018, your existing application doesn't change until you rebuild it. In contrast, for code running in a web browser or something like the JVM, your existing code may run faster or slower without you even knowing about the new version.
    - dragonwriter 11y ago
      > The main difference is that you the developer can test with a known compiler and know what the performance characteristics are before you release your code: when Microsoft releases VisualStudio 2018, your existing application doesn't change until you rebuild it. In contrast, for code running in a web browser or something like the JVM, your existing code may run faster or slower without you even knowing about the new version. While the release of a new AOT compiler version might not affect the performance characteristics of your application without an active choice to recompile, the release of updates to any OS components your application interacts with, may, in much the same way that a new web browser or JVM version would. The only way you get immunity from that is to bundle the whole software stack running on the bare metal as your "app", rather than relying on lower-level software that may change independently.
      - acdha 11y ago
        Right – that's why I didn't say “known compiler and standard library”. That said, it's a difficult comparison because OS updates tend to be more conservative and are definitely less frequent than browser updates, AOT apps in lower-level languages like C/C++ are going to manage a higher percentage of their core data structures because they're pinning something like Boost whereas a JavaScript developer just gets whatever version of Object/Array/etc. is available, and there's not really a concept of a pinned release: if you install Windows apps, you'll pull in major versions of things like .NET or the JVM but there's no analogous practice of pinning a particular browser version. The closest we've come was Internet Explorer's compatibility modes which they've deprecated and which never changed the core JavaScript engine's performance – e.g. IE8-running-as-IE7 still applied the IE8 JavaScript engine's validation and optimizations.
    - nostrademons 11y ago
      I'm not sure this is true anymore even in this case - with multicore processors, cache dependency, and pervasive virtualization, the speed of your program can be significantly affected by what else is running on the box. Remember that the x86 itself has a JIT on the chip, converting x86 machine code to whatever microcode the processor uses. When I was at Google, they provided special dedicated machines for benchmarking, with custom run-times that disabled a lot of the containerization/virtualization features, and there was still a lot of noise in benchmark times.
- pjmlp 11y ago
  That applies to any language implementation. C compilers nowadays praised for speed used to generate worse code than junior Assembly developers on home micros.
david-given 11y ago
I would like to see Babel ES6-to-ES5 converted performance here too, because that's how most real-world ES6 code is going to be run --- there are too many non-ES6-compatible browsers out there.
- masklinn 11y ago
  That's exactly the information the "babel" rows provide? Each row is a given "ES6 implementation" and how it performs compared to the baseline "ES5 native" implementation of a feature. So the first row of each section is the babel-compiled (to ES5) ES6, the second is traceur-compiled, third is typescript-compiled and last is native ES6 runtime. If you look at the test files, most of them only have an es5 and an es6 versions.
  - david-given 11y ago
    Oh, FFS. I managed to spend some time looking at the table without actually seeing what the entire point of it was. Tea, you have failed me! Sorry about that.
- jsprogrammer 11y ago
  Isn't that what the 'babel' rows show?
jameslk 11y ago
For-of is slow (in Babel) because from my understanding it gets transpiled to use Regenerator. That was a bit of a nasty discovery for me.
- deleted 11y ago
  [deleted]
- eltaco 11y ago
  You can use loose mode or the loose option in that case.
atonse 11y ago
It's kind of awesome how good TypeScript looks in all this. Testament to the compiler talent MS has on staff.
venning 11y ago
Note, when it says "1.6x faster" what is really means is "1.6 as fast" or "60% faster", or really "operates at 1.6x the speed of the baseline", not "1.6x increase in speed". For example, when looking at the data for Chrome 48's "arrow" tests [1], the baseline number is 57,858,016 and the traceur number is 91,556,806, which is 158.2% of the baseline, but is reported as "1.6x faster". I know this seems like semantics, but "1.0x faster" sounds a lot like "twice as fast". (In this case, "1.0x faster" is reported as "identical" [2]). [1] https://github.com/kpdecker/six-speed/blob/master/data.json#L1933 https://github.com/kpdecker/six-speed/blob/master/data.json#... [2] https://github.com/kpdecker/six-speed/blob/master/tasks%2Freport.js#L171 https://github.com/kpdecker/six-speed/blob/master/tasks%2Fre...
- atonse 11y ago
  Yea in that example it might matter, but I'm seeing places where it's 10x or 100x slower. At that point, it hardly matters.
  - venning 11y ago
    Correct, at the extremes, the numbers don't matter. But where it says "1.2x slower", it really means "84% of baseline" [1], which might be a bit misleading. [1] https://github.com/kpdecker/six-speed/blob/master/data.json#L5681 https://github.com/kpdecker/six-speed/blob/master/data.json#...
- aschampion 11y ago
  This is the convention when talking about speedup in software. 1x speedup means identical time. [1] https://en.wikipedia.org/wiki/Speedup https://en.wikipedia.org/wiki/Speedup
  - baddox 11y ago
    I agree based on my experience reading these types of comparisons. I've never thought there was any ambiguity to saying "1.6 times faster." It means performance_1 = performance_0 * 1.6.
  - 11y ago
heydenberk 11y ago
These measures are JS performance are important for VM implementors, for people writing node.js applications to be used at large scale and people doing computationally-complex work in the browser (eg creating physics engines). It's worth noting that for most of us, most of the time, this kind of JS performance is less important than user-perceptible optimizations, like batching DOM reads and writes and decreasing asset size
stymaar 11y ago
The results given by this benchmark are bothering me because they do not fit with what I've seen in production. For example, the `for-of-array` is transpiled by babel to something that has 2 nested try/catch blocks, and unfortunately, V8 has a nasty deoptimisation when dealing with these, even if no exception is ever thrown. This deopt led to a several orders of magnitude slowdown compare to a simple `for` loop in Google Chrome, forcing use to abandon the `for of` construction. I suspect that the benchmark shown on this page does not expose this kind of behavior because it doesn't iterate enough for the JIT to kick in. If it's the case, it means that these values reflects only the behavior of the cold, interpreted code, and not the hot one. (which is quite sad for a performance benchmark, because the performance matter only for the former …)
- berdario 11y ago
  I'm not a frontend developer but... is it still truly necessary to transpile stuff like for-of ? It's supported by every desktop browser on the market, modulo some bugs (like inability to iterate over HTMLCollection and co. on Chrome)... as long as you only need to use it for arrays, things might work quite fine
  - rictic 11y ago
    According to caniuse, 12% of web users in the US are on IE 11, which doesn't support for-of.
  - pluma 11y ago
    Two words: Internet Explorer. Unless you're a startup and don't have to give a damn about what browsers your users use, IE marketshare is just too big to ignore. And depending on your audience, IE can mean anything from IE11 all the way back to IE8. Theoretically IE is irrelevant now that Microsoft has started publicly killing it off, but you'd be surprised at the number of corporate users that are still on Windows XP (or maybe Vista/7) running IE 8 and 9. Of course evergreen browsers are less of a problem but amazingly a lot of the same users who are stuck with IE also manage to have extremely outdated versions of Firefox.
- spankalee 11y ago
yoklov 11y ago
More than just these, even innocuous features such as `let` and `const` over `var` cause a substantial performance decrease in V8 (but not SpiderMonkey). I do a lot of one-off demos, that I mostly write in ES6 these days. I don't have a demonstration of pure `let` vs `var`, but if you change Babel to JavaScript in this one (and click run), you'll see a fairly substantial performance decrease in Chrome for this demo (Firefox stays the same though): https://jsfiddle.net/oa1sckzu/ https://jsfiddle.net/oa1sckzu/ . I have others, and the results are largely the same for them, but I'll spare you.
- rraval 11y ago
  There is a recent v8-users mailing list thread on this exact topic. Here's one possible explanation on perf differences between `let` and `var`: https://groups.google.com/d/msg/v8-users/hsUrt4I2D98/ELsfO1e6AQAJ https://groups.google.com/d/msg/v8-users/hsUrt4I2D98/ELsfO1e...
dagurp 11y ago
Why is "identical" in green and "faster" in a slightly darker green? Is es6 a library or is it the browser's implementation of ES6?
- mattashii 11y ago
  ES6 is the native implementation in the browser or JS-engine you are using, indeed. Green is probably chosen because the same performance is nice to have with this (subjectively) better syntax. The darker shade is probably chosen because more performance is nice, as it pushes you to the newer (subjectively better) syntax.
_greim_ 11y ago
I assumed the "es6" row header refers to the engine's native implementation of the feature. In which case it should grey out "destructuring" under Node 4.2.6. But they don't, it's listed as "10x slower". What am I missing?
vegabook 11y ago
The "commentariat" charge is legitimate, which is why there is a field known as "mathematics" which is unambiguous. 1.6x by anybody's non-commentariat definition is 60% faster, as anybody who has ever even scratched the surface of any mathematical, engineering, scientific, statistical or indeed, computer science discipline knows (though this is perhaps unknown to the Creative Suite crowd). This comment is completely bogus, with apologies if you don't understand. But you really should know what you don't know before posting such garbage.
- Dylan16807 11y ago
  Don't be so smug. Also you're wrong. "__ faster" implies addition in many/most cases. Since you know what 1.6x is by itself, I'll let you use your math knowledge to calculate x + 1.6x.
- dang 11y ago
  Please don't post uncivil comments to Hacker News. We detached this subthread from https://news.ycombinator.com/item?id=11204967 https://news.ycombinator.com/item?id=11204967 and marked it off-topic.
snorrah 11y ago
This makes me think of Python 2 vs 3 benchmarks for some reason, although I'm sure I'm not remembering very well. I assume there's plenty of scope for ES6 to improve and (hopefully?) surpass ES5 in performance in many, if not all, areas? What would be causing the large slowdowns in ES6 here? Is this a case of having features that make JS nicer to code in, but giving up some performance? Or is this more down to immature compilers not yet optimising ES6 as substantially?
cromwellian 11y ago
I'd like to see Closure Compiler thrown into the mix. It not only translates ES6 to ES5, but applies optimizations.
alxlu 11y ago
Why is arrow-declare is so much slower on Firefox (16x to 325x slower) compared to everything else (identical to 2.4x slower)?
balupton 11y ago
There is also https://github.com/bevry/esnext-benchmarks https://github.com/bevry/esnext-benchmarks

130 points by shawndumas 11y ago

The thing about JIT languages is that there's no such thing as overall speed of a particular operation; you can only sample current speed on current implementations. New features pursue correctness, then as their place in the JITs mature interesting new approaches will change their speed characteristics later. Things like contrasting the speed of various ways of iterating an array are all semantically equivalent, and as the JIT learns the various semantics they should all theoretically approach similar speeds, even if the newer ones are slower now.

deleted 11y ago
[deleted]
amelius 11y ago
> The thing about JIT languages is that there's no such thing as overall speed of a particular operation Except for the ASM.js subset of operations perhaps.
paulddraper 11y ago
> The thing about JIT languages is that there's no such thing as overall speed of a particular operation; you can only sample current speed on current implementations. How in the world is that specific to JIT? Have you never heard of a Sufficiently Smart Compiler?
- acdha 11y ago
  The main difference is that you the developer can test with a known compiler and know what the performance characteristics are before you release your code: when Microsoft releases VisualStudio 2018, your existing application doesn't change until you rebuild it. In contrast, for code running in a web browser or something like the JVM, your existing code may run faster or slower without you even knowing about the new version.
  - dragonwriter 11y ago
    > The main difference is that you the developer can test with a known compiler and know what the performance characteristics are before you release your code: when Microsoft releases VisualStudio 2018, your existing application doesn't change until you rebuild it. In contrast, for code running in a web browser or something like the JVM, your existing code may run faster or slower without you even knowing about the new version. While the release of a new AOT compiler version might not affect the performance characteristics of your application without an active choice to recompile, the release of updates to any OS components your application interacts with, may, in much the same way that a new web browser or JVM version would. The only way you get immunity from that is to bundle the whole software stack running on the bare metal as your "app", rather than relying on lower-level software that may change independently.
    - acdha 11y ago
      Right – that's why I didn't say “known compiler and standard library”. That said, it's a difficult comparison because OS updates tend to be more conservative and are definitely less frequent than browser updates, AOT apps in lower-level languages like C/C++ are going to manage a higher percentage of their core data structures because they're pinning something like Boost whereas a JavaScript developer just gets whatever version of Object/Array/etc. is available, and there's not really a concept of a pinned release: if you install Windows apps, you'll pull in major versions of things like .NET or the JVM but there's no analogous practice of pinning a particular browser version. The closest we've come was Internet Explorer's compatibility modes which they've deprecated and which never changed the core JavaScript engine's performance – e.g. IE8-running-as-IE7 still applied the IE8 JavaScript engine's validation and optimizations.
  - nostrademons 11y ago
    I'm not sure this is true anymore even in this case - with multicore processors, cache dependency, and pervasive virtualization, the speed of your program can be significantly affected by what else is running on the box. Remember that the x86 itself has a JIT on the chip, converting x86 machine code to whatever microcode the processor uses. When I was at Google, they provided special dedicated machines for benchmarking, with custom run-times that disabled a lot of the containerization/virtualization features, and there was still a lot of noise in benchmark times.
pjmlp 11y ago
That applies to any language implementation. C compilers nowadays praised for speed used to generate worse code than junior Assembly developers on home micros.

david-given 11y ago

I would like to see Babel ES6-to-ES5 converted performance here too, because that's how most real-world ES6 code is going to be run --- there are too many non-ES6-compatible browsers out there.

masklinn 11y ago
That's exactly the information the "babel" rows provide? Each row is a given "ES6 implementation" and how it performs compared to the baseline "ES5 native" implementation of a feature. So the first row of each section is the babel-compiled (to ES5) ES6, the second is traceur-compiled, third is typescript-compiled and last is native ES6 runtime. If you look at the test files, most of them only have an es5 and an es6 versions.
- david-given 11y ago
  Oh, FFS. I managed to spend some time looking at the table without actually seeing what the entire point of it was. Tea, you have failed me! Sorry about that.
jsprogrammer 11y ago
Isn't that what the 'babel' rows show?

jameslk 11y ago

For-of is slow (in Babel) because from my understanding it gets transpiled to use Regenerator. That was a bit of a nasty discovery for me.

deleted 11y ago
[deleted]
eltaco 11y ago
You can use loose mode or the loose option in that case.

atonse 11y ago

It's kind of awesome how good TypeScript looks in all this. Testament to the compiler talent MS has on staff.

venning 11y ago

Note, when it says "1.6x faster" what is really means is "1.6 as fast" or "60% faster", or really "operates at 1.6x the speed of the baseline", not "1.6x increase in speed". For example, when looking at the data for Chrome 48's "arrow" tests [1], the baseline number is 57,858,016 and the traceur number is 91,556,806, which is 158.2% of the baseline, but is reported as "1.6x faster". I know this seems like semantics, but "1.0x faster" sounds a lot like "twice as fast". (In this case, "1.0x faster" is reported as "identical" [2]). [1] https://github.com/kpdecker/six-speed/blob/master/data.json#L1933 https://github.com/kpdecker/six-speed/blob/master/data.json#... [2] https://github.com/kpdecker/six-speed/blob/master/tasks%2Freport.js#L171 https://github.com/kpdecker/six-speed/blob/master/tasks%2Fre...

atonse 11y ago
Yea in that example it might matter, but I'm seeing places where it's 10x or 100x slower. At that point, it hardly matters.
- venning 11y ago
  Correct, at the extremes, the numbers don't matter. But where it says "1.2x slower", it really means "84% of baseline" [1], which might be a bit misleading. [1] https://github.com/kpdecker/six-speed/blob/master/data.json#L5681 https://github.com/kpdecker/six-speed/blob/master/data.json#...
aschampion 11y ago
This is the convention when talking about speedup in software. 1x speedup means identical time. [1] https://en.wikipedia.org/wiki/Speedup https://en.wikipedia.org/wiki/Speedup
- baddox 11y ago
  I agree based on my experience reading these types of comparisons. I've never thought there was any ambiguity to saying "1.6 times faster." It means performance_1 = performance_0 * 1.6.
- 11y ago

heydenberk 11y ago

These measures are JS performance are important for VM implementors, for people writing node.js applications to be used at large scale and people doing computationally-complex work in the browser (eg creating physics engines). It's worth noting that for most of us, most of the time, this kind of JS performance is less important than user-perceptible optimizations, like batching DOM reads and writes and decreasing asset size

stymaar 11y ago

The results given by this benchmark are bothering me because they do not fit with what I've seen in production. For example, the `for-of-array` is transpiled by babel to something that has 2 nested try/catch blocks, and unfortunately, V8 has a nasty deoptimisation when dealing with these, even if no exception is ever thrown. This deopt led to a several orders of magnitude slowdown compare to a simple `for` loop in Google Chrome, forcing use to abandon the `for of` construction. I suspect that the benchmark shown on this page does not expose this kind of behavior because it doesn't iterate enough for the JIT to kick in. If it's the case, it means that these values reflects only the behavior of the cold, interpreted code, and not the hot one. (which is quite sad for a performance benchmark, because the performance matter only for the former …)

berdario 11y ago
I'm not a frontend developer but... is it still truly necessary to transpile stuff like for-of ? It's supported by every desktop browser on the market, modulo some bugs (like inability to iterate over HTMLCollection and co. on Chrome)... as long as you only need to use it for arrays, things might work quite fine
- rictic 11y ago
  According to caniuse, 12% of web users in the US are on IE 11, which doesn't support for-of.
- pluma 11y ago
  Two words: Internet Explorer. Unless you're a startup and don't have to give a damn about what browsers your users use, IE marketshare is just too big to ignore. And depending on your audience, IE can mean anything from IE11 all the way back to IE8. Theoretically IE is irrelevant now that Microsoft has started publicly killing it off, but you'd be surprised at the number of corporate users that are still on Windows XP (or maybe Vista/7) running IE 8 and 9. Of course evergreen browsers are less of a problem but amazingly a lot of the same users who are stuck with IE also manage to have extremely outdated versions of Firefox.
spankalee 11y ago

yoklov 11y ago

More than just these, even innocuous features such as `let` and `const` over `var` cause a substantial performance decrease in V8 (but not SpiderMonkey). I do a lot of one-off demos, that I mostly write in ES6 these days. I don't have a demonstration of pure `let` vs `var`, but if you change Babel to JavaScript in this one (and click run), you'll see a fairly substantial performance decrease in Chrome for this demo (Firefox stays the same though): https://jsfiddle.net/oa1sckzu/ https://jsfiddle.net/oa1sckzu/ . I have others, and the results are largely the same for them, but I'll spare you.

rraval 11y ago
There is a recent v8-users mailing list thread on this exact topic. Here's one possible explanation on perf differences between `let` and `var`: https://groups.google.com/d/msg/v8-users/hsUrt4I2D98/ELsfO1e6AQAJ https://groups.google.com/d/msg/v8-users/hsUrt4I2D98/ELsfO1e...

dagurp 11y ago

Why is "identical" in green and "faster" in a slightly darker green? Is es6 a library or is it the browser's implementation of ES6?

mattashii 11y ago
ES6 is the native implementation in the browser or JS-engine you are using, indeed. Green is probably chosen because the same performance is nice to have with this (subjectively) better syntax. The darker shade is probably chosen because more performance is nice, as it pushes you to the newer (subjectively better) syntax.

_greim_ 11y ago

I assumed the "es6" row header refers to the engine's native implementation of the feature. In which case it should grey out "destructuring" under Node 4.2.6. But they don't, it's listed as "10x slower". What am I missing?

vegabook 11y ago

The "commentariat" charge is legitimate, which is why there is a field known as "mathematics" which is unambiguous. 1.6x by anybody's non-commentariat definition is 60% faster, as anybody who has ever even scratched the surface of any mathematical, engineering, scientific, statistical or indeed, computer science discipline knows (though this is perhaps unknown to the Creative Suite crowd). This comment is completely bogus, with apologies if you don't understand. But you really should know what you don't know before posting such garbage.

Dylan16807 11y ago
Don't be so smug. Also you're wrong. "__ faster" implies addition in many/most cases. Since you know what 1.6x is by itself, I'll let you use your math knowledge to calculate x + 1.6x.
dang 11y ago
Please don't post uncivil comments to Hacker News. We detached this subthread from https://news.ycombinator.com/item?id=11204967 https://news.ycombinator.com/item?id=11204967 and marked it off-topic.

snorrah 11y ago

This makes me think of Python 2 vs 3 benchmarks for some reason, although I'm sure I'm not remembering very well. I assume there's plenty of scope for ES6 to improve and (hopefully?) surpass ES5 in performance in many, if not all, areas? What would be causing the large slowdowns in ES6 here? Is this a case of having features that make JS nicer to code in, but giving up some performance? Or is this more down to immature compilers not yet optimising ES6 as substantially?

cromwellian 11y ago

I'd like to see Closure Compiler thrown into the mix. It not only translates ES6 to ES5, but applies optimizations.

alxlu 11y ago

Why is arrow-declare is so much slower on Firefox (16x to 325x slower) compared to everything else (identical to 2.4x slower)?

balupton 11y ago

There is also https://github.com/bevry/esnext-benchmarks https://github.com/bevry/esnext-benchmarks