7 ms·

There was a back-and-forth on optimizing this (see the link Steve found). The WASM version got faster too by adopting algorithmic changes proposed here, and the

by pornel 5y ago

There was a back-and-forth on optimizing this (see the link Steve found). The WASM version got faster too by adopting algorithmic changes proposed here, and the conclusion of Rust authors was that you do need WASM if you want predictable performance. High-perf JS relies on staying on the JIT happy path, not upsetting GC, and all of this is informal trickery that's not easy to do for all JS engines. There's never any guarantee that JS that optimizes perfectly today won't hit a performance cliff in a new JIT implementation tomorrow.

steveklabnik 5y ago
https://fitzgeraldnick.com/2018/02/26/speed-without-wizardry.html https://fitzgeraldnick.com/2018/02/26/speed-without-wizardry...
- eevilspock 5y ago
  I know we are not supposed to talk about down voting but in this case I would really like to know why? The article Steve links to is great and even the GP (top comment) approves. Yet Steve's comment is now dimmed. I thought HN was supposed to be better.
  - edflsafoiewq 5y ago
    It isn't dimmed. It contains only a link and visited links are styled to be lighter gray than normal text. You can check this with the inspector: normal comments have the class c00 (color: #000000); dimmed comments have classes like c5a (color: #5a5a5a), c73, etc.
    - eevilspock 5y ago
      ok bad. too late to delete.
      - steveklabnik 5y ago
        Have an upvote from me; sorry that my lack of anything but a link contributed to the confusion here!
        eevilspock 5y ago
        thanks Steve!
derefr 5y ago
> There's never any guarantee that JS that optimizes perfectly today won't hit a performance cliff in a new JIT implementation tomorrow. I would say the opposite: there kind of are such guarantees, to JIT development’s detriment. Because JIT developers use the performance of known workloads under current state-of-the-art JITs as a regression-testing baseline. If their JIT underperforms the one it’s aiming to supersede on a known workload, their job isn’t done. This means that there’s little “slack” (https://slatestarcodex.com/2020/05/12/studies-on-slack/ https://slatestarcodex.com/2020/05/12/studies-on-slack/) in JIT development — it’ll be unlikely that we’ll see a JIT released (from a major team) that’s an order-of-magnitude better at certain tasks, at the expense of being less-than-an-order-of-magnitude worse at others. (Maybe from an individual as code ancillary to a research paper, but never commercialized/operationalized.)
- pornel 5y ago
  Browsers have replaced their optimizing backends entirely a few times already. They all optimize similar kinds of code, but the details of when and how change all the time. Vendors have deprecated and regressed on whole benchmark suites when they've decided the tests no longer represent what they want to optimize for. The biggest problem is that JIT is guided by heuristics. Observations from interpreters decide when code is advanced to higher optimization tiers, and that varies between implementations and changes relatively often. Your hottest code is also subject to JIT cache pressue and eviction policy, so you can't even rely on it staying hot across different machines using the exact same browser.
- kannanvijayan 5y ago
  I've spent the last decade working on and thinking about JITs, runtime hidden type modeling, and other optimization problems relating to reactive dynamic systems dealing with heavy polymorphism (specifically Javascript as a programming language). Now, different people in the industry will have different perspectives, so I'll preface this by saying that this is _my_ view on where the future of JIT compilation leads. The order of magnitude gains were mostly plumbed with the techniques JIT engineers have brought into play over the last decade or so. One aspect that remains relevant now is responsiveness to varying workloads, and nible reaction to polymorphic code, and finding the right balance between time spent compiling and the productivity of the optimized code that comes out the other end. There is significant work yet to be done in finding the right runtime type models that quickly and effectively distinguish between monomorphism, polymorphism, and megamorphism, and are able to respond appropriately. The other major avenue for development, and the one that I have yet to see a lot of talk about, is the potential to move many of these techniques and insights out of the domain of language runtime optimization, and into _libraries_, and allowing developers direct API-level access to the optimization strategies that have been developed. If you work on this stuff, you find very quickly that a huge amount of a JIT VM's backend infrastructure has nothing to with _compilation_ per se, and much more to do with the support structures that allow for the discovery of fastpaths for operations over "stable" data structures. In Javascript, the data structure that is of most relevance is a linked list of property-value-maps (javascript objects linked together by proto chains). We use well-known heuristics about the nature of those structures (e.g. "many objects will share the _keyset_ component of the hashtable", and "the linked list of hashtables will organize itself into a tree structure"). Using that information, we factor out common bits of the data-structure "behind the scenes" to optimize object representation and capture shared structural information in hidden types, and then apply techniques (such as inline caches) to optimize on the back of that shared structure. There's no reason that this sort of approach cannot be applied to _user specified structures_ of different sorts. Different "skeletal shapes" that are highly conserved in programs. For me, the big promise that these technologies have brought to the fore is the possibility of effectively doing partial specialization of data structures at runtime. Reorganizing data structures under the hood of an implementation to deliver, transparently to the client developer, optimization opportunities that we simply don't even consider as possible today.
lhorie 5y ago
> There's never any guarantee that JS that optimizes perfectly today won't hit a performance cliff in a new JIT implementation tomorrow. Yep. As someone who played this game a few years ago, JIT implementations do change (e.g. function inlining based on function size was removed from V8, delete performance changed, number of allowed slots in hidden classes changed, etc). Also worth noting that just because an optimization works in V8, there's no guarantee that it will also work in another JS engine.
- masklinn 5y ago
  > Also worth noting that just because an optimization works in V8, there's no guarantee that it will also work in another JS engine. The essay even covers that, when it mentions that matching arities had a significant impact on V8 (14% improvement) but not perceivable effect on SpiderMonkey.
- Hypx_ 5y ago
  Though I wonder... Some decades ago people made similar arguments with regards to C/C++ versus native assembly. That the only way compiled languages could approach assembly languages on performance was if the code was written in a way that the compiler could optimize. If it couldn't, performance would often tank. But as compilers got better this become less and less of an issue. By today it has become very rare to need to write in assembly. So shouldn't history repeat itself? That JIT compilers will be so good that it becomes extremely unlikely that you'll hit the performance cliff.
  - lhorie 5y ago
    I don't think so. Compilers have gotten smart, but so have people. See for example stuff like this[0]. Hot path stuff still sees hand-written assembly even in this day and age because even modern compilers aren't able to provide enough overhead elision guarantees required for the really hardcore optimizations. What really happened is that thanks to the advances in computing power over the decades, we can now afford to skimp on performance most of the time - putting effort in optimizing only the most critical code paths - and still be able to get away with it. But when we're talking about squeezing every drop of performance, one of the big challenges with fighting against the JIT engine is figuring out when things deoptimize. This is because there are patterns (e.g. polymorphic code paths) that are inherently impossible to optimize perfectly (because halting problem). As soon as the runtime runs into a "weird" data type, deoptimization is forced to kick in and that can destroy performance. As a developer you have to be extremely mindful of this. Type systems like Typescript help, but you can still be effectively polymorphic (e.g. large union types, `any`, etc) if you're not being mindful of the underlying machine code. And JIT has one major downside that these benchmarks often intentionally omit in the name of "reducing noise": they usually take a while to warm up to native-like speed. I'm talking in the order of thousands of iterations running at a couple of order of magnitudes slower before JIT speeds fully kicks in. [0] https://github.com/jart/cosmopolitan/blob/master/libc/nexgen32e/memcpy.S https://github.com/jart/cosmopolitan/blob/master/libc/nexgen...
TazeTSchnitzel 5y ago
The dancing required to stay within the happy paths, and the necessity of avoiding certain common abstractions in order to do so, leaves me convinced it must be much easier to write performant high-level code in C, C++, Rust etc than in something like JavaScript or Python. While it's possible to write fast code in the latter, it doesn't seem like much fun.
pjmlp 5y ago
Browser WASM implementations are also JIT based, and GC will eventually be part of the picture for WASM runtimes as well.

by pornel 5y ago

steveklabnik 5y ago

https://fitzgeraldnick.com/2018/02/26/speed-without-wizardry.html https://fitzgeraldnick.com/2018/02/26/speed-without-wizardry...

eevilspock 5y ago
I know we are not supposed to talk about down voting but in this case I would really like to know why? The article Steve links to is great and even the GP (top comment) approves. Yet Steve's comment is now dimmed. I thought HN was supposed to be better.
- edflsafoiewq 5y ago
  It isn't dimmed. It contains only a link and visited links are styled to be lighter gray than normal text. You can check this with the inspector: normal comments have the class c00 (color: #000000); dimmed comments have classes like c5a (color: #5a5a5a), c73, etc.
  - eevilspock 5y ago
    ok bad. too late to delete.
    - steveklabnik 5y ago
      Have an upvote from me; sorry that my lack of anything but a link contributed to the confusion here!
      - eevilspock 5y ago
        thanks Steve!

derefr 5y ago

> There's never any guarantee that JS that optimizes perfectly today won't hit a performance cliff in a new JIT implementation tomorrow. I would say the opposite: there kind of are such guarantees, to JIT development’s detriment. Because JIT developers use the performance of known workloads under current state-of-the-art JITs as a regression-testing baseline. If their JIT underperforms the one it’s aiming to supersede on a known workload, their job isn’t done. This means that there’s little “slack” (https://slatestarcodex.com/2020/05/12/studies-on-slack/ https://slatestarcodex.com/2020/05/12/studies-on-slack/) in JIT development — it’ll be unlikely that we’ll see a JIT released (from a major team) that’s an order-of-magnitude better at certain tasks, at the expense of being less-than-an-order-of-magnitude worse at others. (Maybe from an individual as code ancillary to a research paper, but never commercialized/operationalized.)

pornel 5y ago
Browsers have replaced their optimizing backends entirely a few times already. They all optimize similar kinds of code, but the details of when and how change all the time. Vendors have deprecated and regressed on whole benchmark suites when they've decided the tests no longer represent what they want to optimize for. The biggest problem is that JIT is guided by heuristics. Observations from interpreters decide when code is advanced to higher optimization tiers, and that varies between implementations and changes relatively often. Your hottest code is also subject to JIT cache pressue and eviction policy, so you can't even rely on it staying hot across different machines using the exact same browser.
kannanvijayan 5y ago
I've spent the last decade working on and thinking about JITs, runtime hidden type modeling, and other optimization problems relating to reactive dynamic systems dealing with heavy polymorphism (specifically Javascript as a programming language). Now, different people in the industry will have different perspectives, so I'll preface this by saying that this is _my_ view on where the future of JIT compilation leads. The order of magnitude gains were mostly plumbed with the techniques JIT engineers have brought into play over the last decade or so. One aspect that remains relevant now is responsiveness to varying workloads, and nible reaction to polymorphic code, and finding the right balance between time spent compiling and the productivity of the optimized code that comes out the other end. There is significant work yet to be done in finding the right runtime type models that quickly and effectively distinguish between monomorphism, polymorphism, and megamorphism, and are able to respond appropriately. The other major avenue for development, and the one that I have yet to see a lot of talk about, is the potential to move many of these techniques and insights out of the domain of language runtime optimization, and into _libraries_, and allowing developers direct API-level access to the optimization strategies that have been developed. If you work on this stuff, you find very quickly that a huge amount of a JIT VM's backend infrastructure has nothing to with _compilation_ per se, and much more to do with the support structures that allow for the discovery of fastpaths for operations over "stable" data structures. In Javascript, the data structure that is of most relevance is a linked list of property-value-maps (javascript objects linked together by proto chains). We use well-known heuristics about the nature of those structures (e.g. "many objects will share the _keyset_ component of the hashtable", and "the linked list of hashtables will organize itself into a tree structure"). Using that information, we factor out common bits of the data-structure "behind the scenes" to optimize object representation and capture shared structural information in hidden types, and then apply techniques (such as inline caches) to optimize on the back of that shared structure. There's no reason that this sort of approach cannot be applied to _user specified structures_ of different sorts. Different "skeletal shapes" that are highly conserved in programs. For me, the big promise that these technologies have brought to the fore is the possibility of effectively doing partial specialization of data structures at runtime. Reorganizing data structures under the hood of an implementation to deliver, transparently to the client developer, optimization opportunities that we simply don't even consider as possible today.

lhorie 5y ago

> There's never any guarantee that JS that optimizes perfectly today won't hit a performance cliff in a new JIT implementation tomorrow. Yep. As someone who played this game a few years ago, JIT implementations do change (e.g. function inlining based on function size was removed from V8, delete performance changed, number of allowed slots in hidden classes changed, etc). Also worth noting that just because an optimization works in V8, there's no guarantee that it will also work in another JS engine.

masklinn 5y ago
> Also worth noting that just because an optimization works in V8, there's no guarantee that it will also work in another JS engine. The essay even covers that, when it mentions that matching arities had a significant impact on V8 (14% improvement) but not perceivable effect on SpiderMonkey.
Hypx_ 5y ago
Though I wonder... Some decades ago people made similar arguments with regards to C/C++ versus native assembly. That the only way compiled languages could approach assembly languages on performance was if the code was written in a way that the compiler could optimize. If it couldn't, performance would often tank. But as compilers got better this become less and less of an issue. By today it has become very rare to need to write in assembly. So shouldn't history repeat itself? That JIT compilers will be so good that it becomes extremely unlikely that you'll hit the performance cliff.
- lhorie 5y ago
  I don't think so. Compilers have gotten smart, but so have people. See for example stuff like this[0]. Hot path stuff still sees hand-written assembly even in this day and age because even modern compilers aren't able to provide enough overhead elision guarantees required for the really hardcore optimizations. What really happened is that thanks to the advances in computing power over the decades, we can now afford to skimp on performance most of the time - putting effort in optimizing only the most critical code paths - and still be able to get away with it. But when we're talking about squeezing every drop of performance, one of the big challenges with fighting against the JIT engine is figuring out when things deoptimize. This is because there are patterns (e.g. polymorphic code paths) that are inherently impossible to optimize perfectly (because halting problem). As soon as the runtime runs into a "weird" data type, deoptimization is forced to kick in and that can destroy performance. As a developer you have to be extremely mindful of this. Type systems like Typescript help, but you can still be effectively polymorphic (e.g. large union types, `any`, etc) if you're not being mindful of the underlying machine code. And JIT has one major downside that these benchmarks often intentionally omit in the name of "reducing noise": they usually take a while to warm up to native-like speed. I'm talking in the order of thousands of iterations running at a couple of order of magnitudes slower before JIT speeds fully kicks in. [0] https://github.com/jart/cosmopolitan/blob/master/libc/nexgen32e/memcpy.S https://github.com/jart/cosmopolitan/blob/master/libc/nexgen...

TazeTSchnitzel 5y ago

The dancing required to stay within the happy paths, and the necessity of avoiding certain common abstractions in order to do so, leaves me convinced it must be much easier to write performant high-level code in C, C++, Rust etc than in something like JavaScript or Python. While it's possible to write fast code in the latter, it doesn't seem like much fun.

pjmlp 5y ago

Browser WASM implementations are also JIT based, and GC will eventually be part of the picture for WASM runtimes as well.