7 ms·

> There's never any guarantee that JS that optimizes perfectly today won't hit a performance cliff in a new JIT implementation tomorrow. I would say the opposi

by derefr 5y ago

> There's never any guarantee that JS that optimizes perfectly today won't hit a performance cliff in a new JIT implementation tomorrow. I would say the opposite: there kind of are such guarantees, to JIT development’s detriment. Because JIT developers use the performance of known workloads under current state-of-the-art JITs as a regression-testing baseline. If their JIT underperforms the one it’s aiming to supersede on a known workload, their job isn’t done. This means that there’s little “slack” (https://slatestarcodex.com/2020/05/12/studies-on-slack/ https://slatestarcodex.com/2020/05/12/studies-on-slack/) in JIT development — it’ll be unlikely that we’ll see a JIT released (from a major team) that’s an order-of-magnitude better at certain tasks, at the expense of being less-than-an-order-of-magnitude worse at others. (Maybe from an individual as code ancillary to a research paper, but never commercialized/operationalized.)

pornel 5y ago
Browsers have replaced their optimizing backends entirely a few times already. They all optimize similar kinds of code, but the details of when and how change all the time. Vendors have deprecated and regressed on whole benchmark suites when they've decided the tests no longer represent what they want to optimize for. The biggest problem is that JIT is guided by heuristics. Observations from interpreters decide when code is advanced to higher optimization tiers, and that varies between implementations and changes relatively often. Your hottest code is also subject to JIT cache pressue and eviction policy, so you can't even rely on it staying hot across different machines using the exact same browser.
kannanvijayan 5y ago
I've spent the last decade working on and thinking about JITs, runtime hidden type modeling, and other optimization problems relating to reactive dynamic systems dealing with heavy polymorphism (specifically Javascript as a programming language). Now, different people in the industry will have different perspectives, so I'll preface this by saying that this is _my_ view on where the future of JIT compilation leads. The order of magnitude gains were mostly plumbed with the techniques JIT engineers have brought into play over the last decade or so. One aspect that remains relevant now is responsiveness to varying workloads, and nible reaction to polymorphic code, and finding the right balance between time spent compiling and the productivity of the optimized code that comes out the other end. There is significant work yet to be done in finding the right runtime type models that quickly and effectively distinguish between monomorphism, polymorphism, and megamorphism, and are able to respond appropriately. The other major avenue for development, and the one that I have yet to see a lot of talk about, is the potential to move many of these techniques and insights out of the domain of language runtime optimization, and into _libraries_, and allowing developers direct API-level access to the optimization strategies that have been developed. If you work on this stuff, you find very quickly that a huge amount of a JIT VM's backend infrastructure has nothing to with _compilation_ per se, and much more to do with the support structures that allow for the discovery of fastpaths for operations over "stable" data structures. In Javascript, the data structure that is of most relevance is a linked list of property-value-maps (javascript objects linked together by proto chains). We use well-known heuristics about the nature of those structures (e.g. "many objects will share the _keyset_ component of the hashtable", and "the linked list of hashtables will organize itself into a tree structure"). Using that information, we factor out common bits of the data-structure "behind the scenes" to optimize object representation and capture shared structural information in hidden types, and then apply techniques (such as inline caches) to optimize on the back of that shared structure. There's no reason that this sort of approach cannot be applied to _user specified structures_ of different sorts. Different "skeletal shapes" that are highly conserved in programs. For me, the big promise that these technologies have brought to the fore is the possibility of effectively doing partial specialization of data structures at runtime. Reorganizing data structures under the hood of an implementation to deliver, transparently to the client developer, optimization opportunities that we simply don't even consider as possible today.
- miohtama 5y ago
  Anything explicit JavaScript the language wise to help this? New private attributes? Something similar to Python __slots__. Basically why the guess structures when programmer could easily tell you the static bits and ask them to be frozen.
  - kannanvijayan 5y ago
    On the language side you'd need a way to tell the runtime which properties were relevant to extracting a backbone structure. Maybe a "backbone" field on the property descriptor - e.g.: `Object.defineProperty(obj, "prop", { writable: true, value: ..., backbone: true });` Then you'd need to generalize the hidden type modeler in the VM to interpret the backbone field and lift "prop" (and the value associated with it) into the hidden type representation. The important thing to remember here would be to ensure that these foldings of things into the hidden type are _transitive_. If the value of `obj.prop` itself has one or more "backbone" fields defined on it, then those too would need to be lifted into the hidden type. Assignments to these fields would need to generate a new hidden type for the underlying object, much as new shapes are created and installed for JS objects are mutated to add new properties or change the prototype. You'd then need to implement some mechanism of collapsing the object representations down to some inline format. Lastly, you'd need to teach the code-generator to use the hidden type to optimize accesses through "obj.prop" (or longer property access sequences). At the end of all of that, you then get a really beautiful optimization behaviour where: "obj.prop.anotherBackboneProp.someRegularProp", in suitably monomorphic or lightly-polymorphic locations, becomes not 3 separately-checked property accesses, but a single type-check on "obj", followed by a access into the top-level object. That would give a rudimentary ability for library and framework developers to define common shared, stable backbone structures for their own use. And that's fun to really think about. There's a bunch of work to be done, and it has to be motivated by real use cases, but I'm sure they are there. The idea that this powerful technique is only applicable to the _specific_ set of data structures involved in property lookup in object inheritance chains seems an unimaginative perspective.
heydenberk 5y ago
People are downvoting this comment but I think it's a valuable assertion, and the core fact (JS engine developers use current/real-world empirics to make optimization decisions) is indisputably true: https://v8.dev/blog/sparkplug https://v8.dev/blog/sparkplug
- astrange 5y ago
  Well, it's a bit silly to quote SSC (a blog that rewrites basic philosophy so STEMheads can think they invented it) for a definition of "slack".
  - Smaug123 5y ago
    I'll need someone to step in with a link to the actual post, because I can't think of the right search terms, but he has (of course) written about this in the past. His self-described schtick is to write about existing things in such a way that they reach a new audience, by dint of acting as a translator between some of the mental models that various people have. I'm not sure how much is gained by describing such an approach in the way that you do. If you'd simply omitted the bracketed clause, perhaps your comment might have been useful!
    - astrange 5y ago
      The mental model that causes you to need to get information from nerd essayists is probably not a good one though. You gotta try and be well read. Also, I thought it was bad when he said he was running the blog as a project to promote scientific racism (https://twitter.com/ToWit12/status/1362159170615537667 https://twitter.com/ToWit12/status/1362159170615537667).
  - refenestrator 5y ago
    The only things philosophers do are think and write, and somehow they are ass bad at writing. SSC wouldn't have a niche if they could get concepts across without a bunch of obscurantist bullshit.

by derefr 5y ago

pornel 5y ago

Browsers have replaced their optimizing backends entirely a few times already. They all optimize similar kinds of code, but the details of when and how change all the time. Vendors have deprecated and regressed on whole benchmark suites when they've decided the tests no longer represent what they want to optimize for. The biggest problem is that JIT is guided by heuristics. Observations from interpreters decide when code is advanced to higher optimization tiers, and that varies between implementations and changes relatively often. Your hottest code is also subject to JIT cache pressue and eviction policy, so you can't even rely on it staying hot across different machines using the exact same browser.

kannanvijayan 5y ago

I've spent the last decade working on and thinking about JITs, runtime hidden type modeling, and other optimization problems relating to reactive dynamic systems dealing with heavy polymorphism (specifically Javascript as a programming language). Now, different people in the industry will have different perspectives, so I'll preface this by saying that this is _my_ view on where the future of JIT compilation leads. The order of magnitude gains were mostly plumbed with the techniques JIT engineers have brought into play over the last decade or so. One aspect that remains relevant now is responsiveness to varying workloads, and nible reaction to polymorphic code, and finding the right balance between time spent compiling and the productivity of the optimized code that comes out the other end. There is significant work yet to be done in finding the right runtime type models that quickly and effectively distinguish between monomorphism, polymorphism, and megamorphism, and are able to respond appropriately. The other major avenue for development, and the one that I have yet to see a lot of talk about, is the potential to move many of these techniques and insights out of the domain of language runtime optimization, and into _libraries_, and allowing developers direct API-level access to the optimization strategies that have been developed. If you work on this stuff, you find very quickly that a huge amount of a JIT VM's backend infrastructure has nothing to with _compilation_ per se, and much more to do with the support structures that allow for the discovery of fastpaths for operations over "stable" data structures. In Javascript, the data structure that is of most relevance is a linked list of property-value-maps (javascript objects linked together by proto chains). We use well-known heuristics about the nature of those structures (e.g. "many objects will share the _keyset_ component of the hashtable", and "the linked list of hashtables will organize itself into a tree structure"). Using that information, we factor out common bits of the data-structure "behind the scenes" to optimize object representation and capture shared structural information in hidden types, and then apply techniques (such as inline caches) to optimize on the back of that shared structure. There's no reason that this sort of approach cannot be applied to _user specified structures_ of different sorts. Different "skeletal shapes" that are highly conserved in programs. For me, the big promise that these technologies have brought to the fore is the possibility of effectively doing partial specialization of data structures at runtime. Reorganizing data structures under the hood of an implementation to deliver, transparently to the client developer, optimization opportunities that we simply don't even consider as possible today.

miohtama 5y ago
Anything explicit JavaScript the language wise to help this? New private attributes? Something similar to Python __slots__. Basically why the guess structures when programmer could easily tell you the static bits and ask them to be frozen.
- kannanvijayan 5y ago
  On the language side you'd need a way to tell the runtime which properties were relevant to extracting a backbone structure. Maybe a "backbone" field on the property descriptor - e.g.: `Object.defineProperty(obj, "prop", { writable: true, value: ..., backbone: true });` Then you'd need to generalize the hidden type modeler in the VM to interpret the backbone field and lift "prop" (and the value associated with it) into the hidden type representation. The important thing to remember here would be to ensure that these foldings of things into the hidden type are _transitive_. If the value of `obj.prop` itself has one or more "backbone" fields defined on it, then those too would need to be lifted into the hidden type. Assignments to these fields would need to generate a new hidden type for the underlying object, much as new shapes are created and installed for JS objects are mutated to add new properties or change the prototype. You'd then need to implement some mechanism of collapsing the object representations down to some inline format. Lastly, you'd need to teach the code-generator to use the hidden type to optimize accesses through "obj.prop" (or longer property access sequences). At the end of all of that, you then get a really beautiful optimization behaviour where: "obj.prop.anotherBackboneProp.someRegularProp", in suitably monomorphic or lightly-polymorphic locations, becomes not 3 separately-checked property accesses, but a single type-check on "obj", followed by a access into the top-level object. That would give a rudimentary ability for library and framework developers to define common shared, stable backbone structures for their own use. And that's fun to really think about. There's a bunch of work to be done, and it has to be motivated by real use cases, but I'm sure they are there. The idea that this powerful technique is only applicable to the _specific_ set of data structures involved in property lookup in object inheritance chains seems an unimaginative perspective.

heydenberk 5y ago

People are downvoting this comment but I think it's a valuable assertion, and the core fact (JS engine developers use current/real-world empirics to make optimization decisions) is indisputably true: https://v8.dev/blog/sparkplug https://v8.dev/blog/sparkplug

astrange 5y ago
Well, it's a bit silly to quote SSC (a blog that rewrites basic philosophy so STEMheads can think they invented it) for a definition of "slack".
- Smaug123 5y ago
  I'll need someone to step in with a link to the actual post, because I can't think of the right search terms, but he has (of course) written about this in the past. His self-described schtick is to write about existing things in such a way that they reach a new audience, by dint of acting as a translator between some of the mental models that various people have. I'm not sure how much is gained by describing such an approach in the way that you do. If you'd simply omitted the bracketed clause, perhaps your comment might have been useful!
  - astrange 5y ago
    The mental model that causes you to need to get information from nerd essayists is probably not a good one though. You gotta try and be well read. Also, I thought it was bad when he said he was running the blog as a project to promote scientific racism (https://twitter.com/ToWit12/status/1362159170615537667 https://twitter.com/ToWit12/status/1362159170615537667).
- refenestrator 5y ago
  The only things philosophers do are think and write, and somehow they are ass bad at writing. SSC wouldn't have a niche if they could get concepts across without a bunch of obscurantist bullshit.