5 ms·

It was particularly poignant at the time because JITed languages were looked down on by the “static compilation makes us faster” crowd. So it was a sort of “wai

It was particularly poignant at the time because JITed languages were looked down on by the “static compilation makes us faster” crowd. So it was a sort of “wait a minute Watson!” moment in that particular tech debate. No one cares as much now days, we’ve moved our overrated opinion battlegrounds to other portions of what we do.

kllrnohj 4y ago
I think you're over estimating the impact or relevance of that anecdote. Particularly since the "static compilation makes us faster" crowd turned out to be correct, and people use JITs for non-performance reasons and just pay the performance tax they so often come with. The time-constrained nature in which a JIT has to run just largely kills it's theoretical runtime information gathering advantages. Devirtualization remains about the most advanced trick in the JIT book, which is generally not an issue static compiled languages struggle with in the first place.
- pjmlp 4y ago
  Sure, because everyone ships static linked binaries.
pjmlp 4y ago
I eventually changed my opinion into JIT being the only way to make dynamic languages faster, while strong typed ones can benefit from having both AOT/JIT for different kinds of deployment scenarios, and development workflows.
- mikepurvis 4y ago
  I think I landed in a place where it's basically "the compiler has insufficient information to achieve ideal optimization because some things can only be known at runtime." Which is not exclusively an argument for runtime JIT— it can also be an argument for instrumenting your runtime environment, and feeding that profiling data back to the compiler to help it make smarter decisions the next time. But that's definitely a more involved process than just baking it into the same JavaScript interpreter used by everyone— likely well worth it in the case of things like game engines, though.
  - anyfoo 4y ago
    That's a lesson Intel had to (re-?)learn with Itanium as well.
  - inkyoto 4y ago
    > […] "the compiler has insufficient information to achieve ideal optimization because some things can only be known at runtime." This is where the profile guided optimisation comes in – for statically compiled languages, with a caveat being not always straightforward to come up with a set of inputs that will trigger an execution of all possible code paths. One solution is to provide the coverage specifically for the performance critical code paths and let the rest just be.
  - funstuff007 4y ago
    > it can also be an argument for instrumenting your runtime environment Aren't JITs already self-instrumenting? What would you instrument that the JIT is not already keeping track of?
  - masklinn 4y ago
    It's also an argument for having much more expressive and precise type systems, so the compiler has better information. Once you've managed to debug the codegen anyway (see: The Long and Arduous Story of Noalias).
    - mikepurvis 4y ago
      Is it? I'd love to see a breakdown of what classes of information can be gleaned from profile data, and how much of an impact each one has in isolation in terms of optimization. Naively, I would have assumed that branch information would be most valuable, in terms of being able to guide execution toward the hot path and maximize locality for the memory accesses occurring on the common branches. And that info is not something that would be assisted by more expressive types, I don't think.
  - notriddle 4y ago
    https://tomaszs2.medium.com/how-rust-1-64-became-10-20-faster-on-windows-3a8bb5e81d70 https://tomaszs2.medium.com/how-rust-1-64-became-10-20-faste... https://news.ycombinator.com/item?id=33306945 https://news.ycombinator.com/item?id=33306945
- titzer 4y ago
  Dynamic languages need inline caches, type feedback, and fairly heavy inlining to be competitive. Some of that can be gotten offline, e.g. by doing PGO. But you can't, in general, adapt to a program that suddenly changes phases, or rebinds a global that was assumed a constant, etc. Speculative optimizations with deopt are what make dynamic languages fast.
- hinkley 4y ago
  Before I talked myself out of writing my own programming language, I used to have lunch conversations with my mentor who was also speed obsessed about how JIT could meet Knuth in the middle by creating a collections API with feedback guided optimization, using it for algorithm selection and tuning parameters by call site. For object graphs in Java you can waste exorbitant amounts of memory by having a lot of “children” members that are sized for a default of 10 entries but the normal case is 0-2. I once had to deoptimize code where someone tried to do this by hand and the number they picked was 6 (just over half of the default). So when the average jumped to 7, then the data structure ended up being 20% larger than the default behavior instead of 30% smaller as intended. For a server workflow, having data structured tuned to larger pools of objects with more complex comparison operations can also be valuable, but I don’t want that kitchen sink stuff on mobile or in an embedded app. I still think this is viable, but only if you are clever about gathering data. For instance the incremental increase in runtime for telemetry data is quite high on the happy path. But corner cases are already expensive, so telemetry adds only a few percent there instead of double digits. The nonstarter for this ended up being that most collections APIs violate Liskov, so you almost need to write your own language to pick a decomposition that doesn’t. Variance semantics help a ton but they don’t quite fix LSP.
  - astrange 4y ago
    Swift and Objective-C have collections that change implementation as needed, but they can do it because they're well abstracted (enough) and usually immutable so there's less chance of making a bad assumption. Most other languages only have mutable collections, and name collection types after their implementation details instead of what the user actually wants from them.
  - pjmlp 4y ago
    That is in a sense what JIT caching with PGO feedback loop does to a certain extent. As for mobile, this isn't far off from the whole set of hand written Assembly interpreter, JIT with PGO data caching, AOT compilation on idle with the PGO collected data, sharing PGO metadata with other devices via the PlayStore, that Android happens to do since version 7, and refined later.
saagarjha 4y ago
I take it you are not very familiar with the website known as Hacker News.

kllrnohj 4y ago

I think you're over estimating the impact or relevance of that anecdote. Particularly since the "static compilation makes us faster" crowd turned out to be correct, and people use JITs for non-performance reasons and just pay the performance tax they so often come with. The time-constrained nature in which a JIT has to run just largely kills it's theoretical runtime information gathering advantages. Devirtualization remains about the most advanced trick in the JIT book, which is generally not an issue static compiled languages struggle with in the first place.

pjmlp 4y ago
Sure, because everyone ships static linked binaries.

pjmlp 4y ago

I eventually changed my opinion into JIT being the only way to make dynamic languages faster, while strong typed ones can benefit from having both AOT/JIT for different kinds of deployment scenarios, and development workflows.

mikepurvis 4y ago
I think I landed in a place where it's basically "the compiler has insufficient information to achieve ideal optimization because some things can only be known at runtime." Which is not exclusively an argument for runtime JIT— it can also be an argument for instrumenting your runtime environment, and feeding that profiling data back to the compiler to help it make smarter decisions the next time. But that's definitely a more involved process than just baking it into the same JavaScript interpreter used by everyone— likely well worth it in the case of things like game engines, though.
- anyfoo 4y ago
  That's a lesson Intel had to (re-?)learn with Itanium as well.
- inkyoto 4y ago
  > […] "the compiler has insufficient information to achieve ideal optimization because some things can only be known at runtime." This is where the profile guided optimisation comes in – for statically compiled languages, with a caveat being not always straightforward to come up with a set of inputs that will trigger an execution of all possible code paths. One solution is to provide the coverage specifically for the performance critical code paths and let the rest just be.
- funstuff007 4y ago
  > it can also be an argument for instrumenting your runtime environment Aren't JITs already self-instrumenting? What would you instrument that the JIT is not already keeping track of?
- masklinn 4y ago
  It's also an argument for having much more expressive and precise type systems, so the compiler has better information. Once you've managed to debug the codegen anyway (see: The Long and Arduous Story of Noalias).
  - mikepurvis 4y ago
    Is it? I'd love to see a breakdown of what classes of information can be gleaned from profile data, and how much of an impact each one has in isolation in terms of optimization. Naively, I would have assumed that branch information would be most valuable, in terms of being able to guide execution toward the hot path and maximize locality for the memory accesses occurring on the common branches. And that info is not something that would be assisted by more expressive types, I don't think.
- notriddle 4y ago
  https://tomaszs2.medium.com/how-rust-1-64-became-10-20-faster-on-windows-3a8bb5e81d70 https://tomaszs2.medium.com/how-rust-1-64-became-10-20-faste... https://news.ycombinator.com/item?id=33306945 https://news.ycombinator.com/item?id=33306945
titzer 4y ago
Dynamic languages need inline caches, type feedback, and fairly heavy inlining to be competitive. Some of that can be gotten offline, e.g. by doing PGO. But you can't, in general, adapt to a program that suddenly changes phases, or rebinds a global that was assumed a constant, etc. Speculative optimizations with deopt are what make dynamic languages fast.
hinkley 4y ago
Before I talked myself out of writing my own programming language, I used to have lunch conversations with my mentor who was also speed obsessed about how JIT could meet Knuth in the middle by creating a collections API with feedback guided optimization, using it for algorithm selection and tuning parameters by call site. For object graphs in Java you can waste exorbitant amounts of memory by having a lot of “children” members that are sized for a default of 10 entries but the normal case is 0-2. I once had to deoptimize code where someone tried to do this by hand and the number they picked was 6 (just over half of the default). So when the average jumped to 7, then the data structure ended up being 20% larger than the default behavior instead of 30% smaller as intended. For a server workflow, having data structured tuned to larger pools of objects with more complex comparison operations can also be valuable, but I don’t want that kitchen sink stuff on mobile or in an embedded app. I still think this is viable, but only if you are clever about gathering data. For instance the incremental increase in runtime for telemetry data is quite high on the happy path. But corner cases are already expensive, so telemetry adds only a few percent there instead of double digits. The nonstarter for this ended up being that most collections APIs violate Liskov, so you almost need to write your own language to pick a decomposition that doesn’t. Variance semantics help a ton but they don’t quite fix LSP.
- astrange 4y ago
  Swift and Objective-C have collections that change implementation as needed, but they can do it because they're well abstracted (enough) and usually immutable so there's less chance of making a bad assumption. Most other languages only have mutable collections, and name collection types after their implementation details instead of what the user actually wants from them.
- pjmlp 4y ago
  That is in a sense what JIT caching with PGO feedback loop does to a certain extent. As for mobile, this isn't far off from the whole set of hand written Assembly interpreter, JIT with PGO data caching, AOT compilation on idle with the PGO collected data, sharing PGO metadata with other devices via the PlayStore, that Android happens to do since version 7, and refined later.

saagarjha 4y ago

I take it you are not very familiar with the website known as Hacker News.